* [PATCH 1/5] separate shrink-wrap: New command-line flag, status flag, hooks, and doc
2016-09-23 8:22 [PATCH v3 0/5] Separate shrink-wrapping Segher Boessenkool
2016-09-23 8:22 ` [PATCH 2/5] dce: Don't dead-code delete separately wrapped restores Segher Boessenkool
@ 2016-09-23 8:22 ` Segher Boessenkool
2016-09-26 17:02 ` Jeff Law
2016-09-23 8:23 ` [PATCH 3/5] regrename: Don't rename restores Segher Boessenkool
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Segher Boessenkool @ 2016-09-23 8:22 UTC (permalink / raw)
To: gcc-patches; +Cc: law, dje.gcc, Segher Boessenkool
This patch adds a new command-line flag "-fshrink-wrap-separate", a status
flag "shrink_wrapped_separate", hooks for abstracting the target components,
and documentation for all those.
2016-09-23 Segher Boessenkool <segher@kernel.crashing.org>
* common.opt (-fshrink-wrap-separate): New flag.
* doc/invoke.texi: Document it.
* doc/tm.texi.in (Shrink-wrapping separate components): New subsection.
* doc/tm.texi: Regenerate.
* emit-rtl.h (struct rtl_data): New field shrink_wrapped_separate.
* target.def (shrink_wrap): New hook vector.
(get_separate_components, components_for_bb, disqualify_components,
emit_prologue_components, emit_epilogue_components,
set_handled_components): New hooks.
---
gcc/common.opt | 4 ++++
gcc/doc/invoke.texi | 11 +++++++++-
gcc/doc/tm.texi | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++
gcc/doc/tm.texi.in | 38 ++++++++++++++++++++++++++++++++
gcc/emit-rtl.h | 4 ++++
gcc/target.def | 57 ++++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 176 insertions(+), 1 deletion(-)
diff --git a/gcc/common.opt b/gcc/common.opt
index fa1c036..1a38d12 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -2189,6 +2189,10 @@ Common Report Var(flag_shrink_wrap) Optimization
Emit function prologues only before parts of the function that need it,
rather than at the top of the function.
+fshrink-wrap-separate
+Common Report Var(flag_shrink_wrap_separate) Init(1) Optimization
+Shrink-wrap parts of the prologue and epilogue separately.
+
fsignaling-nans
Common Report Var(flag_signaling_nans) Optimization SetByCombined
Disable optimizations observable by IEEE signaling NaNs.
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 8eb5eff..464b737 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -397,7 +397,8 @@ Objective-C and Objective-C++ Dialects}.
-fschedule-insns -fschedule-insns2 -fsection-anchors @gol
-fselective-scheduling -fselective-scheduling2 @gol
-fsel-sched-pipelining -fsel-sched-pipelining-outer-loops @gol
--fsemantic-interposition -fshrink-wrap -fsignaling-nans @gol
+-fsemantic-interposition -fshrink-wrap -fshrink-wrap-separate @gol
+-fsignaling-nans @gol
-fsingle-precision-constant -fsplit-ivs-in-unroller @gol
-fsplit-paths @gol
-fsplit-wide-types -fssa-backprop -fssa-phiopt @gol
@@ -6396,6 +6397,7 @@ compilation time.
-fmove-loop-invariants @gol
-freorder-blocks @gol
-fshrink-wrap @gol
+-fshrink-wrap-separate @gol
-fsplit-wide-types @gol
-fssa-backprop @gol
-fssa-phiopt @gol
@@ -7306,6 +7308,13 @@ Emit function prologues only before parts of the function that need it,
rather than at the top of the function. This flag is enabled by default at
@option{-O} and higher.
+@item -fshrink-wrap-separate
+@opindex fshrink-wrap-separate
+Shrink-wrap separate parts of the prologue and epilogue separately, so that
+those parts are only executed when needed.
+This option is on by default, but has no effect unless @option{-fshrink-wrap}
+is also turned on.
+
@item -fcaller-saves
@opindex fcaller-saves
Enable allocation of values to registers that are clobbered by
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index ced6274..c439695 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -2924,6 +2924,7 @@ This describes the stack layout and calling conventions.
* Function Entry::
* Profiling::
* Tail Calls::
+* Shrink-wrapping separate components::
* Stack Smashing Protection::
* Miscellaneous Register Hooks::
@end menu
@@ -4853,6 +4854,68 @@ This hook should add additional registers that are computed by the prologue to t
True if a function's return statements should be checked for matching the function's return type. This includes checking for falling off the end of a non-void function. Return false if no such check should be made.
@end deftypefn
+@node Shrink-wrapping separate components
+@subsection Shrink-wrapping separate components
+@cindex shrink-wrapping separate components
+
+The prologue may perform a variety of target dependent tasks such as
+saving callee-saved registers, saving the return address, aligning the
+stack, creating a stack frame, initializing the PIC register, setting
+up the static chain, etc.
+
+On some targets some of these tasks may be independent of others and
+thus may be shrink-wrapped separately. These independent tasks are
+referred to as components and are handled generically by the target
+independent parts of GCC.
+
+Using the following hooks those prologue or epilogue components can be
+shrink-wrapped separately, so that the initialization (and possibly
+teardown) those components do is not done as frequently on execution
+paths where this would unnecessary.
+
+What exactly those components are is up to the target code; the generic
+code treats them abstractly, as a bit in an @code{sbitmap}. These
+@code{sbitmap}s are allocated by the @code{shrink_wrap.get_separate_components}
+and @code{shrink_wrap.components_for_bb} hooks, and deallocated by the
+generic code.
+
+@deftypefn {Target Hook} sbitmap TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS (void)
+This hook should return an @code{sbitmap} with the bits set for those
+components that can be separately shrink-wrapped in the current function.
+Return @code{NULL} if the current function should not get any separate
+shrink-wrapping.
+Don't define this hook if it would always return @code{NULL}.
+If it is defined, the other hooks in this group have to be defined as well.
+@end deftypefn
+
+@deftypefn {Target Hook} sbitmap TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB (basic_block)
+This hook should return an @code{sbitmap} with the bits set for those
+components where either the prologue component has to be executed before
+the @code{basic_block}, or the epilogue component after it, or both.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS (sbitmap @var{components}, edge @var{e}, sbitmap @var{edge_components}, bool @var{is_prologue})
+This hook should clear the bits in the @var{components} bitmap for those
+components in @var{edge_components} that the target cannot handle on edge
+@var{e}, where @var{is_prologue} says if this is for a prologue or an
+epilogue instead.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS (sbitmap)
+Emit prologue insns for the components indicated by the parameter.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS (sbitmap)
+Emit epilogue insns for the components indicated by the parameter.
+@end deftypefn
+
+@deftypefn {Target Hook} void TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS (sbitmap)
+Mark the components in the parameter as handled, so that the
+@code{prologue} and @code{epilogue} named patterns know to ignore those
+components. The target code should not hang on to the @code{sbitmap}, it
+will be deleted after this call.
+@end deftypefn
+
@node Stack Smashing Protection
@subsection Stack smashing protection
@cindex stack smashing protection
diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in
index a5714d1..84164bf 100644
--- a/gcc/doc/tm.texi.in
+++ b/gcc/doc/tm.texi.in
@@ -2530,6 +2530,7 @@ This describes the stack layout and calling conventions.
* Function Entry::
* Profiling::
* Tail Calls::
+* Shrink-wrapping separate components::
* Stack Smashing Protection::
* Miscellaneous Register Hooks::
@end menu
@@ -3775,6 +3776,43 @@ the function prologue. Normally, the profiling code comes after.
@hook TARGET_WARN_FUNC_RETURN
+@node Shrink-wrapping separate components
+@subsection Shrink-wrapping separate components
+@cindex shrink-wrapping separate components
+
+The prologue may perform a variety of target dependent tasks such as
+saving callee-saved registers, saving the return address, aligning the
+stack, creating a stack frame, initializing the PIC register, setting
+up the static chain, etc.
+
+On some targets some of these tasks may be independent of others and
+thus may be shrink-wrapped separately. These independent tasks are
+referred to as components and are handled generically by the target
+independent parts of GCC.
+
+Using the following hooks those prologue or epilogue components can be
+shrink-wrapped separately, so that the initialization (and possibly
+teardown) those components do is not done as frequently on execution
+paths where this would unnecessary.
+
+What exactly those components are is up to the target code; the generic
+code treats them abstractly, as a bit in an @code{sbitmap}. These
+@code{sbitmap}s are allocated by the @code{shrink_wrap.get_separate_components}
+and @code{shrink_wrap.components_for_bb} hooks, and deallocated by the
+generic code.
+
+@hook TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS
+
+@hook TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB
+
+@hook TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS
+
+@hook TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS
+
+@hook TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS
+
+@hook TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS
+
@node Stack Smashing Protection
@subsection Stack smashing protection
@cindex stack smashing protection
diff --git a/gcc/emit-rtl.h b/gcc/emit-rtl.h
index 52c72b1..0a242b1 100644
--- a/gcc/emit-rtl.h
+++ b/gcc/emit-rtl.h
@@ -254,6 +254,10 @@ struct GTY(()) rtl_data {
/* True if we performed shrink-wrapping for the current function. */
bool shrink_wrapped;
+ /* True if we performed shrink-wrapping for separate components for
+ the current function. */
+ bool shrink_wrapped_separate;
+
/* Nonzero if function being compiled doesn't modify the stack pointer
(ignoring the prologue and epilogue). This is only valid after
pass_stack_ptr_mod has run. */
diff --git a/gcc/target.def b/gcc/target.def
index 0e10d6f..c357d65 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -5792,6 +5792,63 @@ DEFHOOK
bool, (tree),
hook_bool_tree_true)
+#undef HOOK_PREFIX
+#define HOOK_PREFIX "TARGET_SHRINK_WRAP_"
+HOOK_VECTOR (TARGET_SHRINK_WRAP_HOOKS, shrink_wrap)
+
+DEFHOOK
+(get_separate_components,
+ "This hook should return an @code{sbitmap} with the bits set for those\n\
+components that can be separately shrink-wrapped in the current function.\n\
+Return @code{NULL} if the current function should not get any separate\n\
+shrink-wrapping.\n\
+Don't define this hook if it would always return @code{NULL}.\n\
+If it is defined, the other hooks in this group have to be defined as well.",
+ sbitmap, (void),
+ NULL)
+
+DEFHOOK
+(components_for_bb,
+ "This hook should return an @code{sbitmap} with the bits set for those\n\
+components where either the prologue component has to be executed before\n\
+the @code{basic_block}, or the epilogue component after it, or both.",
+ sbitmap, (basic_block),
+ NULL)
+
+DEFHOOK
+(disqualify_components,
+ "This hook should clear the bits in the @var{components} bitmap for those\n\
+components in @var{edge_components} that the target cannot handle on edge\n\
+@var{e}, where @var{is_prologue} says if this is for a prologue or an\n\
+epilogue instead.",
+ void, (sbitmap components, edge e, sbitmap edge_components, bool is_prologue),
+ NULL)
+
+DEFHOOK
+(emit_prologue_components,
+ "Emit prologue insns for the components indicated by the parameter.",
+ void, (sbitmap),
+ NULL)
+
+DEFHOOK
+(emit_epilogue_components,
+ "Emit epilogue insns for the components indicated by the parameter.",
+ void, (sbitmap),
+ NULL)
+
+DEFHOOK
+(set_handled_components,
+ "Mark the components in the parameter as handled, so that the\n\
+@code{prologue} and @code{epilogue} named patterns know to ignore those\n\
+components. The target code should not hang on to the @code{sbitmap}, it\n\
+will be deleted after this call.",
+ void, (sbitmap),
+ NULL)
+
+HOOK_VECTOR_END (shrink_wrap)
+#undef HOOK_PREFIX
+#define HOOK_PREFIX "TARGET_"
+
/* Determine the type of unwind info to emit for debugging. */
DEFHOOK
(debug_unwind_info,
--
1.9.3
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH 4/5] shrink-wrap: Shrink-wrapping for separate components
2016-09-23 8:22 [PATCH v3 0/5] Separate shrink-wrapping Segher Boessenkool
` (2 preceding siblings ...)
2016-09-23 8:23 ` [PATCH 3/5] regrename: Don't rename restores Segher Boessenkool
@ 2016-09-23 8:33 ` Segher Boessenkool
2016-09-27 21:25 ` Jeff Law
2016-09-23 8:44 ` [PATCH 5/5] rs6000: Separate shrink-wrapping Segher Boessenkool
4 siblings, 1 reply; 18+ messages in thread
From: Segher Boessenkool @ 2016-09-23 8:33 UTC (permalink / raw)
To: gcc-patches; +Cc: law, dje.gcc, Segher Boessenkool
This is the main substance of this patch series.
Instead of doing all of the prologue and epilogue in one spot, it often
is better to do components of it at different places, so that they are
executed less frequently.
What exactly is a component is completely up to the target; this code
treats it all abstractly, and uses hooks for the target to handle the
more concrete things. Commonly there is one component for each callee-
saved register, for example.
Components can be executed more than once per function execution. This
pass makes sure that a component's epilogue is not called more often
than the corresponding prologue has been, at any point in time; that the
prologue is called more often, wherever the prologue's effect is needed;
and that the epilogue is called as often as the prologue has been, when
the function exits. It does this by first deciding which blocks need
which components "active", and then placing prologue and epilogue
components to make that exactly true.
Deciding what blocks should run with a certain component active so that
the total cost of executing the prologues (and epilogues) is optimal, is
not a computationally feasible problem. Instead, for each basic block,
we estimate the cost of putting a prologue right before the block, and
if that is cheaper than the total cost of putting prologues optimally
(according to the estimated cost) in the dominator subtrees strictly
dominated by this first block, place it at the first block instead.
This simple procedure places the components optimally for any dominator
sub tree where the root node's cost does not depend on anything outside
its subtree.
The cost is the execution frequency of all edges into the block coming
from blocks that do not have this component active. The estimated cost
is the execution frequency of the block, minus the execution frequency
of any backedges (which by definition are coming from subtrees, so if
the "head" block gets a prologue, the source block of any backedge has
that component active as well).
Currently, the epilogues are placed as late as possible, given the
constraints. This does not matter for execution cost, but we could
save a little bit of code size by placing the epilogues in a smarter
way. This is a possible future optimisation.
Now all that is left is inserting prologues and epilogues on all edges
that jump into resp. out of the "active" set of blocks. Often we need
to insert some components' prologues (or epilogues) on all edges into
(or out of) a block. In theory cross-jumping can unify all such, but
in practice that often fails; besides, that is a lot of work. So in
this case we insert the prologue and epilogue components at the "head"
or "tail" of a block, instead.
As a final optimisation, if a block needs a prologue and its immediate
dominator has the block as a post-dominator, that immediate dominator
gets the prologue as well.
2016-09-23 Segher Boessenkool <segher@kernel.crashing.org>
* function.c (thread_prologue_and_epilogue_insns): Recompute the
live info. Call try_shrink_wrapping_separate. Compute the
prologue_seq afterwards, if it has possibly changed. Compute the
split_prologue_seq and epilogue_seq later, too.
* shrink-wrap.c: #include cfgbuild.h.
(dump_components): New function.
(struct sw): New struct.
(SW): New function.
(init_separate_shrink_wrap): New function.
(fini_separate_shrink_wrap): New function.
(place_prologue_for_one_component): New function.
(spread_components): New function.
(disqualify_problematic_components): New function.
(emit_common_heads_for_components): New function.
(emit_common_tails_for_components): New function.
(insert_prologue_epilogue_for_components): New function.
(try_shrink_wrapping_separate): New function.
* shrink-wrap.h: Declare try_shrink_wrapping_separate.
---
gcc/function.c | 9 +-
gcc/shrink-wrap.c | 729 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
gcc/shrink-wrap.h | 1 +
3 files changed, 737 insertions(+), 2 deletions(-)
diff --git a/gcc/function.c b/gcc/function.c
index 53bad87..79e7b4f 100644
--- a/gcc/function.c
+++ b/gcc/function.c
@@ -5920,9 +5920,7 @@ thread_prologue_and_epilogue_insns (void)
edge entry_edge = single_succ_edge (ENTRY_BLOCK_PTR_FOR_FN (cfun));
edge orig_entry_edge = entry_edge;
- rtx_insn *split_prologue_seq = make_split_prologue_seq ();
rtx_insn *prologue_seq = make_prologue_seq ();
- rtx_insn *epilogue_seq = make_epilogue_seq ();
/* Try to perform a kind of shrink-wrapping, making sure the
prologue/epilogue is emitted only around those parts of the
@@ -5930,6 +5928,13 @@ thread_prologue_and_epilogue_insns (void)
try_shrink_wrapping (&entry_edge, prologue_seq);
+ try_shrink_wrapping_separate (entry_edge->dest);
+
+ if (crtl->shrink_wrapped_separate)
+ prologue_seq = make_prologue_seq ();
+
+ rtx_insn *split_prologue_seq = make_split_prologue_seq ();
+ rtx_insn *epilogue_seq = make_epilogue_seq ();
rtl_profile_for_bb (EXIT_BLOCK_PTR_FOR_FN (cfun));
diff --git a/gcc/shrink-wrap.c b/gcc/shrink-wrap.c
index b85b1c3..0dced22 100644
--- a/gcc/shrink-wrap.c
+++ b/gcc/shrink-wrap.c
@@ -34,6 +34,7 @@ along with GCC; see the file COPYING3. If not see
#include "output.h"
#include "tree-pass.h"
#include "cfgrtl.h"
+#include "cfgbuild.h"
#include "params.h"
#include "bb-reorder.h"
#include "shrink-wrap.h"
@@ -1006,3 +1007,731 @@ try_shrink_wrapping (edge *entry_edge, rtx_insn *prologue_seq)
BITMAP_FREE (bb_with);
free_dominance_info (CDI_DOMINATORS);
}
+\f
+/* Separate shrink-wrapping
+
+ Instead of putting all of the prologue and epilogue in one spot, we
+ can put parts of it in places where those components are executed less
+ frequently. The following code does this, for prologue and epilogue
+ components that can be put in more than one location, and where those
+ components can be executed more than once (the epilogue component will
+ always be executed before the prologue component is executed a second
+ time).
+
+ What exactly is a component is target-dependent. The more usual
+ components are simple saves/restores to/from the frame of callee-saved
+ registers. This code treats components abstractly (as an sbitmap),
+ letting the target handle all details.
+
+ Prologue components are placed in such a way that for every component
+ the prologue is executed as infrequently as possible. We do this by
+ walking the dominator tree, comparing the cost of placing a prologue
+ component before a block to the sum of costs determined for all subtrees
+ of that block.
+
+ From this placement, we then determine for each component all blocks
+ where at least one of this block's dominators (including itself) will
+ get a prologue inserted. That then is how the components are placed.
+ We could place the epilogue components a bit smarter (we can save a
+ bit of code size sometimes); this is a possible future improvement.
+
+ Prologues and epilogues are preferably placed into a block, either at
+ the beginning or end of it, if it is needed for all predecessor resp.
+ successor edges; or placed on the edge otherwise.
+
+ If the placement of any prologue/epilogue leads to a situation we cannot
+ handle (for example, an abnormal edge would need to be split, or some
+ targets want to use some specific registers that may not be available
+ where we want to put them), separate shrink-wrapping for the components
+ in that prologue/epilogue is aborted. */
+
+
+/* Print the sbitmap COMPONENTS to the DUMP_FILE if not empty, with the
+ label LABEL. */
+static void
+dump_components (const char *label, sbitmap components)
+{
+ if (bitmap_empty_p (components))
+ return;
+
+ fprintf (dump_file, " [%s", label);
+
+ for (unsigned int j = 0; j < components->n_bits; j++)
+ if (bitmap_bit_p (components, j))
+ fprintf (dump_file, " %u", j);
+
+ fprintf (dump_file, "]");
+}
+
+/* The data we collect for each bb. */
+struct sw {
+ /* What components does this BB need? */
+ sbitmap needs_components;
+
+ /* What components does this BB have? This is the main decision this
+ pass makes. */
+ sbitmap has_components;
+
+ /* The components for which we placed code at the start of the BB (instead
+ of on all incoming edges). */
+ sbitmap head_components;
+
+ /* The components for which we placed code at the end of the BB (instead
+ of on all outgoing edges). */
+ sbitmap tail_components;
+
+ /* The frequency of executing the prologue for this BB, if a prologue is
+ placed on this BB. This is a pessimistic estimate (no prologue is
+ needed for edges from blocks that have the component under consideration
+ active already). */
+ gcov_type own_cost;
+
+ /* The frequency of executing the prologue for this BB and all BBs
+ dominated by it. */
+ gcov_type total_cost;
+};
+
+/* A helper function for accessing the pass-specific info. */
+static inline struct sw *
+SW (basic_block bb)
+{
+ gcc_assert (bb->aux);
+ return (struct sw *) bb->aux;
+}
+
+/* Create the pass-specific data structures for separately shrink-wrapping
+ with components COMPONENTS. */
+static void
+init_separate_shrink_wrap (sbitmap components)
+{
+ basic_block bb;
+ FOR_ALL_BB_FN (bb, cfun)
+ {
+ bb->aux = xcalloc (1, sizeof (struct sw));
+
+ SW (bb)->needs_components = targetm.shrink_wrap.components_for_bb (bb);
+
+ /* Mark all basic blocks without successor as needing all components.
+ This avoids problems in at least cfgcleanup, sel-sched, and
+ regrename (largely to do with all paths to such a block still
+ needing the same dwarf CFI info). */
+ if (EDGE_COUNT (bb->succs) == 0)
+ bitmap_copy (SW (bb)->needs_components, components);
+
+ if (dump_file)
+ {
+ fprintf (dump_file, "bb %d components:", bb->index);
+ dump_components ("has", SW (bb)->needs_components);
+ fprintf (dump_file, "\n");
+ }
+
+ SW (bb)->has_components = sbitmap_alloc (SBITMAP_SIZE (components));
+ SW (bb)->head_components = sbitmap_alloc (SBITMAP_SIZE (components));
+ SW (bb)->tail_components = sbitmap_alloc (SBITMAP_SIZE (components));
+ bitmap_clear (SW (bb)->has_components);
+ bitmap_clear (SW (bb)->head_components);
+ bitmap_clear (SW (bb)->tail_components);
+ }
+}
+
+/* Destroy the pass-specific data. */
+static void
+fini_separate_shrink_wrap (void)
+{
+ basic_block bb;
+ FOR_ALL_BB_FN (bb, cfun)
+ if (bb->aux)
+ {
+ sbitmap_free (SW (bb)->needs_components);
+ sbitmap_free (SW (bb)->has_components);
+ sbitmap_free (SW (bb)->head_components);
+ sbitmap_free (SW (bb)->tail_components);
+ free (bb->aux);
+ bb->aux = 0;
+ }
+}
+
+/* Place the prologue for component WHICH, in the basic blocks dominated
+ by HEAD. Do a DFS over the dominator tree, and set bit WHICH in the
+ HAS_COMPONENTS of a block if either the block has that bit set in
+ NEEDS_COMPONENTS, or it is cheaper to place the prologue here than in all
+ dominator subtrees separately. */
+static void
+place_prologue_for_one_component (unsigned int which, basic_block head)
+{
+ /* The block we are currently dealing with. */
+ basic_block bb = head;
+ /* Is this the first time we visit this block, i.e. have we just gone
+ down the tree. */
+ bool first_visit = true;
+
+ /* Walk the dominator tree, visit one block per iteration of this loop.
+ Each basic block is visited twice: once before visiting any children
+ of the block, and once after visiting all of them (leaf nodes are
+ visited only once). As an optimization, we do not visit subtrees
+ that can no longer influence the prologue placement. */
+ for (;;)
+ {
+ /* First visit of a block: set the (children) cost accumulator to zero;
+ if the block does not have the component itself, walk down. */
+ if (first_visit)
+ {
+ /* Initialize the cost. The cost is the block execution frequency
+ that does not come from backedges. Calculating this by simply
+ adding the cost of all edges that aren't backedges does not
+ work: this does not always add up to the block frequency at
+ all, and even if it does, rounding error makes for bad
+ decisions. */
+ SW (bb)->own_cost = bb->frequency;
+
+ edge e;
+ edge_iterator ei;
+ FOR_EACH_EDGE (e, ei, bb->preds)
+ if (dominated_by_p (CDI_DOMINATORS, e->src, bb))
+ {
+ if (SW (bb)->own_cost > EDGE_FREQUENCY (e))
+ SW (bb)->own_cost -= EDGE_FREQUENCY (e);
+ else
+ SW (bb)->own_cost = 0;
+ }
+
+ SW (bb)->total_cost = 0;
+
+ if (!bitmap_bit_p (SW (bb)->needs_components, which)
+ && first_dom_son (CDI_DOMINATORS, bb))
+ {
+ bb = first_dom_son (CDI_DOMINATORS, bb);
+ continue;
+ }
+ }
+
+ /* If this block does need the component itself, or it is cheaper to
+ put the prologue here than in all the descendants that need it,
+ mark it so. If this block's immediate post-dominator is dominated
+ by this block, and that needs the prologue, we can put it on this
+ block as well (earlier is better). */
+ if (bitmap_bit_p (SW (bb)->needs_components, which)
+ || SW (bb)->total_cost > SW (bb)->own_cost)
+ {
+ SW (bb)->total_cost = SW (bb)->own_cost;
+ bitmap_set_bit (SW (bb)->has_components, which);
+ }
+ else
+ {
+ basic_block kid = get_immediate_dominator (CDI_POST_DOMINATORS, bb);
+ if (dominated_by_p (CDI_DOMINATORS, kid, bb)
+ && bitmap_bit_p (SW (kid)->has_components, which))
+ {
+ SW (bb)->total_cost = SW (bb)->own_cost;
+ bitmap_set_bit (SW (bb)->has_components, which);
+ }
+ }
+
+ /* We are back where we started, so we are done now. */
+ if (bb == head)
+ return;
+
+ /* We now know the cost of the subtree rooted at the current block.
+ Accumulate this cost in the parent. */
+ basic_block parent = get_immediate_dominator (CDI_DOMINATORS, bb);
+ SW (parent)->total_cost += SW (bb)->total_cost;
+
+ /* Don't walk the tree down unless necessary. */
+ if (next_dom_son (CDI_DOMINATORS, bb)
+ && SW (parent)->total_cost <= SW (parent)->own_cost)
+ {
+ bb = next_dom_son (CDI_DOMINATORS, bb);
+ first_visit = true;
+ }
+ else
+ {
+ bb = parent;
+ first_visit = false;
+ }
+ }
+}
+
+/* Mark HAS_COMPONENTS for every block dominated by at least one block with
+ PRO_COMPONENTS set for the respective components, starting at HEAD. */
+static void
+spread_components (basic_block head)
+{
+ basic_block bb = head;
+ bool first_visit = true;
+ /* This keeps a tally of all components active. */
+ sbitmap components = SW (head)->has_components;
+
+ for (;;)
+ {
+ if (first_visit)
+ {
+ bitmap_ior (SW (bb)->has_components, SW (bb)->has_components,
+ components);
+
+ if (first_dom_son (CDI_DOMINATORS, bb))
+ {
+ components = SW (bb)->has_components;
+ bb = first_dom_son (CDI_DOMINATORS, bb);
+ continue;
+ }
+ }
+
+ components = SW (bb)->has_components;
+
+ if (next_dom_son (CDI_DOMINATORS, bb))
+ {
+ bb = next_dom_son (CDI_DOMINATORS, bb);
+ basic_block parent = get_immediate_dominator (CDI_DOMINATORS, bb);
+ components = SW (parent)->has_components;
+ first_visit = true;
+ }
+ else
+ {
+ if (bb == head)
+ return;
+ bb = get_immediate_dominator (CDI_DOMINATORS, bb);
+ first_visit = false;
+ }
+ }
+}
+
+/* If we cannot handle placing some component's prologues or epilogues where
+ we decided we should place them, unmark that component in COMPONENTS so
+ that it is not wrapped separately. */
+static void
+disqualify_problematic_components (sbitmap components)
+{
+ sbitmap pro = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap epi = sbitmap_alloc (SBITMAP_SIZE (components));
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ edge e;
+ edge_iterator ei;
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ {
+ /* Find which components we want pro/epilogues for here. */
+ bitmap_and_compl (epi, SW (e->src)->has_components,
+ SW (e->dest)->has_components);
+ bitmap_and_compl (pro, SW (e->dest)->has_components,
+ SW (e->src)->has_components);
+
+ /* Ask the target what it thinks about things. */
+ if (!bitmap_empty_p (epi))
+ targetm.shrink_wrap.disqualify_components (components, e, epi,
+ false);
+ if (!bitmap_empty_p (pro))
+ targetm.shrink_wrap.disqualify_components (components, e, pro,
+ true);
+
+ /* If this edge doesn't need splitting, we're fine. */
+ if (single_pred_p (e->dest)
+ && e->dest != EXIT_BLOCK_PTR_FOR_FN (cfun))
+ continue;
+
+ /* If the edge can be split, that is fine too. */
+ if ((e->flags & EDGE_ABNORMAL) == 0)
+ continue;
+
+ /* We also can handle sibcalls. */
+ if (e->dest == EXIT_BLOCK_PTR_FOR_FN (cfun))
+ {
+ gcc_assert (e->flags & EDGE_SIBCALL);
+ continue;
+ }
+
+ /* Remove from consideration those components we would need
+ pro/epilogues for on edges where we cannot insert them. */
+ bitmap_and_compl (components, components, epi);
+ bitmap_and_compl (components, components, pro);
+
+ if (dump_file && !bitmap_subset_p (epi, components))
+ {
+ fprintf (dump_file, " BAD epi %d->%d", e->src->index,
+ e->dest->index);
+ if (e->flags & EDGE_EH)
+ fprintf (dump_file, " for EH");
+ dump_components ("epi", epi);
+ fprintf (dump_file, "\n");
+ }
+
+ if (dump_file && !bitmap_subset_p (pro, components))
+ {
+ fprintf (dump_file, " BAD pro %d->%d", e->src->index,
+ e->dest->index);
+ if (e->flags & EDGE_EH)
+ fprintf (dump_file, " for EH");
+ dump_components ("pro", pro);
+ fprintf (dump_file, "\n");
+ }
+ }
+ }
+
+ sbitmap_free (pro);
+ sbitmap_free (epi);
+}
+
+/* Place code for prologues and epilogues for COMPONENTS where we can put
+ that code at the start of basic blocks. */
+static void
+emit_common_heads_for_components (sbitmap components)
+{
+ sbitmap pro = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap epi = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap tmp = sbitmap_alloc (SBITMAP_SIZE (components));
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ /* Find which prologue resp. epilogue components are needed for all
+ predecessor edges to this block. */
+
+ /* First, select all possible components. */
+ bitmap_copy (epi, components);
+ bitmap_copy (pro, components);
+
+ edge e;
+ edge_iterator ei;
+ FOR_EACH_EDGE (e, ei, bb->preds)
+ {
+ if (e->flags & EDGE_ABNORMAL)
+ {
+ bitmap_clear (epi);
+ bitmap_clear (pro);
+ break;
+ }
+
+ /* Deselect those epilogue components that should not be inserted
+ for this edge. */
+ bitmap_and_compl (tmp, SW (e->src)->has_components,
+ SW (e->dest)->has_components);
+ bitmap_and (epi, epi, tmp);
+
+ /* Similar, for the prologue. */
+ bitmap_and_compl (tmp, SW (e->dest)->has_components,
+ SW (e->src)->has_components);
+ bitmap_and (pro, pro, tmp);
+ }
+
+ if (dump_file && !(bitmap_empty_p (epi) && bitmap_empty_p (pro)))
+ fprintf (dump_file, " bb %d", bb->index);
+
+ if (dump_file && !bitmap_empty_p (epi))
+ dump_components ("epi", epi);
+ if (dump_file && !bitmap_empty_p (pro))
+ dump_components ("pro", pro);
+
+ if (dump_file && !(bitmap_empty_p (epi) && bitmap_empty_p (pro)))
+ fprintf (dump_file, "\n");
+
+ /* Place code after the BB note. */
+ if (!bitmap_empty_p (pro))
+ {
+ start_sequence ();
+ targetm.shrink_wrap.emit_prologue_components (pro);
+ rtx_insn *seq = get_insns ();
+ end_sequence ();
+
+ emit_insn_after (seq, bb_note (bb));
+
+ bitmap_ior (SW (bb)->head_components, SW (bb)->head_components, pro);
+ }
+
+ if (!bitmap_empty_p (epi))
+ {
+ start_sequence ();
+ targetm.shrink_wrap.emit_epilogue_components (epi);
+ rtx_insn *seq = get_insns ();
+ end_sequence ();
+
+ emit_insn_after (seq, bb_note (bb));
+
+ bitmap_ior (SW (bb)->head_components, SW (bb)->head_components, epi);
+ }
+ }
+
+ sbitmap_free (pro);
+ sbitmap_free (epi);
+ sbitmap_free (tmp);
+}
+
+/* Place code for prologues and epilogues for COMPONENTS where we can put
+ that code at the end of basic blocks. */
+static void
+emit_common_tails_for_components (sbitmap components)
+{
+ sbitmap pro = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap epi = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap tmp = sbitmap_alloc (SBITMAP_SIZE (components));
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ /* Find which prologue resp. epilogue components are needed for all
+ successor edges from this block. */
+ if (EDGE_COUNT (bb->succs) == 0)
+ continue;
+
+ /* First, select all possible components. */
+ bitmap_copy (epi, components);
+ bitmap_copy (pro, components);
+
+ edge e;
+ edge_iterator ei;
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ {
+ if (e->flags & EDGE_ABNORMAL)
+ {
+ bitmap_clear (epi);
+ bitmap_clear (pro);
+ break;
+ }
+
+ /* Deselect those epilogue components that should not be inserted
+ for this edge, and also those that are already put at the head
+ of the successor block. */
+ bitmap_and_compl (tmp, SW (e->src)->has_components,
+ SW (e->dest)->has_components);
+ bitmap_and_compl (tmp, tmp, SW (e->dest)->head_components);
+ bitmap_and (epi, epi, tmp);
+
+ /* Similarly, for the prologue. */
+ bitmap_and_compl (tmp, SW (e->dest)->has_components,
+ SW (e->src)->has_components);
+ bitmap_and_compl (tmp, tmp, SW (e->dest)->head_components);
+ bitmap_and (pro, pro, tmp);
+ }
+
+ if (dump_file && !(bitmap_empty_p (epi) && bitmap_empty_p (pro)))
+ fprintf (dump_file, " bb %d", bb->index);
+
+ if (dump_file && !bitmap_empty_p (epi))
+ dump_components ("epi", epi);
+ if (dump_file && !bitmap_empty_p (pro))
+ dump_components ("pro", pro);
+
+ if (dump_file && !(bitmap_empty_p (epi) && bitmap_empty_p (pro)))
+ fprintf (dump_file, "\n");
+
+ /* Put the code at the end of the BB, but before any final jump. */
+ if (!bitmap_empty_p (epi))
+ {
+ start_sequence ();
+ targetm.shrink_wrap.emit_epilogue_components (epi);
+ rtx_insn *seq = get_insns ();
+ end_sequence ();
+
+ rtx_insn *insn = BB_END (bb);
+ if (control_flow_insn_p (insn))
+ emit_insn_before (seq, insn);
+ else
+ emit_insn_after (seq, insn);
+
+ bitmap_ior (SW (bb)->tail_components, SW (bb)->tail_components, epi);
+ }
+
+ if (!bitmap_empty_p (pro))
+ {
+ start_sequence ();
+ targetm.shrink_wrap.emit_prologue_components (pro);
+ rtx_insn *seq = get_insns ();
+ end_sequence ();
+
+ rtx_insn *insn = BB_END (bb);
+ if (JUMP_P (insn) || (CALL_P (insn) && SIBLING_CALL_P (insn)))
+ emit_insn_before (seq, insn);
+ else
+ emit_insn_after (seq, insn);
+
+ bitmap_ior (SW (bb)->tail_components, SW (bb)->tail_components, pro);
+ }
+ }
+
+ sbitmap_free (pro);
+ sbitmap_free (epi);
+ sbitmap_free (tmp);
+}
+
+/* Place prologues and epilogues for COMPONENTS on edges, if we haven't already
+ placed them inside blocks directly. */
+static void
+insert_prologue_epilogue_for_components (sbitmap components)
+{
+ sbitmap pro = sbitmap_alloc (SBITMAP_SIZE (components));
+ sbitmap epi = sbitmap_alloc (SBITMAP_SIZE (components));
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ {
+ if (!bb->aux)
+ continue;
+
+ edge e;
+ edge_iterator ei;
+ FOR_EACH_EDGE (e, ei, bb->succs)
+ {
+ /* Find which pro/epilogue components are needed on this edge. */
+ bitmap_and_compl (epi, SW (e->src)->has_components,
+ SW (e->dest)->has_components);
+ bitmap_and_compl (pro, SW (e->dest)->has_components,
+ SW (e->src)->has_components);
+ bitmap_and (epi, epi, components);
+ bitmap_and (pro, pro, components);
+
+ /* Deselect those we already have put at the head or tail of the
+ edge's dest resp. src. */
+ bitmap_and_compl (epi, epi, SW (e->dest)->head_components);
+ bitmap_and_compl (pro, pro, SW (e->dest)->head_components);
+ bitmap_and_compl (epi, epi, SW (e->src)->tail_components);
+ bitmap_and_compl (pro, pro, SW (e->src)->tail_components);
+
+ if (!bitmap_empty_p (epi) || !bitmap_empty_p (pro))
+ {
+ if (dump_file)
+ {
+ fprintf (dump_file, " %d->%d", e->src->index,
+ e->dest->index);
+ dump_components ("epi", epi);
+ dump_components ("pro", pro);
+ fprintf (dump_file, "\n");
+ }
+
+ /* Put the epilogue components in place. */
+ start_sequence ();
+ targetm.shrink_wrap.emit_epilogue_components (epi);
+ rtx_insn *seq = get_insns ();
+ end_sequence ();
+
+ if (e->flags & EDGE_SIBCALL)
+ {
+ gcc_assert (e->dest == EXIT_BLOCK_PTR_FOR_FN (cfun));
+
+ rtx_insn *insn = BB_END (e->src);
+ gcc_assert (CALL_P (insn) && SIBLING_CALL_P (insn));
+ emit_insn_before (seq, insn);
+ }
+ else if (e->dest == EXIT_BLOCK_PTR_FOR_FN (cfun))
+ {
+ gcc_assert (e->flags & EDGE_FALLTHRU);
+ basic_block new_bb = split_edge (e);
+ emit_insn_after (seq, BB_END (new_bb));
+ }
+ else
+ insert_insn_on_edge (seq, e);
+
+ /* Put the prologue components in place. */
+ start_sequence ();
+ targetm.shrink_wrap.emit_prologue_components (pro);
+ seq = get_insns ();
+ end_sequence ();
+
+ insert_insn_on_edge (seq, e);
+ }
+ }
+ }
+
+ sbitmap_free (pro);
+ sbitmap_free (epi);
+
+ commit_edge_insertions ();
+}
+
+/* The main entry point to this subpass. FIRST_BB is where the prologue
+ would be normally put. */
+void
+try_shrink_wrapping_separate (basic_block first_bb)
+{
+ if (!(SHRINK_WRAPPING_ENABLED
+ && flag_shrink_wrap_separate
+ && optimize_function_for_speed_p (cfun)
+ && targetm.shrink_wrap.get_separate_components))
+ return;
+
+ /* We don't handle "strange" functions. */
+ if (cfun->calls_alloca
+ || cfun->calls_setjmp
+ || cfun->can_throw_non_call_exceptions
+ || crtl->calls_eh_return
+ || crtl->has_nonlocal_goto
+ || crtl->saves_all_registers)
+ return;
+
+ /* Ask the target what components there are. If it returns NULL, don't
+ do anything. */
+ sbitmap components = targetm.shrink_wrap.get_separate_components ();
+ if (!components)
+ return;
+
+ /* We need LIVE info. */
+ df_live_add_problem ();
+ df_live_set_all_dirty ();
+ df_analyze ();
+
+ calculate_dominance_info (CDI_DOMINATORS);
+ calculate_dominance_info (CDI_POST_DOMINATORS);
+
+ init_separate_shrink_wrap (components);
+
+ sbitmap_iterator sbi;
+ unsigned int j;
+ EXECUTE_IF_SET_IN_BITMAP (components, 0, j, sbi)
+ place_prologue_for_one_component (j, first_bb);
+
+ spread_components (first_bb);
+
+ disqualify_problematic_components (components);
+
+ /* Don't separately shrink-wrap anything where the "main" prologue will
+ go; the target code can often optimize things if it is presented with
+ all components together (say, if it generates store-multiple insns). */
+ bitmap_and_compl (components, components, SW (first_bb)->has_components);
+
+ if (bitmap_empty_p (components))
+ {
+ if (dump_file)
+ fprintf (dump_file, "Not wrapping anything separately.\n");
+ }
+ else
+ {
+ if (dump_file)
+ {
+ fprintf (dump_file, "The components we wrap separately are");
+ dump_components ("sep", components);
+ fprintf (dump_file, "\n");
+
+ fprintf (dump_file, "... Inserting common heads...\n");
+ }
+
+ emit_common_heads_for_components (components);
+
+ if (dump_file)
+ fprintf (dump_file, "... Inserting common tails...\n");
+
+ emit_common_tails_for_components (components);
+
+ if (dump_file)
+ fprintf (dump_file, "... Inserting the more difficult ones...\n");
+
+ insert_prologue_epilogue_for_components (components);
+
+ if (dump_file)
+ fprintf (dump_file, "... Done.\n");
+
+ targetm.shrink_wrap.set_handled_components (components);
+
+ crtl->shrink_wrapped_separate = true;
+ }
+
+ fini_separate_shrink_wrap ();
+
+ sbitmap_free (components);
+ free_dominance_info (CDI_DOMINATORS);
+ free_dominance_info (CDI_POST_DOMINATORS);
+
+ if (crtl->shrink_wrapped_separate)
+ {
+ df_live_set_all_dirty ();
+ df_analyze ();
+ }
+}
diff --git a/gcc/shrink-wrap.h b/gcc/shrink-wrap.h
index e06ab37..05fcb41 100644
--- a/gcc/shrink-wrap.h
+++ b/gcc/shrink-wrap.h
@@ -25,6 +25,7 @@ along with GCC; see the file COPYING3. If not see
/* In shrink-wrap.c. */
extern bool requires_stack_frame_p (rtx_insn *, HARD_REG_SET, HARD_REG_SET);
extern void try_shrink_wrapping (edge *entry_edge, rtx_insn *prologue_seq);
+extern void try_shrink_wrapping_separate (basic_block first_bb);
#define SHRINK_WRAPPING_ENABLED \
(flag_shrink_wrap && targetm.have_simple_return ())
--
1.9.3
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH 5/5] rs6000: Separate shrink-wrapping
2016-09-23 8:22 [PATCH v3 0/5] Separate shrink-wrapping Segher Boessenkool
` (3 preceding siblings ...)
2016-09-23 8:33 ` [PATCH 4/5] shrink-wrap: Shrink-wrapping for separate components Segher Boessenkool
@ 2016-09-23 8:44 ` Segher Boessenkool
2016-09-26 16:39 ` Jeff Law
2016-09-26 18:16 ` David Edelsohn
4 siblings, 2 replies; 18+ messages in thread
From: Segher Boessenkool @ 2016-09-23 8:44 UTC (permalink / raw)
To: gcc-patches; +Cc: law, dje.gcc, Segher Boessenkool
This implements the hooks for separate shrink-wrapping for rs6000.
It handles GPRs and LR. The GPRs get a component number corresponding
to their register number; LR gets component number 0.
2016-09-23 Segher Boessenkool <segher@kernel.crashing.org>
* config/rs6000/rs6000.c (machine_function): Add new fields
gpr_is_wrapped_separately and lr_is_wrapped_separately.
(TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS,
TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB,
TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS,
TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS,
TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS,
TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS): Define.
(rs6000_get_separate_components): New function.
(rs6000_components_for_bb): New function.
(rs6000_disqualify_components): New function.
(rs6000_emit_prologue_components): New function.
(rs6000_emit_epilogue_components): New function.
(rs6000_set_handled_components): New function.
(rs6000_emit_prologue): Don't emit LR save if lr_is_wrapped_separately.
Don't emit GPR saves if gpr_is_wrapped_separately for that register.
(rs6000_emit_epilogue): Don't emit GPR restores if
gpr_is_wrapped_separately for that register. Don't make a
REG_CFA_RESTORE note for registers we did not restore, either.
---
gcc/config/rs6000/rs6000.c | 269 ++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 253 insertions(+), 16 deletions(-)
diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c
index 557009f..ec8d637 100644
--- a/gcc/config/rs6000/rs6000.c
+++ b/gcc/config/rs6000/rs6000.c
@@ -152,6 +152,10 @@ typedef struct GTY(()) machine_function
bool split_stack_argp_used;
/* Flag if r2 setup is needed with ELFv2 ABI. */
bool r2_setup_needed;
+ /* The components already handled by separate shrink-wrapping, which should
+ not be considered by the prologue and epilogue. */
+ bool gpr_is_wrapped_separately[32];
+ bool lr_is_wrapped_separately;
} machine_function;
/* Support targetm.vectorize.builtin_mask_for_load. */
@@ -1513,6 +1517,19 @@ static const struct attribute_spec rs6000_attribute_table[] =
#undef TARGET_SET_UP_BY_PROLOGUE
#define TARGET_SET_UP_BY_PROLOGUE rs6000_set_up_by_prologue
+#undef TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS
+#define TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS rs6000_get_separate_components
+#undef TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB
+#define TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB rs6000_components_for_bb
+#undef TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS
+#define TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS rs6000_disqualify_components
+#undef TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS
+#define TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS rs6000_emit_prologue_components
+#undef TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS
+#define TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS rs6000_emit_epilogue_components
+#undef TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS
+#define TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS rs6000_set_handled_components
+
#undef TARGET_EXTRA_LIVE_ON_ENTRY
#define TARGET_EXTRA_LIVE_ON_ENTRY rs6000_live_on_entry
@@ -26850,6 +26867,212 @@ rs6000_global_entry_point_needed_p (void)
return cfun->machine->r2_setup_needed;
}
+/* Implement TARGET_SHRINK_WRAP_GET_SEPARATE_COMPONENTS. */
+static sbitmap
+rs6000_get_separate_components (void)
+{
+ rs6000_stack_t *info = rs6000_stack_info ();
+
+ if (!(info->savres_strategy & SAVE_INLINE_GPRS)
+ || !(info->savres_strategy & REST_INLINE_GPRS)
+ || WORLD_SAVE_P (info))
+ return NULL;
+
+ sbitmap components = sbitmap_alloc (32);
+ bitmap_clear (components);
+
+ /* The GPRs we need saved to the frame. */
+ int reg_size = TARGET_32BIT ? 4 : 8;
+ int offset = info->gp_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+
+ for (unsigned regno = info->first_gp_reg_save; regno < 32; regno++)
+ {
+ if (IN_RANGE (offset, -0x8000, 0x7fff)
+ && rs6000_reg_live_or_pic_offset_p (regno))
+ bitmap_set_bit (components, regno);
+
+ offset += reg_size;
+ }
+
+ /* Don't mess with the hard frame pointer. */
+ if (frame_pointer_needed)
+ bitmap_clear_bit (components, HARD_FRAME_POINTER_REGNUM);
+
+ /* Don't mess with the fixed TOC register. */
+ if ((TARGET_TOC && TARGET_MINIMAL_TOC)
+ || (flag_pic == 1 && DEFAULT_ABI == ABI_V4)
+ || (flag_pic && DEFAULT_ABI == ABI_DARWIN))
+ bitmap_clear_bit (components, RS6000_PIC_OFFSET_TABLE_REGNUM);
+
+ /* Optimize LR save and restore if we can. This is component 0. */
+ if (info->lr_save_p
+ && !(flag_pic && (DEFAULT_ABI == ABI_V4 || DEFAULT_ABI == ABI_DARWIN)))
+ {
+ offset = info->lr_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+ if (IN_RANGE (offset, -0x8000, 0x7fff))
+ bitmap_set_bit (components, 0);
+ }
+
+ return components;
+}
+
+/* Implement TARGET_SHRINK_WRAP_COMPONENTS_FOR_BB. */
+static sbitmap
+rs6000_components_for_bb (basic_block bb)
+{
+ rs6000_stack_t *info = rs6000_stack_info ();
+
+ bitmap in = DF_LIVE_IN (bb);
+ bitmap gen = &DF_LIVE_BB_INFO (bb)->gen;
+ bitmap kill = &DF_LIVE_BB_INFO (bb)->kill;
+
+ sbitmap components = sbitmap_alloc (32);
+ bitmap_clear (components);
+
+ /* GPRs are used in a bb if they are in the IN, GEN, or KILL sets. */
+ for (unsigned regno = info->first_gp_reg_save; regno < 32; regno++)
+ if (bitmap_bit_p (in, regno)
+ || bitmap_bit_p (gen, regno)
+ || bitmap_bit_p (kill, regno))
+ bitmap_set_bit (components, regno);
+
+ /* LR needs to be saved around a bb if it is killed in that bb. */
+ if (bitmap_bit_p (gen, LR_REGNO)
+ || bitmap_bit_p (kill, LR_REGNO))
+ bitmap_set_bit (components, 0);
+
+ return components;
+}
+
+/* Implement TARGET_SHRINK_WRAP_DISQUALIFY_COMPONENTS. */
+static void
+rs6000_disqualify_components (sbitmap components, edge e,
+ sbitmap edge_components, bool /*is_prologue*/)
+{
+ /* Our LR pro/epilogue code moves LR via R0, so R0 had better not be
+ live where we want to place that code. */
+ if (bitmap_bit_p (edge_components, 0)
+ && bitmap_bit_p (DF_LIVE_IN (e->dest), 0))
+ {
+ if (dump_file)
+ fprintf (dump_file, "Disqualifying LR because GPR0 is live "
+ "on entry to bb %d\n", e->dest->index);
+ bitmap_clear_bit (components, 0);
+ }
+}
+
+/* Implement TARGET_SHRINK_WRAP_EMIT_PROLOGUE_COMPONENTS. */
+static void
+rs6000_emit_prologue_components (sbitmap components)
+{
+ rs6000_stack_t *info = rs6000_stack_info ();
+ rtx ptr_reg = gen_rtx_REG (Pmode, frame_pointer_needed
+ ? HARD_FRAME_POINTER_REGNUM
+ : STACK_POINTER_REGNUM);
+ int reg_size = TARGET_32BIT ? 4 : 8;
+
+ /* Prologue for LR. */
+ if (bitmap_bit_p (components, 0))
+ {
+ rtx reg = gen_rtx_REG (Pmode, 0);
+ rtx_insn *insn = emit_move_insn (reg, gen_rtx_REG (Pmode, LR_REGNO));
+ RTX_FRAME_RELATED_P (insn) = 1;
+ add_reg_note (insn, REG_CFA_REGISTER, NULL);
+
+ int offset = info->lr_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+
+ insn = emit_insn (gen_frame_store (reg, ptr_reg, offset));
+ RTX_FRAME_RELATED_P (insn) = 1;
+ rtx lr = gen_rtx_REG (Pmode, LR_REGNO);
+ rtx mem = copy_rtx (SET_DEST (single_set (insn)));
+ add_reg_note (insn, REG_CFA_OFFSET, gen_rtx_SET (mem, lr));
+ }
+
+ /* Prologue for the GPRs. */
+ int offset = info->gp_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+
+ for (int i = info->first_gp_reg_save; i < 32; i++)
+ {
+ if (bitmap_bit_p (components, i))
+ {
+ rtx reg = gen_rtx_REG (Pmode, i);
+ rtx_insn *insn = emit_insn (gen_frame_store (reg, ptr_reg, offset));
+ RTX_FRAME_RELATED_P (insn) = 1;
+ rtx set = copy_rtx (single_set (insn));
+ add_reg_note (insn, REG_CFA_OFFSET, set);
+ }
+
+ offset += reg_size;
+ }
+}
+
+/* Implement TARGET_SHRINK_WRAP_EMIT_EPILOGUE_COMPONENTS. */
+static void
+rs6000_emit_epilogue_components (sbitmap components)
+{
+ rs6000_stack_t *info = rs6000_stack_info ();
+ rtx ptr_reg = gen_rtx_REG (Pmode, frame_pointer_needed
+ ? HARD_FRAME_POINTER_REGNUM
+ : STACK_POINTER_REGNUM);
+ int reg_size = TARGET_32BIT ? 4 : 8;
+
+ /* Epilogue for the GPRs. */
+ int offset = info->gp_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+
+ for (int i = info->first_gp_reg_save; i < 32; i++)
+ {
+ if (bitmap_bit_p (components, i))
+ {
+ rtx reg = gen_rtx_REG (Pmode, i);
+ rtx_insn *insn = emit_insn (gen_frame_load (reg, ptr_reg, offset));
+ RTX_FRAME_RELATED_P (insn) = 1;
+ add_reg_note (insn, REG_CFA_RESTORE, reg);
+ }
+
+ offset += reg_size;
+ }
+
+ /* Epilogue for LR. */
+ if (bitmap_bit_p (components, 0))
+ {
+ int offset = info->lr_save_offset;
+ if (info->push_p)
+ offset += info->total_size;
+
+ rtx reg = gen_rtx_REG (Pmode, 0);
+ rtx_insn *insn = emit_insn (gen_frame_load (reg, ptr_reg, offset));
+
+ rtx lr = gen_rtx_REG (Pmode, LR_REGNO);
+ insn = emit_move_insn (lr, reg);
+ RTX_FRAME_RELATED_P (insn) = 1;
+ add_reg_note (insn, REG_CFA_RESTORE, lr);
+ }
+}
+
+/* Implement TARGET_SHRINK_WRAP_SET_HANDLED_COMPONENTS. */
+static void
+rs6000_set_handled_components (sbitmap components)
+{
+ rs6000_stack_t *info = rs6000_stack_info ();
+
+ for (int i = info->first_gp_reg_save; i < 32; i++)
+ if (bitmap_bit_p (components, i))
+ cfun->machine->gpr_is_wrapped_separately[i] = true;
+
+ if (bitmap_bit_p (components, 0))
+ cfun->machine->lr_is_wrapped_separately = true;
+}
+
/* Emit function prologue as insns. */
void
@@ -27107,7 +27330,8 @@ rs6000_emit_prologue (void)
}
/* If we use the link register, get it into r0. */
- if (!WORLD_SAVE_P (info) && info->lr_save_p)
+ if (!WORLD_SAVE_P (info) && info->lr_save_p
+ && !cfun->machine->lr_is_wrapped_separately)
{
rtx addr, reg, mem;
@@ -27335,13 +27559,16 @@ rs6000_emit_prologue (void)
}
else if (!WORLD_SAVE_P (info))
{
- int i;
- for (i = 0; i < 32 - info->first_gp_reg_save; i++)
- if (rs6000_reg_live_or_pic_offset_p (info->first_gp_reg_save + i))
- emit_frame_save (frame_reg_rtx, reg_mode,
- info->first_gp_reg_save + i,
- info->gp_save_offset + frame_off + reg_size * i,
- sp_off - frame_off);
+ int offset = info->gp_save_offset + frame_off;
+ for (int i = info->first_gp_reg_save; i < 32; i++)
+ {
+ if (rs6000_reg_live_or_pic_offset_p (i)
+ && !cfun->machine->gpr_is_wrapped_separately[i])
+ emit_frame_save (frame_reg_rtx, reg_mode, i, offset,
+ sp_off - frame_off);
+
+ offset += reg_size;
+ }
}
if (crtl->calls_eh_return)
@@ -28264,7 +28491,9 @@ rs6000_emit_epilogue (int sibcall)
&& (restoring_FPRs_inline
|| (strategy & REST_NOINLINE_FPRS_DOESNT_RESTORE_LR))
&& (restoring_GPRs_inline
- || info->first_fp_reg_save < 64));
+ || info->first_fp_reg_save < 64)
+ && !cfun->machine->lr_is_wrapped_separately);
+
if (WORLD_SAVE_P (info))
{
@@ -28899,12 +29128,18 @@ rs6000_emit_epilogue (int sibcall)
}
else
{
- for (i = 0; i < 32 - info->first_gp_reg_save; i++)
- if (rs6000_reg_live_or_pic_offset_p (info->first_gp_reg_save + i))
- emit_insn (gen_frame_load
- (gen_rtx_REG (reg_mode, info->first_gp_reg_save + i),
- frame_reg_rtx,
- info->gp_save_offset + frame_off + reg_size * i));
+ int offset = info->gp_save_offset + frame_off;
+ for (i = info->first_gp_reg_save; i < 32; i++)
+ {
+ if (rs6000_reg_live_or_pic_offset_p (i)
+ && !cfun->machine->gpr_is_wrapped_separately[i])
+ {
+ rtx reg = gen_rtx_REG (reg_mode, i);
+ emit_insn (gen_frame_load (reg, frame_reg_rtx, offset));
+ }
+
+ offset += reg_size;
+ }
}
if (DEFAULT_ABI == ABI_V4 || flag_shrink_wrap)
@@ -28943,8 +29178,10 @@ rs6000_emit_epilogue (int sibcall)
|| using_load_multiple
|| rs6000_reg_live_or_pic_offset_p (i))
{
- rtx reg = gen_rtx_REG (reg_mode, i);
+ if (cfun->machine->gpr_is_wrapped_separately[i])
+ continue;
+ rtx reg = gen_rtx_REG (reg_mode, i);
cfa_restores = alloc_reg_note (REG_CFA_RESTORE, reg, cfa_restores);
}
}
--
1.9.3
^ permalink raw reply [flat|nested] 18+ messages in thread