* [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA
@ 2024-04-24 10:05 Lehua Ding
2024-04-24 10:05 ` [PATCH 1/4] df: Add -ftrack-subreg-liveness option Lehua Ding
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Lehua Ding @ 2024-04-24 10:05 UTC (permalink / raw)
To: gcc-patches; +Cc: juzhe.zhong, shuo.chen, jin.xia, vmakarov, richard.sandiford
Hi Vladimir and Richard,
These patches are used to add a new data flow DF_LIVE_SUBREG,
which will track subreg liveness and then apply it to IRA and LRA
passes (enabled via -O3 or -ftrack-subreg-liveness). These patches
are for GCC 15. And these codes are pushed to the devel/subreg-coalesce
branch. In addition, my colleague Shuo Chen will also be involved in some
of the remain work, thank you for your support.
These patches are separated from the subreg-coalesce patches submitted
a few months ago. I refactored the code according to comments. The next
patches will support subreg coalesce base on they. Here are some data
abot build time of SPEC INT 2017 (x86-64 target):
baseline baseline(+track-subreg-liveness)
specint2017 build time : 1892s 1883s
Regarding build times, I've run it a few times, but they all seem to take
much less time. Since the difference is small, it's possible that it's just
a change in environment. But it's theoretically possible, since supporting
subreg-liveness could have reduced the number of living regs.
For memory usage, I trided PR 69609 by valgrind, peak memory size grow from
2003910656 to 2003947520, very small increase.
No regression on x86-64
Co-authored-by: Shuo Chen <shuo.chen@rivai.ai>
Best,
Lehua
Lehua Ding (4):
df: Add -ftrack-subreg-liveness option
df: Add DF_LIVE_SUBREG problem
ira: Apply DF_LIVE_SUBREG data
lra: Apply DF_LIVE_SUBREG data
gcc/Makefile.in | 1 +
gcc/common.opt | 4 +
gcc/common.opt.urls | 3 +
gcc/df-problems.cc | 855 ++++++++++++++++++++++++++++++++++++++-
gcc/df.h | 155 +++++++
gcc/doc/invoke.texi | 8 +
gcc/ira-build.cc | 7 +-
gcc/ira-color.cc | 8 +-
gcc/ira-emit.cc | 12 +-
gcc/ira-lives.cc | 7 +-
gcc/ira.cc | 19 +-
gcc/lra-coalesce.cc | 27 +-
gcc/lra-constraints.cc | 109 ++++-
gcc/lra-int.h | 4 +
gcc/lra-lives.cc | 357 ++++++++++++----
gcc/lra-remat.cc | 8 +-
gcc/lra-spills.cc | 27 +-
gcc/lra.cc | 10 +-
gcc/opts.cc | 1 +
gcc/regs.h | 5 +
gcc/sbitmap.cc | 98 +++++
gcc/sbitmap.h | 2 +
gcc/subreg-live-range.cc | 53 +++
gcc/subreg-live-range.h | 206 ++++++++++
gcc/timevar.def | 1 +
25 files changed, 1851 insertions(+), 136 deletions(-)
create mode 100644 gcc/subreg-live-range.cc
create mode 100644 gcc/subreg-live-range.h
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/4] df: Add -ftrack-subreg-liveness option
2024-04-24 10:05 [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Lehua Ding
@ 2024-04-24 10:05 ` Lehua Ding
2024-04-24 10:05 ` [PATCH 2/4] df: Add DF_LIVE_SUBREG problem Lehua Ding
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Lehua Ding @ 2024-04-24 10:05 UTC (permalink / raw)
To: gcc-patches; +Cc: juzhe.zhong, shuo.chen, jin.xia, vmakarov, richard.sandiford
Add new flag -ftrack-subreg-liveness to enable track-subreg-liveness.
This flag is enabled at -O3/fast.
gcc/ChangeLog:
* common.opt: add -ftrack-subreg-liveness option.
* opts.cc: auto aneble -ftrack-subreg-liveness in -O3/fast
---
gcc/common.opt | 4 ++++
gcc/common.opt.urls | 3 +++
gcc/doc/invoke.texi | 8 ++++++++
gcc/opts.cc | 1 +
4 files changed, 16 insertions(+)
diff --git a/gcc/common.opt b/gcc/common.opt
index ad348844775..bd030973434 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -2157,6 +2157,10 @@ fira-share-spill-slots
Common Var(flag_ira_share_spill_slots) Init(1) Optimization
Share stack slots for spilled pseudo-registers.
+ftrack-subreg-liveness
+Common Var(flag_track_subreg_liveness) Init(0) Optimization
+Track subreg liveness information.
+
fira-verbose=
Common RejectNegative Joined UInteger Var(flag_ira_verbose) Init(5)
-fira-verbose=<number> Control IRA's level of diagnostic messages.
diff --git a/gcc/common.opt.urls b/gcc/common.opt.urls
index f71ed80a34b..59f27a6f7c6 100644
--- a/gcc/common.opt.urls
+++ b/gcc/common.opt.urls
@@ -880,6 +880,9 @@ UrlSuffix(gcc/Optimize-Options.html#index-fira-share-save-slots)
fira-share-spill-slots
UrlSuffix(gcc/Optimize-Options.html#index-fira-share-spill-slots)
+ftrack-subreg-liveness
+UrlSuffix(gcc/Optimize-Options.html#index-ftrack-subreg-liveness)
+
fira-verbose=
UrlSuffix(gcc/Developer-Options.html#index-fira-verbose)
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 27c31ab0c86..9724cbb32ba 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -13186,6 +13186,14 @@ Disable sharing of stack slots allocated for pseudo-registers. Each
pseudo-register that does not get a hard register gets a separate
stack slot, and as a result function stack frames are larger.
+@opindex ftrack-subreg-liveness
+@item -ftrack-subreg-liveness
+Enable tracking subreg liveness information. This infomation allows IRA
+and LRA to support subreg coalesce feature which can improve the quality
+of register allocation.
+
+This option is enabled at level @option{-O3} for all targets.
+
@opindex flra-remat
@item -flra-remat
Enable CFG-sensitive rematerialization in LRA. Instead of loading
diff --git a/gcc/opts.cc b/gcc/opts.cc
index a90dc57f8b5..7b5d905a241 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -689,6 +689,7 @@ static const struct default_options default_options_table[] =
{ OPT_LEVELS_3_PLUS, OPT_funswitch_loops, NULL, 1 },
{ OPT_LEVELS_3_PLUS, OPT_fvect_cost_model_, NULL, VECT_COST_MODEL_DYNAMIC },
{ OPT_LEVELS_3_PLUS, OPT_fversion_loops_for_strides, NULL, 1 },
+ { OPT_LEVELS_3_PLUS, OPT_ftrack_subreg_liveness, NULL, 1 },
/* -O3 parameters. */
{ OPT_LEVELS_3_PLUS, OPT__param_max_inline_insns_auto_, NULL, 30 },
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/4] df: Add DF_LIVE_SUBREG problem
2024-04-24 10:05 [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Lehua Ding
2024-04-24 10:05 ` [PATCH 1/4] df: Add -ftrack-subreg-liveness option Lehua Ding
@ 2024-04-24 10:05 ` Lehua Ding
2024-04-25 20:56 ` Dimitar Dimitrov
2024-04-24 10:05 ` [PATCH 3/4] ira: Apply DF_LIVE_SUBREG data Lehua Ding
2024-04-24 10:05 ` [PATCH 4/4] lra: " Lehua Ding
3 siblings, 1 reply; 6+ messages in thread
From: Lehua Ding @ 2024-04-24 10:05 UTC (permalink / raw)
To: gcc-patches; +Cc: juzhe.zhong, shuo.chen, jin.xia, vmakarov, richard.sandiford
This patch add a new DF problem, named DF_LIVE_SUBREG. This problem
is extended from the DF_LR problem and support track the subreg liveness
of multireg pseudo if these pseudo satisfy the following conditions:
1. the mode size greater than it's REGMODE_NATURAL_SIZE.
2. the reg is used in insns via subreg pattern.
The main methods are as follows:
1. split bitmap in/out/def/use fileds to full_in/out/def/use and
partial_in/out/def/use. If a pseudo need to be tracked it's subreg
liveness, then it is recorded in partial_in/out/def/use fileds.
Meantimes, there are range_in/out/def/use fileds which records the live
range of the tracked pseudo.
2. in the df_live_subreg_finalize function, we move the tracked pseudo from
the partial_in/out/def/use to full_in/out/def/use if the pseudo's live
range is full.
gcc/ChangeLog:
* Makefile.in: Add subreg-live-range object file.
* df-problems.cc (struct df_live_subreg_problem_data): Private struct
for DF_LIVE_SUBREG problem.
(df_live_subreg_get_bb_info): getting bb regs in/out data.
(get_live_subreg_local_bb_info): getting bb regs use/def data.
(multireg_p): checking is the regno a pseudo multireg.
(need_track_subreg_p): checking is the regno need to be tracked.
(init_range): getting the range of subreg rtx.
(remove_subreg_range): removing use data for the reg/subreg rtx.
(add_subreg_range): adding def/use data for the reg/subreg rtx.
(df_live_subreg_free_bb_info): Free basic block df data.
(df_live_subreg_alloc): Allocate and init df data.
(df_live_subreg_reset): Reset the live in/out df data.
(df_live_subreg_bb_local_compute): Compute basic block df data.
(df_live_subreg_local_compute): Compute all basic blocks df data.
(df_live_subreg_init): Init the in/out df data.
(df_live_subreg_check_result): Assert the full and partial df data.
(df_live_subreg_confluence_0): Confluence function for infinite loops.
(df_live_subreg_confluence_n): Confluence function for normal edge.
(df_live_subreg_transfer_function): Transfer function.
(df_live_subreg_finalize): Finalize the all_in/all_out df data.
(df_live_subreg_free): Free the df data.
(df_live_subreg_top_dump): Dump top df data.
(df_live_subreg_bottom_dump): Dump bottom df data.
(df_live_subreg_add_problem): Add the DF_LIVE_SUBREG problem.
* df.h (enum df_problem_id): Add DF_LIVE_SUBREG.
(class subregs_live): Simple decalare.
(class df_live_subreg_local_bb_info): New class for full/partial def/use
df data.
(class df_live_subreg_bb_info): New class for full/partial in/out
df data.
(df_live_subreg): getting the df_live_subreg data.
(df_live_subreg_add_problem): Exported.
(df_live_subreg_finalize): Ditto.
(df_live_subreg_check_result): Ditto.
(multireg_p): Ditto.
(init_range): Ditto.
(add_subreg_range): Ditto.
(remove_subreg_range): Ditto.
(df_get_subreg_live_in): Accessor the all_in df data.
(df_get_subreg_live_out): Accessor the all_out df data.
(df_get_subreg_live_full_in): Accessor the full_in df data.
(df_get_subreg_live_full_out): Accessor the full_out df data.
(df_get_subreg_live_partial_in): Accessor the partial_in df data.
(df_get_subreg_live_partial_out): Accessor the partial_out df data.
(df_get_subreg_live_range_in): Accessor the range_in df data.
(df_get_subreg_live_range_out): Accessor the range_out df data.
* regs.h (get_nblocks): Get the blocks of mode.
* sbitmap.cc (bitmap_full_p): sbitmap predicator.
(bitmap_same_p): sbitmap predicator.
(test_full): test bitmap_full_p.
(test_same): test bitmap_same_p.
(sbitmap_cc_tests): Add test_full and test_same.
* sbitmap.h (bitmap_full_p): Exported.
(bitmap_same_p): Ditto.
* timevar.def (TV_DF_LIVE_SUBREG): add DF_LIVE_SUBREG timevar.
* subreg-live-range.cc: New file.
* subreg-live-range.h: New file.
---
gcc/Makefile.in | 1 +
gcc/df-problems.cc | 855 ++++++++++++++++++++++++++++++++++++++-
gcc/df.h | 155 +++++++
gcc/regs.h | 5 +
gcc/sbitmap.cc | 98 +++++
gcc/sbitmap.h | 2 +
gcc/subreg-live-range.cc | 53 +++
gcc/subreg-live-range.h | 206 ++++++++++
gcc/timevar.def | 1 +
9 files changed, 1375 insertions(+), 1 deletion(-)
create mode 100644 gcc/subreg-live-range.cc
create mode 100644 gcc/subreg-live-range.h
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index a74761b7ab3..e195238f6ab 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1684,6 +1684,7 @@ OBJS = \
store-motion.o \
streamer-hooks.o \
stringpool.o \
+ subreg-live-range.o \
substring-locations.o \
target-globals.o \
targhooks.o \
diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc
index 88ee0dd67fc..93c5b52146c 100644
--- a/gcc/df-problems.cc
+++ b/gcc/df-problems.cc
@@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see
#include "target.h"
#include "rtl.h"
#include "df.h"
+#include "subreg-live-range.h"
#include "memmodel.h"
#include "tm_p.h"
#include "insn-config.h"
@@ -1344,8 +1345,860 @@ df_lr_verify_transfer_functions (void)
bitmap_clear (&all_blocks);
}
+/*----------------------------------------------------------------------------
+ REGISTER AND SUBREGS LIVES
+ Like DF_LR, but include tracking subreg liveness. Currently used to provide
+ subreg liveness related information to the register allocator. The subreg
+ information is currently tracked for registers that satisfy the following
+ conditions:
+ 1. REG is a pseudo register
+ 2. MODE_SIZE > UNIT_SIZE
+ 3. MODE_SIZE is a multiple of UNIT_SIZE
+ 4. REG is used via subreg pattern
+ Assuming: MODE = the machine mode of the REG
+ MODE_SIZE = GET_MODE_SIZE (MODE)
+ UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE)
+ Condition 3 is currently strict, maybe it can be removed in the future, but
+ for now it is sufficient.
+ ----------------------------------------------------------------------------*/
+
+/* These two empty data are used as default data in case the user does not turn
+ * on the track-subreg-liveness feature. */
+bitmap_head empty_bitmap;
+subregs_live empty_live;
+
+/* Private data for live_subreg problem. */
+struct df_live_subreg_problem_data
+{
+ /* Record registers that need to track subreg liveness. */
+ bitmap_head tracked_regs;
+ /* An obstack for the bitmaps we need for this problem. */
+ bitmap_obstack live_subreg_bitmaps;
+};
+
+/* Helper functions */
+
+static df_live_subreg_bb_info *
+df_live_subreg_get_bb_info (unsigned int index)
+{
+ if (index < df_live_subreg->block_info_size)
+ return &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[index];
+ else
+ return NULL;
+}
+
+static df_live_subreg_local_bb_info *
+get_live_subreg_local_bb_info (unsigned int bb_index)
+{
+ return df_live_subreg_get_bb_info (bb_index);
+}
+
+/* Return true if regno is a multireg. */
+bool
+multireg_p (int regno)
+{
+ if (regno < FIRST_PSEUDO_REGISTER)
+ return false;
+ rtx regno_rtx = regno_reg_rtx[regno];
+ machine_mode reg_mode = GET_MODE (regno_rtx);
+ poly_int64 total_size = GET_MODE_SIZE (reg_mode);
+ poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode);
+ return maybe_gt (total_size, natural_size)
+ && multiple_p (total_size, natural_size);
+}
+
+/* Return true if the REGNO need be track with subreg liveness. */
+
+static bool
+need_track_subreg_p (unsigned regno)
+{
+ struct df_live_subreg_problem_data *problem_data
+ = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ return bitmap_bit_p (&problem_data->tracked_regs, regno);
+}
+
+/* Return the subreg_range of REF. */
+void
+init_range (rtx op, sbitmap range)
+{
+ rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op;
+ machine_mode reg_mode = GET_MODE (reg);
+
+ if (!read_modify_subreg_p (op))
+ {
+ bitmap_set_range (range, 0, get_nblocks (reg_mode));
+ return;
+ }
+
+ rtx subreg = op;
+ machine_mode subreg_mode = GET_MODE (subreg);
+ poly_int64 offset = SUBREG_BYTE (subreg);
+ int nblocks = get_nblocks (reg_mode);
+ poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode);
+ poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode);
+ poly_int64 left = offset + subreg_size;
+
+ int subreg_start = -1;
+ int subreg_nblocks = -1;
+ for (int i = 0; i < nblocks; i += 1)
+ {
+ poly_int64 right = unit_size * (i + 1);
+ if (subreg_start < 0 && maybe_lt (offset, right))
+ subreg_start = i;
+ if (subreg_nblocks < 0 && maybe_le (left, right))
+ {
+ subreg_nblocks = i + 1 - subreg_start;
+ break;
+ }
+ }
+ gcc_assert (subreg_start >= 0 && subreg_nblocks > 0);
+
+ bitmap_set_range (range, subreg_start, subreg_nblocks);
+}
+
+/* Remove RANGE from BB_INFO's use data. */
+void
+remove_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
+ const_sbitmap range)
+{
+ bitmap partial = &bb_info->partial_use;
+
+ bb_info->range_use->remove_range (regno, range);
+ if (bb_info->range_use->empty_p (regno))
+ bitmap_clear_bit (partial, regno);
+}
+
+/* Remove RANGE of REF from BB_INFO's use data. */
+static void
+remove_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref)
+{
+ unsigned int regno = DF_REF_REGNO (ref);
+ machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
+
+ if (need_track_subreg_p (regno))
+ {
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (DF_REF_REG (ref), range);
+ remove_subreg_range (bb_info, regno, range);
+ }
+ else
+ {
+ bitmap_clear_bit (&bb_info->full_use, regno);
+ gcc_assert (!bitmap_bit_p (&bb_info->partial_use, regno));
+ gcc_assert (!bitmap_bit_p (&bb_info->partial_def, regno));
+ }
+}
+
+/* add RANGE to BB_INFO's def/use. If is_def is true, means for BB_INFO's def,
+ otherwise for BB_INFO's use. */
+void
+add_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
+ const_sbitmap range, bool is_def)
+{
+ bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use;
+ subregs_live *range_live = is_def ? bb_info->range_def : bb_info->range_use;
+
+ bitmap_set_bit (partial, regno);
+ range_live->add_range (regno, range);
+}
+
+/* add RANGE of REF to BB_INFO def/use. If is_def is true, means for BB_INFO's
+ def, otherwise for BB_INFO's use. */
+static void
+add_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref,
+ bool is_def)
+{
+ unsigned int regno = DF_REF_REGNO (ref);
+ machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
+
+ if (need_track_subreg_p (regno))
+ {
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (DF_REF_REG (ref), range);
+ add_subreg_range (bb_info, regno, range, is_def);
+ }
+ else
+ {
+ bitmap full = is_def ? &bb_info->full_def : &bb_info->full_use;
+ bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use;
+ bitmap_set_bit (full, regno);
+ gcc_assert (!bitmap_bit_p (partial, regno));
+ }
+}
+
+/* Free basic block info. */
+
+static void
+df_live_subreg_free_bb_info (basic_block bb ATTRIBUTE_UNUSED, void *vbb_info)
+{
+ df_live_subreg_bb_info *bb_info = (df_live_subreg_bb_info *) vbb_info;
+ if (bb_info)
+ {
+ delete bb_info->range_in;
+ bb_info->range_in = NULL;
+ delete bb_info->range_out;
+ bb_info->range_out = NULL;
+
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ }
+}
+
+/* Allocate or reset bitmaps for DF_LIVE_SUBREG blocks. The solution bits are
+ not touched unless the block is new. */
+
+static void
+df_live_subreg_alloc (bitmap all_blocks ATTRIBUTE_UNUSED)
+{
+ struct df_live_subreg_problem_data *problem_data;
+ df_grow_bb_info (df_live_subreg);
+ if (df_live_subreg->problem_data)
+ problem_data
+ = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ else
+ {
+ problem_data = XNEW (struct df_live_subreg_problem_data);
+ df_live_subreg->problem_data = problem_data;
+
+ bitmap_obstack_initialize (&problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&problem_data->tracked_regs,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ bitmap_clear (&problem_data->tracked_regs);
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, bb->index);
+
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, ENTRY_BLOCK);
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, EXIT_BLOCK);
+
+ unsigned int bb_index;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ /* Find the regs which we need to track it's subreg liveness. */
+ rtx_insn *insn;
+ df_ref use;
+ FOR_BB_INSNS (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* A multireg which is used via subreg pattern. */
+ if (multireg_p (regno)
+ && DF_REF_FLAGS (use) & (DF_REF_SUBREG))
+ bitmap_set_bit (&problem_data->tracked_regs, regno);
+ }
+ }
+ }
+
+ if (dump_file)
+ {
+ fprintf (dump_file, ";; regs need to be tracked subreg liveness: ");
+ df_print_regset (dump_file, &problem_data->tracked_regs);
+ }
+
+ size_t n = bitmap_count_bits (&problem_data->tracked_regs);
+
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ /* Clean global infos. */
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+
+ /* When bitmaps are already initialized, just clear them. */
+ if (bb_info->all_in.obstack)
+ {
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ }
+ else
+ {
+ bitmap_initialize (&bb_info->all_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->full_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->partial_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->all_out,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->full_out,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->partial_out,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ if (bb_info->range_in)
+ {
+ bb_info->range_in->clear (n);
+ bb_info->range_out->clear (n);
+ }
+ else
+ {
+ bb_info->range_in = new subregs_live (n);
+ bb_info->range_out = new subregs_live (n);
+ }
+
+ /* Clean local infos. */
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ /* When bitmaps are already initialized, just clear them. */
+ if (local_bb_info->full_use.obstack)
+ {
+ bitmap_clear (&local_bb_info->full_def);
+ bitmap_clear (&local_bb_info->partial_def);
+ bitmap_clear (&local_bb_info->full_use);
+ bitmap_clear (&local_bb_info->partial_use);
+ }
+ else
+ {
+ bitmap_initialize (&local_bb_info->full_def,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->partial_def,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->full_use,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->partial_use,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ if (local_bb_info->range_def)
+ {
+ local_bb_info->range_def->clear (n);
+ local_bb_info->range_use->clear (n);
+ }
+ else
+ {
+ local_bb_info->range_def = new subregs_live (n);
+ local_bb_info->range_use = new subregs_live (n);
+ }
+ }
+
+ df_live_subreg->optional_p = true;
+}
+
+/* Reset the global solution for recalculation. */
+
+static void
+df_live_subreg_reset (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ gcc_assert (bb_info);
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ bb_info->range_in->clear ();
+ bb_info->range_out->clear ();
+ }
+}
+
+/* Compute local live register info for basic block BB. */
+
+static void
+df_live_subreg_bb_local_compute (unsigned int bb_index)
+{
+ basic_block bb = BASIC_BLOCK_FOR_FN (cfun, bb_index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ rtx_insn *insn;
+ df_ref def, use;
+
+ /* Process the registers set in an exception handler. */
+ FOR_EACH_ARTIFICIAL_DEF (def, bb_index)
+ if ((DF_REF_FLAGS (def) & DF_REF_AT_TOP) == 0)
+ {
+ add_subreg_range (local_bb_info, def, true);
+ remove_subreg_range (local_bb_info, def);
+ }
+
+ /* Process the hardware registers that are always live. */
+ FOR_EACH_ARTIFICIAL_USE (use, bb_index)
+ if ((DF_REF_FLAGS (use) & DF_REF_AT_TOP) == 0)
+ add_subreg_range (local_bb_info, use, false);
+
+ FOR_BB_INSNS_REVERSE (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+ FOR_EACH_INSN_INFO_DEF (def, insn_info)
+ {
+ remove_subreg_range (local_bb_info, def);
+ add_subreg_range (local_bb_info, def, true);
+ }
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* Ignore the use of subreg which is used as dest operand. */
+ if (need_track_subreg_p (regno)
+ && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG))
+ continue;
+ add_subreg_range (local_bb_info, use, false);
+ }
+ }
+
+ /* Process the registers set in an exception handler or the hard
+ frame pointer if this block is the target of a non local
+ goto. */
+ FOR_EACH_ARTIFICIAL_DEF (def, bb_index)
+ if (DF_REF_FLAGS (def) & DF_REF_AT_TOP)
+ {
+ add_subreg_range (local_bb_info, def, true);
+ remove_subreg_range (local_bb_info, def);
+ }
+
+#ifdef EH_USES
+ /* Process the uses that are live into an exception handler. */
+ FOR_EACH_ARTIFICIAL_USE (use, bb_index)
+ /* Add use to set of uses in this BB. */
+ if (DF_REF_FLAGS (use) & DF_REF_AT_TOP)
+ add_subreg_range (local_bb_info, use, false);
+#endif
+}
+
+/* Compute local live register info for each basic block within BLOCKS. */
+
+static void
+df_live_subreg_local_compute (bitmap all_blocks ATTRIBUTE_UNUSED)
+{
+ unsigned int bb_index, i;
+ bitmap_iterator bi;
+
+ bitmap_clear (&df->hardware_regs_used);
+
+ /* The all-important stack pointer must always be live. */
+ bitmap_set_bit (&df->hardware_regs_used, STACK_POINTER_REGNUM);
+
+ /* Global regs are always live, too. */
+ for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
+ if (global_regs[i])
+ bitmap_set_bit (&df->hardware_regs_used, i);
+
+ /* Before reload, there are a few registers that must be forced
+ live everywhere -- which might not already be the case for
+ blocks within infinite loops. */
+ if (!reload_completed)
+ {
+ unsigned int pic_offset_table_regnum = PIC_OFFSET_TABLE_REGNUM;
+ /* Any reference to any pseudo before reload is a potential
+ reference of the frame pointer. */
+ bitmap_set_bit (&df->hardware_regs_used, FRAME_POINTER_REGNUM);
+
+ /* Pseudos with argument area equivalences may require
+ reloading via the argument pointer. */
+ if (FRAME_POINTER_REGNUM != ARG_POINTER_REGNUM
+ && fixed_regs[ARG_POINTER_REGNUM])
+ bitmap_set_bit (&df->hardware_regs_used, ARG_POINTER_REGNUM);
+
+ /* Any constant, or pseudo with constant equivalences, may
+ require reloading from memory using the pic register. */
+ if (pic_offset_table_regnum != INVALID_REGNUM
+ && fixed_regs[pic_offset_table_regnum])
+ bitmap_set_bit (&df->hardware_regs_used, pic_offset_table_regnum);
+ }
+
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ if (bb_index == EXIT_BLOCK)
+ {
+ /* The exit block is special for this problem and its bits are
+ computed from thin air. */
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (EXIT_BLOCK);
+ bitmap_copy (&local_bb_info->full_use, df->exit_block_uses);
+ }
+ else
+ df_live_subreg_bb_local_compute (bb_index);
+ }
+
+ bitmap_clear (df_live_subreg->out_of_date_transfer_functions);
+}
+
+/* Initialize the solution vectors. */
+
+static void
+df_live_subreg_init (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+ bitmap_copy (&bb_info->full_in, &local_bb_info->full_use);
+ bitmap_copy (&bb_info->partial_in, &local_bb_info->partial_use);
+ bb_info->range_in->copy_lives (*local_bb_info->range_use);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ bb_info->range_out->clear ();
+ }
+}
+
+/* Check that the status of the final result is correct. */
+
+void
+df_live_subreg_check_result (bitmap full, bitmap partial,
+ subregs_live *partial_live)
+{
+ unsigned int regno;
+ bitmap_iterator bi;
+ /* Make sure that full and partial do not overlap. */
+ gcc_assert (!bitmap_intersect_p (full, partial));
+ /* Ensure that registers belonging to full are not partially live. */
+ EXECUTE_IF_SET_IN_BITMAP (full, 0, regno, bi)
+ gcc_assert (partial_live->empty_p (regno));
+ /* Ensure that registers belonging to partial are partially live. */
+ EXECUTE_IF_SET_IN_BITMAP (partial, 0, regno, bi)
+ gcc_assert (!partial_live->empty_p (regno));
+}
+
+/* Confluence function that processes infinite loops. This might be a
+ noreturn function that throws. And even if it isn't, getting the
+ unwind info right helps debugging. */
+static void
+df_live_subreg_confluence_0 (basic_block bb)
+{
+ bitmap full_out = &df_live_subreg_get_bb_info (bb->index)->full_out;
+ if (bb != EXIT_BLOCK_PTR_FOR_FN (cfun))
+ bitmap_copy (full_out, &df->hardware_regs_used);
+}
+
+/* Confluence function that ignores fake edges. */
+
+static bool
+df_live_subreg_confluence_n (edge e)
+{
+ class df_live_subreg_bb_info *src_bb_info
+ = df_live_subreg_get_bb_info (e->src->index);
+ class df_live_subreg_bb_info *dest_bb_info
+ = df_live_subreg_get_bb_info (e->dest->index);
+
+ bool changed = false;
+
+ /* Call-clobbered registers die across exception and call edges.
+ Conservatively treat partially-clobbered registers as surviving
+ across the edges; they might or might not, depending on what
+ mode they have. */
+ /* ??? Abnormal call edges ignored for the moment, as this gets
+ confused by sibling call edges, which crashes reg-stack. */
+ if (e->flags & EDGE_EH)
+ {
+ bitmap_view<HARD_REG_SET> eh_kills (eh_edge_abi.full_reg_clobbers ());
+ changed = bitmap_ior_and_compl_into (&src_bb_info->full_out,
+ &dest_bb_info->full_in, eh_kills);
+ }
+ else
+ changed = bitmap_ior_into (&src_bb_info->full_out, &dest_bb_info->full_in);
+
+ changed |= bitmap_ior_into (&src_bb_info->full_out, &df->hardware_regs_used);
+
+ /* Handle partial live case. */
+ if (!bitmap_empty_p (&dest_bb_info->partial_in))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (&dest_bb_info->partial_in,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap dest_range = dest_bb_info->range_in->get_range (regno);
+ changed |= src_bb_info->range_out->add_range (regno, dest_range);
+ }
+ changed |= bitmap_ior_into (&src_bb_info->partial_out,
+ &dest_bb_info->partial_in);
+
+ df_live_subreg_check_result (&src_bb_info->full_out,
+ &src_bb_info->partial_out,
+ src_bb_info->range_out);
+ }
+
+ return changed;
+}
+
+/* Transfer function. IN = USE | (OUT & ~DEF). */
+
+static bool
+df_live_subreg_transfer_function (int bb_index)
+{
+ class df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ class df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ bool changed = false;
+
+ changed
+ |= bitmap_ior_and_compl (&bb_info->full_in, &local_bb_info->full_use,
+ &bb_info->full_out, &local_bb_info->full_def);
+
+ /* Handle partial live case. */
+ if (!bitmap_empty_p (&bb_info->partial_out)
+ || !bitmap_empty_p (&local_bb_info->partial_use))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ bitmap_head temp_partial_out;
+ subregs_live temp_range_out;
+
+ /* TEMP = (OUT & ~DEF) */
+ bitmap_initialize (&temp_partial_out, &bitmap_default_obstack);
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER,
+ regno, bi)
+ {
+ sbitmap out_range = bb_info->range_out->get_range (regno);
+ temp_range_out.add_range (regno, out_range);
+ if (bitmap_bit_p (&local_bb_info->partial_def, regno))
+ {
+ sbitmap def_range = local_bb_info->range_def->get_range (regno);
+ temp_range_out.remove_range (regno, def_range);
+ if (!temp_range_out.empty_p (regno))
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+ else
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+
+ /* TEMP = USE | TEMP */
+ EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap use_range = local_bb_info->range_use->get_range (regno);
+ temp_range_out.add_range (regno, use_range);
+ }
+ bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use);
+
+ /* IN = TEMP */
+ changed |= bb_info->range_in->copy_lives (temp_range_out);
+ bitmap_copy (&bb_info->partial_in, &temp_partial_out);
+ df_live_subreg_check_result (&bb_info->full_in, &bb_info->partial_in,
+ bb_info->range_in);
+ }
+ else if (!bitmap_empty_p (&bb_info->partial_in))
+ {
+ changed = true;
+ bitmap_clear (&bb_info->partial_in);
+ bb_info->range_in->clear ();
+ }
+
+ return changed;
+}
+
+/* Calculating ALL_IN from FULL_IN and PARTIAL_IN
+ and ALL_OUT from FULL_OUT and PARTIAL_OUT. */
+
+void
+df_live_subreg_finalize (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ class df_live_subreg_bb_info *bb_info
+ = df_live_subreg_get_bb_info (bb_index);
+ gcc_assert (bb_info);
+ bitmap_ior (&bb_info->all_in, &bb_info->full_in, &bb_info->partial_in);
+ bitmap_ior (&bb_info->all_out, &bb_info->full_out, &bb_info->partial_out);
+
+ /* Move full live reg in partial_in/partial_out to full_in/full_out. */
+ unsigned int regno;
+ bitmap_iterator ri;
+ bool changed = false;
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_in, FIRST_PSEUDO_REGISTER,
+ regno, ri)
+ {
+ if (bb_info->range_in->full_p (regno))
+ {
+ bitmap_set_bit (&bb_info->full_in, regno);
+ bb_info->range_in->remove_range (regno);
+ changed = true;
+ }
+ }
+ if (changed)
+ bitmap_and_compl_into (&bb_info->partial_in, &bb_info->full_in);
+
+ changed = false;
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER,
+ regno, ri)
+ {
+ if (bb_info->range_out->full_p (regno))
+ {
+ bitmap_set_bit (&bb_info->full_out, regno);
+ bb_info->range_out->remove_range (regno);
+ changed = true;
+ }
+ }
+ if (changed)
+ bitmap_and_compl_into (&bb_info->partial_out, &bb_info->full_out);
+ }
+}
+
+/* Free all storage associated with the problem. */
+
+static void
+df_live_subreg_free (void)
+{
+ df_live_subreg_problem_data *problem_data
+ = (df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ if (df_live_subreg->block_info)
+ {
+ bitmap_obstack_release (&problem_data->live_subreg_bitmaps);
+ basic_block bb;
+ FOR_ALL_BB_FN (bb, cfun)
+ {
+ df_live_subreg_bb_info *bb_info
+ = df_live_subreg_get_bb_info (bb->index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb->index);
+ delete bb_info->range_in;
+ delete bb_info->range_out;
+ delete local_bb_info->range_def;
+ delete local_bb_info->range_use;
+ }
+ free (problem_data);
+ df_live_subreg->problem_data = NULL;
+
+ df_live_subreg->block_info_size = 0;
+ free (df_live_subreg->block_info);
+ df_live_subreg->block_info = NULL;
+ }
+
+ BITMAP_FREE (df_live_subreg->out_of_date_transfer_functions);
+ free (df_live_subreg);
+}
+
+/* Debugging info at top of bb. */
+
+static void
+df_live_subreg_top_dump (basic_block bb, FILE *file)
+{
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb->index);
+
+ if (!bb_info)
+ return;
+
+ fprintf (file, ";; subreg live all in \t");
+ df_print_regset (file, &bb_info->all_in);
+ fprintf (file, ";; subreg live full in \t");
+ df_print_regset (file, &bb_info->full_in);
+ fprintf (file, ";; subreg live partial in \t");
+ df_print_regset (file, &bb_info->partial_in);
+ fprintf (file, ";; subreg live range in \t");
+ bb_info->range_in->dump (file, "");
+
+ fprintf (file, "\n;; subreg live full use \t");
+ df_print_regset (file, &local_bb_info->full_use);
+ fprintf (file, ";; subreg live partial use \t");
+ df_print_regset (file, &local_bb_info->partial_use);
+ fprintf (file, ";; subreg live range use \t");
+ local_bb_info->range_use->dump (file, "");
+
+ fprintf (file, "\n;; subreg live full def \t");
+ df_print_regset (file, &local_bb_info->full_def);
+ fprintf (file, ";; subreg live partial def \t");
+ df_print_regset (file, &local_bb_info->partial_def);
+ fprintf (file, ";; subreg live range def \t");
+ local_bb_info->range_def->dump (file, "");
+}
+
+/* Debugging info at bottom of bb. */
+
+static void
+df_live_subreg_bottom_dump (basic_block bb, FILE *file)
+{
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index);
+ if (!bb_info)
+ return;
+
+ fprintf (file, ";; subreg live all out \t");
+ df_print_regset (file, &bb_info->all_out);
+ fprintf (file, ";; subreg live full out \t");
+ df_print_regset (file, &bb_info->full_out);
+ fprintf (file, ";; subreg live partial out \t");
+ df_print_regset (file, &bb_info->partial_out);
+ fprintf (file, ";; subreg live range out \t");
+ bb_info->range_out->dump (file, "");
+}
+
+/* All of the information associated with every instance of the problem. */
+
+static const struct df_problem problem_LIVE_SUBREG = {
+ DF_LIVE_SUBREG, /* Problem id. */
+ DF_BACKWARD, /* Direction. */
+ df_live_subreg_alloc, /* Allocate the problem specific data. */
+ df_live_subreg_reset, /* Reset global information. */
+ df_live_subreg_free_bb_info, /* Free basic block info. */
+ df_live_subreg_local_compute, /* Local compute function. */
+ df_live_subreg_init, /* Init the solution specific data. */
+ df_worklist_dataflow, /* Worklist solver. */
+ df_live_subreg_confluence_0, /* Confluence operator 0. */
+ df_live_subreg_confluence_n, /* Confluence operator n. */
+ df_live_subreg_transfer_function, /* Transfer function. */
+ df_live_subreg_finalize, /* Finalize function. */
+ df_live_subreg_free, /* Free all of the problem information. */
+ df_live_subreg_free, /* Remove this problem from the stack of dataflow
+ problems. */
+ NULL, /* Debugging. */
+ df_live_subreg_top_dump, /* Debugging start block. */
+ df_live_subreg_bottom_dump, /* Debugging end block. */
+ NULL, /* Debugging start insn. */
+ NULL, /* Debugging end insn. */
+ NULL, /* Incremental solution verify start. */
+ NULL, /* Incremental solution verify end. */
+ &problem_LR, /* Dependent problem. */
+ sizeof (df_live_subreg_bb_info), /* Size of entry of block_info array. */
+ TV_DF_LIVE_SUBREG, /* Timing variable. */
+ false /* Reset blocks on dropping out of blocks_to_analyze. */
+};
+
+/* Create a new DATAFLOW instance and add it to an existing instance
+ of DF. The returned structure is what is used to get at the
+ solution. */
+
+void
+df_live_subreg_add_problem (void)
+{
+ gcc_assert (flag_track_subreg_liveness);
+
+ df_add_problem (&problem_LIVE_SUBREG);
+
+ /* These will be initialized when df_scan_blocks processes each
+ block. */
+ df_live_subreg->out_of_date_transfer_functions
+ = BITMAP_ALLOC (&df_bitmap_obstack);
+}
-\f
/*----------------------------------------------------------------------------
LIVE AND MAY-INITIALIZED REGISTERS.
diff --git a/gcc/df.h b/gcc/df.h
index 84e5aa8b524..78c57273cf1 100644
--- a/gcc/df.h
+++ b/gcc/df.h
@@ -47,6 +47,7 @@ enum df_problem_id
{
DF_SCAN,
DF_LR, /* Live Registers backward. */
+ DF_LIVE_SUBREG, /* Live Regs and Subregs */
DF_LIVE, /* Live Registers & Uninitialized Registers */
DF_RD, /* Reaching Defs. */
DF_CHAIN, /* Def-Use and/or Use-Def Chains. */
@@ -878,6 +879,33 @@ public:
bitmap_head out; /* At the bottom of the block. */
};
+class subregs_live; /* Defined in subreg-range.{h,cc} */
+
+/* Local live info in basic block. Use by live_subreg problem and LRA pass. */
+class df_live_subreg_local_bb_info
+{
+public:
+ bitmap_head full_def;
+ bitmap_head full_use;
+ /* Only for pseudo registers. */
+ bitmap_head partial_def;
+ bitmap_head partial_use;
+ subregs_live *range_def = nullptr;
+ subregs_live *range_use = nullptr;
+};
+
+/* Live in/out infos of each basic. */
+class df_live_subreg_bb_info : public df_live_subreg_local_bb_info
+{
+public:
+ bitmap_head all_in, full_in;
+ bitmap_head all_out, full_out;
+ /* Only for pseudo registers. */
+ bitmap_head partial_in;
+ bitmap_head partial_out;
+ subregs_live *range_in = nullptr;
+ subregs_live *range_out = nullptr;
+};
/* Uninitialized registers. All bitmaps are referenced by the
register number. Anded results of the forwards and backward live
@@ -940,6 +968,7 @@ extern class df_d *df;
#define df_scan (df->problems_by_index[DF_SCAN])
#define df_rd (df->problems_by_index[DF_RD])
#define df_lr (df->problems_by_index[DF_LR])
+#define df_live_subreg (df->problems_by_index[DF_LIVE_SUBREG])
#define df_live (df->problems_by_index[DF_LIVE])
#define df_chain (df->problems_by_index[DF_CHAIN])
#define df_word_lr (df->problems_by_index[DF_WORD_LR])
@@ -1031,6 +1060,8 @@ extern void df_lr_add_problem (void);
extern void df_lr_verify_transfer_functions (void);
extern void df_live_verify_transfer_functions (void);
extern void df_live_add_problem (void);
+extern void df_live_subreg_add_problem ();
+extern void df_live_subreg_finalize (bitmap all_blocks);
extern void df_live_set_all_dirty (void);
extern void df_chain_add_problem (unsigned int);
extern void df_word_lr_add_problem (void);
@@ -1059,6 +1090,16 @@ extern bool can_move_insns_across (rtx_insn *, rtx_insn *,
rtx_insn *, rtx_insn *,
basic_block, regset,
regset, rtx_insn **);
+extern void
+df_live_subreg_check_result (bitmap, bitmap, subregs_live *);
+extern bool multireg_p (int);
+extern void init_range (rtx, sbitmap);
+extern void
+add_subreg_range (df_live_subreg_local_bb_info *, unsigned int, const_sbitmap,
+ bool);
+extern void
+remove_subreg_range (df_live_subreg_local_bb_info *, unsigned int,
+ const_sbitmap);
/* Functions defined in df-scan.cc. */
extern void df_scan_alloc (bitmap);
@@ -1192,6 +1233,120 @@ df_get_live_in (basic_block bb)
return DF_LR_IN (bb);
}
+/* Get the subreg live at in set for BB. The live set include full and partial
+ * live. we only track and use subreg liveness when -ftrack-subreg-liveness,
+ * otherwise use DF_LR_IN. This function is used by the register allocators. */
+
+inline bitmap
+df_get_subreg_live_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->all_in);
+ }
+ return DF_LR_IN (bb);
+}
+
+/* Get the subreg live at out set for BB. The live set include full and
+ * partial live. we only track and use subreg liveness when
+ * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by
+ * the register allocators. */
+
+inline bitmap
+df_get_subreg_live_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->all_out);
+ }
+ return DF_LR_OUT (bb);
+}
+
+/* Get the subreg live at in set for BB. The live set only include full and
+ * partial live. we only track and use subreg liveness when
+ * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by
+ * the register allocators. */
+
+inline bitmap
+df_get_subreg_live_full_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->full_in);
+ }
+ return DF_LR_IN (bb);
+}
+
+inline bitmap
+df_get_subreg_live_full_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->full_out);
+ }
+ return DF_LR_OUT (bb);
+}
+
+/* Define in df-problems.cc, used when disable track-subreg-liveness. */
+extern bitmap_head empty_bitmap;
+extern subregs_live empty_live;
+
+inline bitmap
+df_get_subreg_live_partial_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->partial_in);
+ }
+ return &empty_bitmap;
+}
+
+inline bitmap
+df_get_subreg_live_partial_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->partial_out);
+ }
+ return &empty_bitmap;
+}
+
+inline subregs_live *
+df_get_subreg_live_range_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return bb_info->range_in;
+ }
+ return &empty_live;
+}
+
+inline subregs_live *
+df_get_subreg_live_range_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return bb_info->range_out;
+ }
+ return &empty_live;
+}
+
/* Get basic block info. */
/* Get the artificial defs for a basic block. */
diff --git a/gcc/regs.h b/gcc/regs.h
index 16189c58fd1..3b524096533 100644
--- a/gcc/regs.h
+++ b/gcc/regs.h
@@ -389,4 +389,9 @@ range_in_hard_reg_set_p (const_hard_reg_set set, unsigned regno, int nregs)
return true;
}
+/* Return the number of blocks the MODE overlap. One block equal to mode's
+ natural size. */
+#define get_nblocks(mode) \
+ (exact_div (GET_MODE_SIZE (mode), REGMODE_NATURAL_SIZE (mode)).to_constant ())
+
#endif /* GCC_REGS_H */
diff --git a/gcc/sbitmap.cc b/gcc/sbitmap.cc
index 03bb2c6d44b..5d960adcabe 100644
--- a/gcc/sbitmap.cc
+++ b/gcc/sbitmap.cc
@@ -208,6 +208,29 @@ bitmap_empty_p (const_sbitmap bmap)
return true;
}
+/* Return true if the bitmap is full, i.e. all bits are set. */
+
+bool
+bitmap_full_p (const_sbitmap bmap)
+{
+ unsigned int end_word = bmap->n_bits / SBITMAP_ELT_BITS;
+ unsigned int i = 0;
+ for (; i < end_word; i++)
+ if (bmap->elms[i] != (SBITMAP_ELT_TYPE) -1)
+ return false;
+
+ unsigned int end_bitno = bmap->n_bits % SBITMAP_ELT_BITS;
+
+ if (end_bitno == 0)
+ return true;
+
+ gcc_checking_assert (i + 1 == bmap->size);
+
+ SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1;
+ /* Make sure the tail bits are set. */
+ return (bmap->elms[i] & mask) == mask;
+}
+
/* Clear COUNT bits from START in BMAP. */
void
@@ -662,6 +685,36 @@ bitmap_subset_p (const_sbitmap a, const_sbitmap b)
return true;
}
+/* Return true if the bits in A and B are set the same and the number of bits is
+ the same. */
+
+bool
+bitmap_same_p (const_sbitmap a, const_sbitmap b)
+{
+ if (a->n_bits != b->n_bits)
+ return false;
+
+ gcc_checking_assert (a->size == b->size);
+
+ unsigned int end_word = a->n_bits / SBITMAP_ELT_BITS;
+ unsigned int i = 0;
+ for (; i < end_word; i++)
+ if (a->elms[i] != b->elms[i])
+ return false;
+
+ unsigned int end_bitno = a->n_bits % SBITMAP_ELT_BITS;
+
+ if (end_bitno == 0)
+ return true;
+
+ gcc_checking_assert (i + 1 == a->size);
+
+ SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1;
+
+ /* Make sure the tail bits are same. */
+ return (a->elms[i] & mask) == (b->elms[i] & mask);
+}
+
/* Set DST to be (A or (B and C)).
Return nonzero if any change is made. */
@@ -994,6 +1047,49 @@ test_bit_in_range ()
sbitmap_free (s);
}
+/* Verify bitmap_full_p functions for sbitmap. */
+
+static void
+test_full ()
+{
+ sbitmap s = sbitmap_alloc (193);
+
+ bitmap_clear (s);
+ ASSERT_FALSE (bitmap_full_p (s));
+
+ bitmap_ones (s);
+ ASSERT_TRUE (bitmap_full_p (s));
+
+ bitmap_clear_bit (s, 192);
+ ASSERT_FALSE (bitmap_full_p (s));
+
+ bitmap_ones (s);
+ bitmap_clear_bit (s, 17);
+ ASSERT_FALSE (bitmap_full_p (s));
+}
+
+/* Verify bitmap_same_p functions for sbitmap. */
+
+static void
+test_same ()
+{
+ sbitmap s1 = sbitmap_alloc (193);
+ sbitmap s2 = sbitmap_alloc (193);
+ sbitmap s3 = sbitmap_alloc (192);
+
+ ASSERT_FALSE (bitmap_same_p (s1, s3));
+
+ bitmap_clear (s1);
+ bitmap_clear (s2);
+ ASSERT_TRUE (bitmap_same_p (s1, s2));
+
+ bitmap_set_bit (s2, 192);
+ ASSERT_FALSE (bitmap_same_p (s1, s2));
+
+ bitmap_set_bit (s1, 192);
+ ASSERT_TRUE (bitmap_same_p (s1, s2));
+}
+
/* Run all of the selftests within this file. */
void
@@ -1001,6 +1097,8 @@ sbitmap_cc_tests ()
{
test_set_range ();
test_bit_in_range ();
+ test_full ();
+ test_same ();
}
} // namespace selftest
diff --git a/gcc/sbitmap.h b/gcc/sbitmap.h
index da6116ce925..71cfded9fb2 100644
--- a/gcc/sbitmap.h
+++ b/gcc/sbitmap.h
@@ -267,6 +267,7 @@ extern void bitmap_copy (sbitmap, const_sbitmap);
extern bool bitmap_equal_p (const_sbitmap, const_sbitmap);
extern unsigned int bitmap_count_bits (const_sbitmap);
extern bool bitmap_empty_p (const_sbitmap);
+extern bool bitmap_full_p (const_sbitmap);
extern void bitmap_clear (sbitmap);
extern void bitmap_clear_range (sbitmap, unsigned, unsigned);
extern void bitmap_set_range (sbitmap, unsigned, unsigned);
@@ -287,6 +288,7 @@ extern bool bitmap_and (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_ior (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_xor (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_subset_p (const_sbitmap, const_sbitmap);
+extern bool bitmap_same_p (const_sbitmap, const_sbitmap);
extern bool bitmap_bit_in_range_p (const_sbitmap, unsigned int, unsigned int);
extern int bitmap_first_set_bit (const_sbitmap);
diff --git a/gcc/subreg-live-range.cc b/gcc/subreg-live-range.cc
new file mode 100644
index 00000000000..7e8e081844f
--- /dev/null
+++ b/gcc/subreg-live-range.cc
@@ -0,0 +1,53 @@
+/* SUBREG liveness tracking classes for DF & IRA & LRA.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ Contributed by Lehua Ding (lehua.ding@rivai.ai), RiVAI Technologies Ltd.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#include "subreg-live-range.h"
+
+void
+subregs_live::dump (FILE *file, const char *indent) const
+{
+ if (lives.empty ())
+ {
+ fprintf (file, "%sempty\n", indent);
+ return;
+ }
+ fprintf (file, "%s", indent);
+ for (auto &kv : lives)
+ {
+ const_sbitmap range = kv.second;
+ if (bitmap_empty_p (range))
+ continue;
+ fprintf (file, "%d: ", kv.first);
+ if (!bitmap_full_p (range))
+ {
+ dump_bitmap_file (file, range);
+ fprintf (file, ", ");
+ }
+ else
+ fprintf (file, "full, ");
+ }
+ fprintf (file, "\n");
+}
+
+DEBUG_FUNCTION void
+debug (const subregs_live &l)
+{
+ l.dump (stderr, "");
+}
diff --git a/gcc/subreg-live-range.h b/gcc/subreg-live-range.h
new file mode 100644
index 00000000000..4eafe006935
--- /dev/null
+++ b/gcc/subreg-live-range.h
@@ -0,0 +1,206 @@
+/* SUBREG liveness tracking classes for DF & IRA & LRA.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ Contributed by Lehua Ding (lehua.ding@rivai.ai), RiVAI Technologies Ltd.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_SUBREG_LIVE_RANGE_H
+#define GCC_SUBREG_LIVE_RANGE_H
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include <unordered_map>
+#include "sbitmap.h"
+
+/* class subregs_live record the live subreg_ranges of registers. */
+class subregs_live
+{
+public:
+ /* The key is usually the register's regno. */
+ std::unordered_map<unsigned int, sbitmap> lives;
+
+ subregs_live () : lives () {}
+
+ subregs_live (size_t n) : lives (n) {}
+
+ ~subregs_live ()
+ {
+ for (auto &kv : lives)
+ sbitmap_free (kv.second);
+ }
+
+ void clear ()
+ {
+ for (auto &kv : lives)
+ sbitmap_free (kv.second);
+ lives.clear ();
+ }
+
+ void clear (size_t n)
+ {
+ clear ();
+ lives.rehash (n);
+ }
+
+ bool find_p (unsigned int regno) const
+ {
+ return lives.find (regno) != lives.end ();
+ }
+
+ sbitmap get_range (unsigned int regno)
+ {
+ gcc_assert (find_p (regno));
+ return lives.at (regno);
+ }
+
+ const_sbitmap get_range (unsigned int regno) const
+ {
+ gcc_assert (find_p (regno));
+ return lives.at (regno);
+ }
+
+ /* Added RANGE to regno's ranges. Return true if leads to a change. */
+ bool add_range (unsigned int regno, const_sbitmap range)
+ {
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ gcc_assert (range->n_bits == curr->n_bits);
+ return bitmap_ior (curr, curr, range);
+ }
+ else
+ {
+ sbitmap a = sbitmap_alloc (range->n_bits);
+ lives.insert ({regno, a});
+ sbitmap curr = get_range (regno);
+ bitmap_copy (curr, range);
+ return !bitmap_empty_p (range);
+ }
+ }
+ /* Removed RANGE from regno's ranges. Return true if leads to a change. */
+ bool remove_range (unsigned int regno, const_sbitmap range)
+ {
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ bitmap_check_sizes (curr, range);
+ if (bitmap_subset_p (curr, range))
+ return remove_range (regno);
+
+ auto_sbitmap a (range->n_bits);
+ bitmap_not (a, range);
+ return bitmap_and (curr, curr, a);
+ }
+ return false;
+ }
+ /* Removed the whole range of REGNO. Return true if leads to a change. */
+ bool remove_range (unsigned int regno)
+ {
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ bool changed = !bitmap_empty_p (curr);
+ sbitmap_free (curr);
+ lives.erase (regno);
+ return changed;
+ }
+ return false;
+ }
+ /* Replace the range of REGNO with RANGE. Return true if leads to a change.
+ */
+ bool replace_range (unsigned int regno, const_sbitmap range)
+ {
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ if (!bitmap_same_p (curr, range))
+ {
+ bitmap_copy (curr, range);
+ return true;
+ }
+ else
+ return false;
+ }
+ else
+ return add_range (regno, range);
+ }
+ /* Copy subregs_live SL. Return true if leads to a change. */
+ bool copy_lives (const subregs_live &sl)
+ {
+ bool changed = false;
+ for (auto &kv : sl.lives)
+ {
+ unsigned int regno = kv.first;
+ const_sbitmap range = kv.second;
+ if (bitmap_empty_p (range))
+ continue;
+ if (find_p (regno))
+ changed |= replace_range (regno, range);
+ else
+ changed |= add_range (regno, range);
+ }
+
+ for (auto it = lives.cbegin (); it != lives.cend ();)
+ {
+ unsigned int regno = it->first;
+ auto prev_it = it;
+ it++;
+ if (sl.empty_p (regno))
+ {
+ changed |= bitmap_empty_p (it->second);
+ lives.erase (prev_it);
+ }
+ }
+
+ return changed;
+ }
+ /* Added subregs_live SL. Return true if leads to a change. */
+ bool add_lives (const subregs_live &sl)
+ {
+ bool changed = false;
+ for (auto &kv : sl.lives)
+ {
+ unsigned int regno = kv.first;
+ const_sbitmap range = kv.second;
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ changed |= bitmap_ior (curr, curr, range);
+ }
+ else
+ changed |= add_range (regno, range);
+ }
+ return changed;
+ }
+
+ /* Return true if regno's live range is full. */
+ bool full_p (unsigned int regno) const
+ {
+ return find_p (regno) && bitmap_full_p (get_range (regno));
+ }
+ /* Return true if regno's live range is empty. */
+ bool empty_p (unsigned int regno) const
+ {
+ return !find_p (regno) || bitmap_empty_p (get_range (regno));
+ }
+
+ /* Debug helper. */
+ void dump (FILE *file, const char *indent = ";; ") const;
+};
+
+#endif /* GCC_SUBREG_LIVE_RANGE_H */
diff --git a/gcc/timevar.def b/gcc/timevar.def
index 8e2168e0817..337a2b7225b 100644
--- a/gcc/timevar.def
+++ b/gcc/timevar.def
@@ -120,6 +120,7 @@ DEFTIMEVAR (TV_DF_SCAN , "df scan insns")
DEFTIMEVAR (TV_DF_MD , "df multiple defs")
DEFTIMEVAR (TV_DF_RD , "df reaching defs")
DEFTIMEVAR (TV_DF_LR , "df live regs")
+DEFTIMEVAR (TV_DF_LIVE_SUBREG , "df live subregs")
DEFTIMEVAR (TV_DF_LIVE , "df live&initialized regs")
DEFTIMEVAR (TV_DF_MIR , "df must-initialized regs")
DEFTIMEVAR (TV_DF_CHAIN , "df use-def / def-use chains")
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 3/4] ira: Apply DF_LIVE_SUBREG data
2024-04-24 10:05 [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Lehua Ding
2024-04-24 10:05 ` [PATCH 1/4] df: Add -ftrack-subreg-liveness option Lehua Ding
2024-04-24 10:05 ` [PATCH 2/4] df: Add DF_LIVE_SUBREG problem Lehua Ding
@ 2024-04-24 10:05 ` Lehua Ding
2024-04-24 10:05 ` [PATCH 4/4] lra: " Lehua Ding
3 siblings, 0 replies; 6+ messages in thread
From: Lehua Ding @ 2024-04-24 10:05 UTC (permalink / raw)
To: gcc-patches; +Cc: juzhe.zhong, shuo.chen, jin.xia, vmakarov, richard.sandiford
This patch simple replace df_get_live_in to df_get_subreg_live_in
and replace df_get_live_out to df_get_subreg_live_out.
gcc/ChangeLog:
* ira-build.cc (create_bb_allocnos): Switch to DF_LIVE_SUBREG df data.
(create_loop_allocnos): Ditto.
* ira-color.cc (ira_loop_edge_freq): Ditto.
* ira-emit.cc (generate_edge_moves): Ditto.
(add_ranges_and_copies): Ditto.
* ira-lives.cc (process_out_of_region_eh_regs): Ditto.
(add_conflict_from_region_landing_pads): Ditto.
(process_bb_node_lives): Ditto.
* ira.cc (find_moveable_pseudos): Ditto.
(interesting_dest_for_shprep_1): Ditto.
(allocate_initial_values): Ditto.
(ira): Ditto.
---
gcc/ira-build.cc | 7 ++++---
gcc/ira-color.cc | 8 ++++----
gcc/ira-emit.cc | 12 ++++++------
gcc/ira-lives.cc | 7 ++++---
gcc/ira.cc | 19 ++++++++++++-------
5 files changed, 30 insertions(+), 23 deletions(-)
diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc
index ea593d5a087..283ff36d3dd 100644
--- a/gcc/ira-build.cc
+++ b/gcc/ira-build.cc
@@ -1921,7 +1921,8 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node)
create_insn_allocnos (PATTERN (insn), NULL, false);
/* It might be a allocno living through from one subloop to
another. */
- EXECUTE_IF_SET_IN_REG_SET (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, i, bi)
+ EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ i, bi)
if (ira_curr_regno_allocno_map[i] == NULL)
ira_create_allocno (i, false, ira_curr_loop_tree_node);
}
@@ -1937,9 +1938,9 @@ create_loop_allocnos (edge e)
bitmap_iterator bi;
ira_loop_tree_node_t parent;
- live_in_regs = df_get_live_in (e->dest);
+ live_in_regs = df_get_subreg_live_in (e->dest);
border_allocnos = ira_curr_loop_tree_node->border_allocnos;
- EXECUTE_IF_SET_IN_REG_SET (df_get_live_out (e->src),
+ EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_out (e->src),
FIRST_PSEUDO_REGISTER, i, bi)
if (bitmap_bit_p (live_in_regs, i))
{
diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc
index b9ae32d1b4d..bfebc48ef83 100644
--- a/gcc/ira-color.cc
+++ b/gcc/ira-color.cc
@@ -2786,8 +2786,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p)
FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds)
if (e->src != loop_node->loop->latch
&& (regno < 0
- || (bitmap_bit_p (df_get_live_out (e->src), regno)
- && bitmap_bit_p (df_get_live_in (e->dest), regno))))
+ || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno)
+ && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno))))
freq += EDGE_FREQUENCY (e);
}
else
@@ -2795,8 +2795,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p)
auto_vec<edge> edges = get_loop_exit_edges (loop_node->loop);
FOR_EACH_VEC_ELT (edges, i, e)
if (regno < 0
- || (bitmap_bit_p (df_get_live_out (e->src), regno)
- && bitmap_bit_p (df_get_live_in (e->dest), regno)))
+ || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno)
+ && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno)))
freq += EDGE_FREQUENCY (e);
}
diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc
index d347f11fa02..8075b082e36 100644
--- a/gcc/ira-emit.cc
+++ b/gcc/ira-emit.cc
@@ -510,8 +510,8 @@ generate_edge_moves (edge e)
return;
src_map = src_loop_node->regno_allocno_map;
dest_map = dest_loop_node->regno_allocno_map;
- regs_live_in_dest = df_get_live_in (e->dest);
- regs_live_out_src = df_get_live_out (e->src);
+ regs_live_in_dest = df_get_subreg_live_in (e->dest);
+ regs_live_out_src = df_get_subreg_live_out (e->src);
EXECUTE_IF_SET_IN_REG_SET (regs_live_in_dest,
FIRST_PSEUDO_REGISTER, regno, bi)
if (bitmap_bit_p (regs_live_out_src, regno))
@@ -1229,16 +1229,16 @@ add_ranges_and_copies (void)
destination block) to use for searching allocnos by their
regnos because of subsequent IR flattening. */
node = IRA_BB_NODE (bb)->parent;
- bitmap_copy (live_through, df_get_live_in (bb));
+ bitmap_copy (live_through, df_get_subreg_live_in (bb));
add_range_and_copies_from_move_list
(at_bb_start[bb->index], node, live_through, REG_FREQ_FROM_BB (bb));
- bitmap_copy (live_through, df_get_live_out (bb));
+ bitmap_copy (live_through, df_get_subreg_live_out (bb));
add_range_and_copies_from_move_list
(at_bb_end[bb->index], node, live_through, REG_FREQ_FROM_BB (bb));
FOR_EACH_EDGE (e, ei, bb->succs)
{
- bitmap_and (live_through,
- df_get_live_in (e->dest), df_get_live_out (bb));
+ bitmap_and (live_through, df_get_subreg_live_in (e->dest),
+ df_get_subreg_live_out (bb));
add_range_and_copies_from_move_list
((move_t) e->aux, node, live_through,
REG_FREQ_FROM_EDGE_FREQ (EDGE_FREQUENCY (e)));
diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc
index e07d3dc3e89..7641184069d 100644
--- a/gcc/ira-lives.cc
+++ b/gcc/ira-lives.cc
@@ -1254,7 +1254,8 @@ process_out_of_region_eh_regs (basic_block bb)
if (! eh_p)
return;
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, i, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER,
+ i, bi)
{
ira_allocno_t a = ira_curr_regno_allocno_map[i];
for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--)
@@ -1288,7 +1289,7 @@ add_conflict_from_region_landing_pads (eh_region region, ira_object_t obj,
if ((landing_label = lp->landing_pad) != NULL
&& (landing_bb = BLOCK_FOR_INSN (landing_label)) != NULL
&& (region->type != ERT_CLEANUP
- || bitmap_bit_p (df_get_live_in (landing_bb),
+ || bitmap_bit_p (df_get_subreg_live_in (landing_bb),
ALLOCNO_REGNO (a))))
{
HARD_REG_SET new_conflict_regs
@@ -1325,7 +1326,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node)
high_pressure_start_point[ira_pressure_classes[i]] = -1;
}
curr_bb_node = loop_tree_node;
- reg_live_out = df_get_live_out (bb);
+ reg_live_out = df_get_subreg_live_out (bb);
sparseset_clear (objects_live);
REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out);
hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs);
diff --git a/gcc/ira.cc b/gcc/ira.cc
index 5642aea3caa..bebf6bb3e9b 100644
--- a/gcc/ira.cc
+++ b/gcc/ira.cc
@@ -4735,8 +4735,8 @@ find_moveable_pseudos (void)
bitmap_initialize (local, 0);
bitmap_initialize (transp, 0);
bitmap_initialize (moveable, 0);
- bitmap_copy (live, df_get_live_out (bb));
- bitmap_and_into (live, df_get_live_in (bb));
+ bitmap_copy (live, df_get_subreg_live_out (bb));
+ bitmap_and_into (live, df_get_subreg_live_in (bb));
bitmap_copy (transp, live);
bitmap_clear (moveable);
bitmap_clear (live);
@@ -5036,7 +5036,8 @@ interesting_dest_for_shprep_1 (rtx set, basic_block call_dom)
rtx dest = SET_DEST (set);
if (!REG_P (src) || !HARD_REGISTER_P (src)
|| !REG_P (dest) || HARD_REGISTER_P (dest)
- || (call_dom && !bitmap_bit_p (df_get_live_in (call_dom), REGNO (dest))))
+ || (call_dom
+ && !bitmap_bit_p (df_get_subreg_live_in (call_dom), REGNO (dest))))
return NULL;
return dest;
}
@@ -5514,10 +5515,12 @@ allocate_initial_values (void)
/* Update global register liveness information. */
FOR_EACH_BB_FN (bb, cfun)
{
- if (REGNO_REG_SET_P (df_get_live_in (bb), regno))
- SET_REGNO_REG_SET (df_get_live_in (bb), new_regno);
- if (REGNO_REG_SET_P (df_get_live_out (bb), regno))
- SET_REGNO_REG_SET (df_get_live_out (bb), new_regno);
+ if (REGNO_REG_SET_P (df_get_subreg_live_in (bb), regno))
+ SET_REGNO_REG_SET (df_get_subreg_live_in (bb),
+ new_regno);
+ if (REGNO_REG_SET_P (df_get_subreg_live_out (bb), regno))
+ SET_REGNO_REG_SET (df_get_subreg_live_out (bb),
+ new_regno);
}
}
}
@@ -5679,6 +5682,8 @@ ira (FILE *f)
if (optimize > 1)
df_remove_problem (df_live);
gcc_checking_assert (df_live == NULL);
+ if (flag_track_subreg_liveness)
+ df_live_subreg_add_problem ();
if (flag_checking)
df->changeable_flags |= DF_VERIFY_SCHEDULED;
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 4/4] lra: Apply DF_LIVE_SUBREG data
2024-04-24 10:05 [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Lehua Ding
` (2 preceding siblings ...)
2024-04-24 10:05 ` [PATCH 3/4] ira: Apply DF_LIVE_SUBREG data Lehua Ding
@ 2024-04-24 10:05 ` Lehua Ding
3 siblings, 0 replies; 6+ messages in thread
From: Lehua Ding @ 2024-04-24 10:05 UTC (permalink / raw)
To: gcc-patches; +Cc: juzhe.zhong, shuo.chen, jin.xia, vmakarov, richard.sandiford
This patch apply the DF_LIVE_SUBREG to LRA pass. More changes were made
to the LRA than the IRA since the LRA will modify the DF data directly.
The main big changes are centered on the lra-lives.cc file.
gcc/ChangeLog:
* lra-coalesce.cc (update_live_info): Extend to DF_LIVE_SUBREG.
(lra_coalesce): Ditto.
* lra-constraints.cc (update_ebb_live_info): Ditto.
(get_live_on_other_edges): Ditto.
(inherit_in_ebb): Ditto.
(lra_inheritance): Ditto.
(fix_bb_live_info): Ditto.
(remove_inheritance_pseudos): Ditto.
* lra-int.h (GCC_LRA_INT_H): include subreg-live-range.h
(struct lra_insn_reg): Add op filed to record the corresponding rtx.
* lra-lives.cc (class bb_data_pseudos): Extend the bb_data_pseudos to
include new partial_def/use and range_def/use fileds for DF_LIVE_SUBREG
problem.
(need_track_subreg_p): checking is the regno need to be tracked.
(make_hard_regno_live): switch to live_subreg filed.
(make_hard_regno_dead): Ditto.
(mark_regno_live): Support record subreg liveness.
(mark_regno_dead): Ditto.
(live_trans_fun): Adjust transfer function to support subreg liveness.
(live_con_fun_0): Adjust Confluence function to support subreg liveness.
(live_con_fun_n): Ditto.
(initiate_live_solver): Ditto.
(finish_live_solver): Ditto.
(process_bb_lives): Ditto.
(lra_create_live_ranges_1): Dump subreg liveness.
* lra-remat.cc (dump_candidates_and_remat_bb_data): Switch to
DF_LIVE_SUBREG df data.
(calculate_livein_cands): Ditto.
(do_remat): Ditto.
* lra-spills.cc (spill_pseudos): Ditto.
* lra.cc (new_insn_reg): New argument op.
(add_regs_to_insn_regno_info): Add new argument op.
---
gcc/lra-coalesce.cc | 27 +++-
gcc/lra-constraints.cc | 109 ++++++++++---
gcc/lra-int.h | 4 +
gcc/lra-lives.cc | 357 ++++++++++++++++++++++++++++++++---------
gcc/lra-remat.cc | 8 +-
gcc/lra-spills.cc | 27 +++-
gcc/lra.cc | 10 +-
7 files changed, 430 insertions(+), 112 deletions(-)
diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc
index a9b5b51cb3f..9416775a009 100644
--- a/gcc/lra-coalesce.cc
+++ b/gcc/lra-coalesce.cc
@@ -186,19 +186,28 @@ static bitmap_head used_pseudos_bitmap;
/* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info
bitmap). */
static void
-update_live_info (bitmap lr_bitmap)
+update_live_info (bitmap all, bitmap full, bitmap partial)
{
unsigned int j;
bitmap_iterator bi;
bitmap_clear (&used_pseudos_bitmap);
- EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap,
+ EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all,
FIRST_PSEUDO_REGISTER, j, bi)
bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]);
- if (! bitmap_empty_p (&used_pseudos_bitmap))
+ if (!bitmap_empty_p (&used_pseudos_bitmap))
{
- bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap);
- bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap);
+ bitmap_and_compl_into (all, &coalesced_pseudos_bitmap);
+ bitmap_ior_into (all, &used_pseudos_bitmap);
+
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_and_compl_into (full, &coalesced_pseudos_bitmap);
+ bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial);
+
+ bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap);
+ bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full);
+ }
}
}
@@ -301,8 +310,12 @@ lra_coalesce (void)
bitmap_initialize (&used_pseudos_bitmap, ®_obstack);
FOR_EACH_BB_FN (bb, cfun)
{
- update_live_info (df_get_live_in (bb));
- update_live_info (df_get_live_out (bb));
+ update_live_info (df_get_subreg_live_in (bb),
+ df_get_subreg_live_full_in (bb),
+ df_get_subreg_live_partial_in (bb));
+ update_live_info (df_get_subreg_live_out (bb),
+ df_get_subreg_live_full_out (bb),
+ df_get_subreg_live_partial_out (bb));
FOR_BB_INSNS_SAFE (bb, insn, next)
if (INSN_P (insn)
&& bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn)))
diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc
index 10e3d4e4097..9586e5602e4 100644
--- a/gcc/lra-constraints.cc
+++ b/gcc/lra-constraints.cc
@@ -6515,34 +6515,86 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail)
{
if (prev_bb != NULL)
{
- /* Update df_get_live_in (prev_bb): */
+ /* Update subreg live (prev_bb): */
+ bitmap subreg_all_in = df_get_subreg_live_in (prev_bb);
+ bitmap subreg_full_in = df_get_subreg_live_full_in (prev_bb);
+ bitmap subreg_partial_in = df_get_subreg_live_partial_in (prev_bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (prev_bb);
EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi)
if (bitmap_bit_p (&live_regs, j))
- bitmap_set_bit (df_get_live_in (prev_bb), j);
- else
- bitmap_clear_bit (df_get_live_in (prev_bb), j);
+ {
+ bitmap_set_bit (subreg_all_in, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_set_bit (subreg_full_in, j);
+ if (bitmap_bit_p (subreg_partial_in, j))
+ {
+ bitmap_clear_bit (subreg_partial_in, j);
+ range_in->remove_range (j);
+ }
+ }
+ }
+ else if (bitmap_bit_p (subreg_all_in, j))
+ {
+ bitmap_clear_bit (subreg_all_in, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (subreg_full_in, j);
+ if (bitmap_bit_p (subreg_partial_in, j))
+ {
+ bitmap_clear_bit (subreg_partial_in, j);
+ range_in->remove_range (j);
+ }
+ }
+ }
}
+ bitmap subreg_all_out = df_get_subreg_live_out (curr_bb);
if (curr_bb != last_bb)
{
- /* Update df_get_live_out (curr_bb): */
+ /* Update subreg live (curr_bb): */
+ bitmap subreg_full_out = df_get_subreg_live_full_out (curr_bb);
+ bitmap subreg_partial_out = df_get_subreg_live_partial_out (curr_bb);
+ subregs_live *range_out = df_get_subreg_live_range_out (curr_bb);
EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi)
{
live_p = bitmap_bit_p (&live_regs, j);
if (! live_p)
FOR_EACH_EDGE (e, ei, curr_bb->succs)
- if (bitmap_bit_p (df_get_live_in (e->dest), j))
+ if (bitmap_bit_p (df_get_subreg_live_in (e->dest), j))
{
live_p = true;
break;
}
if (live_p)
- bitmap_set_bit (df_get_live_out (curr_bb), j);
- else
- bitmap_clear_bit (df_get_live_out (curr_bb), j);
+ {
+ bitmap_set_bit (subreg_all_out, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_set_bit (subreg_full_out, j);
+ if (bitmap_bit_p (subreg_partial_out, j))
+ {
+ bitmap_clear_bit (subreg_partial_out, j);
+ range_out->remove_range (j);
+ }
+ }
+ }
+ else if (bitmap_bit_p (subreg_all_out, j))
+ {
+ bitmap_clear_bit (subreg_all_out, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (subreg_full_out, j);
+ if (bitmap_bit_p (subreg_partial_out, j))
+ {
+ bitmap_clear_bit (subreg_partial_out, j);
+ range_out->remove_range (j);
+ }
+ }
+ }
}
}
prev_bb = curr_bb;
- bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb));
+ bitmap_and (&live_regs, &check_only_regs, subreg_all_out);
}
if (! NONDEBUG_INSN_P (curr_insn))
continue;
@@ -6659,7 +6711,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res)
bitmap_clear (res);
FOR_EACH_EDGE (e, ei, from->succs)
if (e->dest != to)
- bitmap_ior_into (res, df_get_live_in (e->dest));
+ bitmap_ior_into (res, df_get_subreg_live_in (e->dest));
last = get_last_insertion_point (from);
if (! JUMP_P (last))
return;
@@ -6731,7 +6783,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail)
{
/* We are at the end of BB. Add qualified living
pseudos for potential splitting. */
- to_process = df_get_live_out (curr_bb);
+ to_process = df_get_subreg_live_out (curr_bb);
if (last_processed_bb != NULL)
{
/* We are somewhere in the middle of EBB. */
@@ -7103,7 +7155,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail)
{
/* We reached the beginning of the current block -- do
rest of spliting in the current BB. */
- to_process = df_get_live_in (curr_bb);
+ to_process = df_get_subreg_live_in (curr_bb);
if (BLOCK_FOR_INSN (head) != curr_bb)
{
/* We are somewhere in the middle of EBB. */
@@ -7180,7 +7232,7 @@ lra_inheritance (void)
fprintf (lra_dump_file, "EBB");
/* Form a EBB starting with BB. */
bitmap_clear (&ebb_global_regs);
- bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb));
+ bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_in (bb));
for (;;)
{
if (lra_dump_file != NULL)
@@ -7196,7 +7248,7 @@ lra_inheritance (void)
break;
bb = bb->next_bb;
}
- bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb));
+ bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_out (bb));
if (lra_dump_file != NULL)
fprintf (lra_dump_file, "\n");
if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb)))
@@ -7225,15 +7277,26 @@ int lra_undo_inheritance_iter;
/* Fix BB live info LIVE after removing pseudos created on pass doing
inheritance/split which are REMOVED_PSEUDOS. */
static void
-fix_bb_live_info (bitmap live, bitmap removed_pseudos)
+fix_bb_live_info (bitmap all, bitmap full, bitmap partial,
+ bitmap removed_pseudos)
{
unsigned int regno;
bitmap_iterator bi;
EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi)
- if (bitmap_clear_bit (live, regno)
- && REG_P (lra_reg_info[regno].restore_rtx))
- bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx));
+ {
+ if (bitmap_clear_bit (all, regno)
+ && REG_P (lra_reg_info[regno].restore_rtx))
+ {
+ bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx));
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (full, regno);
+ bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx));
+ gcc_assert (!bitmap_bit_p (partial, regno));
+ }
+ }
+ }
}
/* Return regno of the (subreg of) REG. Otherwise, return a negative
@@ -7299,8 +7362,12 @@ remove_inheritance_pseudos (bitmap remove_pseudos)
constraint pass. */
FOR_EACH_BB_FN (bb, cfun)
{
- fix_bb_live_info (df_get_live_in (bb), remove_pseudos);
- fix_bb_live_info (df_get_live_out (bb), remove_pseudos);
+ fix_bb_live_info (df_get_subreg_live_in (bb),
+ df_get_subreg_live_full_in (bb),
+ df_get_subreg_live_partial_in (bb), remove_pseudos);
+ fix_bb_live_info (df_get_subreg_live_out (bb),
+ df_get_subreg_live_full_out (bb),
+ df_get_subreg_live_partial_out (bb), remove_pseudos);
FOR_BB_INSNS_REVERSE (bb, curr_insn)
{
if (! INSN_P (curr_insn))
diff --git a/gcc/lra-int.h b/gcc/lra-int.h
index 5f605c3ae41..d73ff4e31b5 100644
--- a/gcc/lra-int.h
+++ b/gcc/lra-int.h
@@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_LRA_INT_H
#define GCC_LRA_INT_H
+#include "subreg-live-range.h"
+
#define lra_assert(c) gcc_checking_assert (c)
/* The parameter used to prevent infinite reloading for an insn. Each
@@ -161,6 +163,8 @@ struct lra_insn_reg
int regno;
/* Next reg info of the same insn. */
struct lra_insn_reg *next;
+ /* The corresponding reg or subreg RTX. */
+ rtx op;
};
/* Static part (common info for insns with the same ICODE) of LRA
diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc
index 66c6577e5d6..291f9420e06 100644
--- a/gcc/lra-lives.cc
+++ b/gcc/lra-lives.cc
@@ -272,8 +272,34 @@ update_pseudo_point (int regno, int point, enum point_type type)
}
}
-/* The corresponding bitmaps of BB currently being processed. */
-static bitmap bb_killed_pseudos, bb_gen_pseudos;
+/* Structure describing local BB data used for pseudo
+ live-analysis. */
+class bb_data_pseudos : public df_live_subreg_local_bb_info
+{
+public:
+ /* Basic block about which the below data are. */
+ basic_block bb;
+};
+
+/* Array for all BB data. Indexed by the corresponding BB index. */
+typedef class bb_data_pseudos *bb_data_t;
+
+/* All basic block data are referred through the following array. */
+static bb_data_t bb_data;
+
+/* The corresponding df local data of BB currently being processed. */
+static bb_data_t curr_bb_info;
+
+/* The regs which need to be tracked it's subreg liveness. */
+static bitmap_head tracked_regs;
+
+/* Return true if the REGNO need be track with subreg liveness. */
+
+static bool
+need_track_subreg_p (unsigned regno)
+{
+ return bitmap_bit_p (&tracked_regs, regno);
+}
/* Record hard register REGNO as now being live. It updates
living hard regs and START_LIVING. */
@@ -287,7 +313,7 @@ make_hard_regno_live (int regno)
SET_HARD_REG_BIT (hard_regs_live, regno);
sparseset_set_bit (start_living, regno);
if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno))
- bitmap_set_bit (bb_gen_pseudos, regno);
+ bitmap_set_bit (&curr_bb_info->full_use, regno);
}
/* Process the definition of hard register REGNO. This updates
@@ -310,8 +336,8 @@ make_hard_regno_dead (int regno)
sparseset_set_bit (start_dying, regno);
if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno))
{
- bitmap_clear_bit (bb_gen_pseudos, regno);
- bitmap_set_bit (bb_killed_pseudos, regno);
+ bitmap_clear_bit (&curr_bb_info->full_use, regno);
+ bitmap_set_bit (&curr_bb_info->full_def, regno);
}
}
@@ -343,7 +369,7 @@ mark_pseudo_dead (int regno)
/* Mark register REGNO (pseudo or hard register) in MODE as being live
and update BB_GEN_PSEUDOS. */
static void
-mark_regno_live (int regno, machine_mode mode)
+mark_regno_live (int regno, machine_mode mode, struct lra_insn_reg *reg)
{
int last;
@@ -355,15 +381,23 @@ mark_regno_live (int regno, machine_mode mode)
else
{
mark_pseudo_live (regno);
- bitmap_set_bit (bb_gen_pseudos, regno);
+ if (!need_track_subreg_p (regno))
+ bitmap_set_bit (&curr_bb_info->full_use, regno);
+ else
+ {
+ machine_mode reg_mode
+ = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op);
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (reg->op, range);
+ add_subreg_range (curr_bb_info, regno, range, false);
+ }
}
}
-
/* Mark register REGNO (pseudo or hard register) in MODE as being dead
and update BB_GEN_PSEUDOS and BB_KILLED_PSEUDOS. */
static void
-mark_regno_dead (int regno, machine_mode mode)
+mark_regno_dead (int regno, machine_mode mode, struct lra_insn_reg *reg)
{
int last;
@@ -375,8 +409,20 @@ mark_regno_dead (int regno, machine_mode mode)
else
{
mark_pseudo_dead (regno);
- bitmap_clear_bit (bb_gen_pseudos, regno);
- bitmap_set_bit (bb_killed_pseudos, regno);
+ if (!need_track_subreg_p (regno))
+ {
+ bitmap_clear_bit (&curr_bb_info->full_use, regno);
+ bitmap_set_bit (&curr_bb_info->full_def, regno);
+ }
+ else
+ {
+ machine_mode reg_mode
+ = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op);
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (reg->op, range);
+ remove_subreg_range (curr_bb_info, regno, range);
+ add_subreg_range (curr_bb_info, regno, range, true);
+ }
}
}
@@ -387,23 +433,6 @@ mark_regno_dead (int regno, machine_mode mode)
border. That might be a consequence of some global transformations
in LRA, e.g. PIC pseudo reuse or rematerialization. */
-/* Structure describing local BB data used for pseudo
- live-analysis. */
-class bb_data_pseudos
-{
-public:
- /* Basic block about which the below data are. */
- basic_block bb;
- bitmap_head killed_pseudos; /* pseudos killed in the BB. */
- bitmap_head gen_pseudos; /* pseudos generated in the BB. */
-};
-
-/* Array for all BB data. Indexed by the corresponding BB index. */
-typedef class bb_data_pseudos *bb_data_t;
-
-/* All basic block data are referred through the following array. */
-static bb_data_t bb_data;
-
/* Two small functions for access to the bb data. */
static inline bb_data_t
get_bb_data (basic_block bb)
@@ -429,14 +458,73 @@ static bitmap_head all_hard_regs_bitmap;
static bool
live_trans_fun (int bb_index)
{
- basic_block bb = get_bb_data_by_index (bb_index)->bb;
- bitmap bb_liveout = df_get_live_out (bb);
- bitmap bb_livein = df_get_live_in (bb);
- bb_data_t bb_info = get_bb_data (bb);
-
- bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap);
- return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos,
- &temp_bitmap, &bb_info->killed_pseudos);
+ bb_data_t local_bb_info = get_bb_data_by_index (bb_index);
+ bitmap full_in = df_get_subreg_live_full_in (local_bb_info->bb);
+ bitmap full_out = df_get_subreg_live_full_out (local_bb_info->bb);
+
+ bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap);
+ bool changed = bitmap_ior_and_compl (full_in, &local_bb_info->full_use,
+ &temp_bitmap, &local_bb_info->full_def);
+
+ /* Handle partial live case. */
+ if (flag_track_subreg_liveness)
+ {
+ bitmap partial_in = df_get_subreg_live_partial_in (local_bb_info->bb);
+ bitmap partial_out = df_get_subreg_live_partial_out (local_bb_info->bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (local_bb_info->bb);
+ subregs_live *range_out
+ = df_get_subreg_live_range_out (local_bb_info->bb);
+
+ if (!bitmap_empty_p (partial_out)
+ || !bitmap_empty_p (&local_bb_info->partial_use))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ bitmap_head temp_partial_out;
+ subregs_live temp_range_out;
+
+ /* TEMP = (OUT & ~DEF) */
+ bitmap_initialize (&temp_partial_out, &bitmap_default_obstack);
+ EXECUTE_IF_SET_IN_BITMAP (partial_out, FIRST_PSEUDO_REGISTER, regno,
+ bi)
+ {
+ sbitmap out_range = range_out->get_range (regno);
+ temp_range_out.add_range (regno, out_range);
+ if (bitmap_bit_p (&local_bb_info->partial_def, regno))
+ {
+ sbitmap def_range
+ = local_bb_info->range_def->get_range (regno);
+ temp_range_out.remove_range (regno, def_range);
+ if (!temp_range_out.empty_p (regno))
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+ else
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+
+ /* TEMP = USE | TEMP */
+ EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap use_range = local_bb_info->range_use->get_range (regno);
+ temp_range_out.add_range (regno, use_range);
+ }
+ bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use);
+
+ /* IN = TEMP */
+ changed |= range_in->copy_lives (temp_range_out);
+ bitmap_copy (partial_in, &temp_partial_out);
+ df_live_subreg_check_result (full_in, partial_in, range_in);
+ }
+ else if (!bitmap_empty_p (partial_in))
+ {
+ changed = true;
+ bitmap_clear (partial_in);
+ range_in->clear ();
+ }
+ }
+
+ return changed;
}
/* The confluence function used by the DF equation solver to set up
@@ -444,7 +532,9 @@ live_trans_fun (int bb_index)
static void
live_con_fun_0 (basic_block bb)
{
- bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap);
+ bitmap_and_into (df_get_subreg_live_out (bb), &all_hard_regs_bitmap);
+ if (flag_track_subreg_liveness)
+ bitmap_and_into (df_get_subreg_live_full_out (bb), &all_hard_regs_bitmap);
}
/* The confluence function used by the DF equation solver to propagate
@@ -456,13 +546,37 @@ live_con_fun_0 (basic_block bb)
static bool
live_con_fun_n (edge e)
{
- basic_block bb = e->src;
- basic_block dest = e->dest;
- bitmap bb_liveout = df_get_live_out (bb);
- bitmap dest_livein = df_get_live_in (dest);
+ bitmap src_full_out = df_get_subreg_live_full_out (e->src);
+ bitmap dest_full_in = df_get_subreg_live_full_in (e->dest);
+
+ bool changed = bitmap_ior_and_compl_into (src_full_out, dest_full_in,
+ &all_hard_regs_bitmap);
+ /* Handle partial live case. */
+ if (flag_track_subreg_liveness)
+ {
+ bitmap src_partial_out = df_get_subreg_live_partial_out (e->src);
+ subregs_live *src_range_out = df_get_subreg_live_range_out (e->src);
+ bitmap dest_partial_in = df_get_subreg_live_partial_in (e->dest);
+ subregs_live *dest_range_in = df_get_subreg_live_range_in (e->dest);
+
+ if (bitmap_empty_p (dest_partial_in))
+ return changed;
+
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (dest_partial_in, FIRST_PSEUDO_REGISTER, regno,
+ bi)
+ {
+ sbitmap dest_range = dest_range_in->get_range (regno);
+ changed |= src_range_out->add_range (regno, dest_range);
+ }
+ changed |= bitmap_ior_into (src_partial_out, dest_partial_in);
- return bitmap_ior_and_compl_into (bb_liveout,
- dest_livein, &all_hard_regs_bitmap);
+ df_live_subreg_check_result (src_full_out, src_partial_out,
+ src_range_out);
+ }
+
+ return changed;
}
/* Indexes of all function blocks. */
@@ -479,12 +593,47 @@ initiate_live_solver (void)
bitmap_initialize (&all_blocks, ®_obstack);
basic_block bb;
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_initialize (&tracked_regs, ®_obstack);
+ FOR_ALL_BB_FN (bb, cfun)
+ {
+ rtx_insn *insn;
+ df_ref use;
+ FOR_BB_INSNS (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* A multireg which is used via subreg pattern. */
+ if (multireg_p (regno)
+ && DF_REF_FLAGS (use) & (DF_REF_SUBREG))
+ bitmap_set_bit (&tracked_regs, regno);
+ }
+ }
+ }
+ }
+
+
FOR_ALL_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
bb_info->bb = bb;
- bitmap_initialize (&bb_info->killed_pseudos, ®_obstack);
- bitmap_initialize (&bb_info->gen_pseudos, ®_obstack);
+ bitmap_initialize (&bb_info->full_def, ®_obstack);
+ bitmap_initialize (&bb_info->full_use, ®_obstack);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_initialize (&bb_info->partial_def, ®_obstack);
+ bitmap_initialize (&bb_info->partial_use, ®_obstack);
+ size_t num_regs = bitmap_count_bits (&tracked_regs);
+ bb_info->range_def = new subregs_live (num_regs);
+ bb_info->range_use = new subregs_live (num_regs);
+ }
bitmap_set_bit (&all_blocks, bb->index);
}
}
@@ -496,11 +645,19 @@ finish_live_solver (void)
basic_block bb;
bitmap_clear (&all_blocks);
+ bitmap_clear (&tracked_regs);
FOR_ALL_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
- bitmap_clear (&bb_info->killed_pseudos);
- bitmap_clear (&bb_info->gen_pseudos);
+ bitmap_clear (&bb_info->full_def);
+ bitmap_clear (&bb_info->full_use);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear (&bb_info->partial_def);
+ bitmap_clear (&bb_info->partial_use);
+ delete bb_info->range_def;
+ delete bb_info->range_use;
+ }
}
free (bb_data);
bitmap_clear (&all_hard_regs_bitmap);
@@ -663,7 +820,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
/* Only has a meaningful value once we've seen a call. */
function_abi last_call_abi = default_function_abi;
- reg_live_out = df_get_live_out (bb);
+ reg_live_out = df_get_subreg_live_out (bb);
sparseset_clear (pseudos_live);
sparseset_clear (pseudos_live_through_calls);
sparseset_clear (pseudos_live_through_setjumps);
@@ -675,10 +832,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
mark_pseudo_live (j);
}
- bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos;
- bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos;
- bitmap_clear (bb_gen_pseudos);
- bitmap_clear (bb_killed_pseudos);
+ curr_bb_info = get_bb_data (bb);
+ bitmap_clear (&curr_bb_info->full_use);
+ bitmap_clear (&curr_bb_info->full_def);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear (&curr_bb_info->partial_use);
+ bitmap_clear (&curr_bb_info->partial_def);
+ curr_bb_info->range_use->clear ();
+ curr_bb_info->range_def->clear ();
+ }
freq = REG_FREQ_FROM_BB (bb);
if (lra_dump_file != NULL)
@@ -860,7 +1023,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
if (reg->type != OP_IN)
{
update_pseudo_point (reg->regno, curr_point, USE_POINT);
- mark_regno_live (reg->regno, reg->biggest_mode);
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
/* ??? Should be a no-op for unused registers. */
check_pseudos_live_through_calls (reg->regno, last_call_abi);
}
@@ -886,7 +1049,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_OUT)
update_pseudo_point (reg->regno, curr_point, DEF_POINT);
- mark_regno_dead (reg->regno, reg->biggest_mode);
+ mark_regno_dead (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -945,8 +1108,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_IN)
update_pseudo_point (reg->regno, curr_point, USE_POINT);
- mark_regno_live (reg->regno, reg->biggest_mode);
check_pseudos_live_through_calls (reg->regno, last_call_abi);
+ /* Ignore the use of subreg which is used as dest operand. */
+ if (need_track_subreg_p (reg->regno) && reg->subreg_p
+ && reg->type == OP_INOUT)
+ continue;
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -970,12 +1137,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_OUT)
update_pseudo_point (reg->regno, curr_point, DEF_POINT);
- mark_regno_dead (reg->regno, reg->biggest_mode);
+ mark_regno_dead (reg->regno, reg->biggest_mode, reg);
/* We're done processing inputs, so make sure early clobber
operands that are both inputs and outputs are still live. */
if (reg->type == OP_INOUT)
- mark_regno_live (reg->regno, reg->biggest_mode);
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -1099,8 +1266,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
bool live_change_p = false;
/* Check if bb border live info was changed. */
unsigned int live_pseudos_num = 0;
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb),
- FIRST_PSEUDO_REGISTER, j, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ j, bi)
{
live_pseudos_num++;
if (! sparseset_bit_p (pseudos_live, j))
@@ -1118,7 +1285,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
live_change_p = true;
if (lra_dump_file != NULL)
EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j)
- if (! bitmap_bit_p (df_get_live_in (bb), j))
+ if (! bitmap_bit_p (df_get_subreg_live_in (bb), j))
fprintf (lra_dump_file,
" r%d is added to live at bb%d start\n", j, bb->index);
}
@@ -1133,7 +1300,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
mark_pseudo_dead (i);
}
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ j, bi)
{
if (sparseset_cardinality (pseudos_live_through_calls) == 0)
break;
@@ -1149,7 +1317,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i))
continue;
- if (bitmap_bit_p (df_get_live_in (bb), i))
+ if (bitmap_bit_p (df_get_subreg_live_in (bb), i))
continue;
live_change_p = true;
@@ -1157,7 +1325,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
fprintf (lra_dump_file,
" hard reg r%d is added to live at bb%d start\n", i,
bb->index);
- bitmap_set_bit (df_get_live_in (bb), i);
+ bitmap_set_bit (df_get_subreg_live_in (bb), i);
+ if (flag_track_subreg_liveness)
+ bitmap_set_bit (df_get_subreg_live_full_in (bb), i);
}
if (need_curr_point_incr)
@@ -1421,12 +1591,30 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
{
/* We need to clear pseudo live info as some pseudos can
disappear, e.g. pseudos with used equivalences. */
- FOR_EACH_BB_FN (bb, cfun)
+ FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun),
+ EXIT_BLOCK_PTR_FOR_FN (cfun), next_bb)
{
- bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER,
+ bitmap_clear_range (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
max_regno - FIRST_PSEUDO_REGISTER);
- bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER,
+ bitmap_clear_range (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER,
max_regno - FIRST_PSEUDO_REGISTER);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_range (df_get_subreg_live_full_in (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_partial_in (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_full_out (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_partial_out (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ df_get_subreg_live_range_in (bb)->clear ();
+ df_get_subreg_live_range_out (bb)->clear ();
+ }
}
/* As we did not change CFG since LRA start we can use
DF-infrastructure solver to solve live data flow problem. */
@@ -1439,6 +1627,10 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
(DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n,
live_trans_fun, &all_blocks,
df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD));
+
+ if (flag_track_subreg_liveness)
+ df_live_subreg_finalize (&all_blocks);
+
if (lra_dump_file != NULL)
{
fprintf (lra_dump_file,
@@ -1447,16 +1639,33 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
FOR_EACH_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
- bitmap bb_livein = df_get_live_in (bb);
- bitmap bb_liveout = df_get_live_out (bb);
fprintf (lra_dump_file, "\nBB %d:\n", bb->index);
- lra_dump_bitmap_with_title (" gen:",
- &bb_info->gen_pseudos, bb->index);
- lra_dump_bitmap_with_title (" killed:",
- &bb_info->killed_pseudos, bb->index);
- lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index);
- lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index);
+ lra_dump_bitmap_with_title (" full use", &bb_info->full_use,
+ bb->index);
+ lra_dump_bitmap_with_title (" full def", &bb_info->full_def,
+ bb->index);
+ lra_dump_bitmap_with_title (" live in full",
+ df_get_subreg_live_full_in (bb),
+ bb->index);
+ lra_dump_bitmap_with_title (" live out full",
+ df_get_subreg_live_full_out (bb),
+ bb->index);
+ if (flag_track_subreg_liveness)
+ {
+ lra_dump_bitmap_with_title (" partial use",
+ &bb_info->partial_use, bb->index);
+ lra_dump_bitmap_with_title (" partial def",
+ &bb_info->partial_def, bb->index);
+ lra_dump_bitmap_with_title (" live in partial",
+ df_get_subreg_live_partial_in (
+ bb),
+ bb->index);
+ lra_dump_bitmap_with_title (" live out partial",
+ df_get_subreg_live_partial_out (
+ bb),
+ bb->index);
+ }
}
}
}
diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc
index c84bf3c9938..ef0157513b0 100644
--- a/gcc/lra-remat.cc
+++ b/gcc/lra-remat.cc
@@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void)
fprintf (lra_dump_file, "\nBB %d:\n", bb->index);
/* Livein */
fprintf (lra_dump_file, " register live in:");
- dump_regset (df_get_live_in (bb), lra_dump_file);
+ dump_regset (df_get_subreg_live_in (bb), lra_dump_file);
putc ('\n', lra_dump_file);
/* Liveout */
fprintf (lra_dump_file, " register live out:");
- dump_regset (df_get_live_out (bb), lra_dump_file);
+ dump_regset (df_get_subreg_live_out (bb), lra_dump_file);
putc ('\n', lra_dump_file);
/* Changed/dead regs: */
fprintf (lra_dump_file, " changed regs:");
@@ -727,7 +727,7 @@ calculate_livein_cands (void)
FOR_EACH_BB_FN (bb, cfun)
{
- bitmap livein_regs = df_get_live_in (bb);
+ bitmap livein_regs = df_get_subreg_live_in (bb);
bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands;
for (unsigned int i = 0; i < cands_num; i++)
{
@@ -1064,7 +1064,7 @@ do_remat (void)
FOR_EACH_BB_FN (bb, cfun)
{
CLEAR_HARD_REG_SET (live_hard_regs);
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), 0, regno, bi)
{
int hard_regno = regno < FIRST_PSEUDO_REGISTER
? regno
diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc
index 8b1a945d632..08ec247bc00 100644
--- a/gcc/lra-spills.cc
+++ b/gcc/lra-spills.cc
@@ -566,8 +566,31 @@ spill_pseudos (void)
"Debug insn #%u is reset because it referenced "
"removed pseudo\n", INSN_UID (insn));
}
- bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos);
- bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos);
+ bitmap_and_compl_into (df_get_subreg_live_in (bb), spilled_pseudos);
+ bitmap_and_compl_into (df_get_subreg_live_out (bb), spilled_pseudos);
+
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_and_compl_into (df_get_subreg_live_full_in (bb),
+ spilled_pseudos);
+ bitmap partial_in = df_get_subreg_live_partial_in (bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (bb);
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ range_in->remove_range (regno);
+ bitmap_and_compl_into (partial_in, spilled_pseudos);
+
+ bitmap_and_compl_into (df_get_subreg_live_full_out (bb),
+ spilled_pseudos);
+ bitmap partial_out = df_get_subreg_live_partial_out (bb);
+ subregs_live *range_out = df_get_subreg_live_range_out (bb);
+ EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ range_out->remove_range (regno);
+ bitmap_and_compl_into (partial_out, spilled_pseudos);
+ }
}
}
}
diff --git a/gcc/lra.cc b/gcc/lra.cc
index fb32e134004..a6fabe7d118 100644
--- a/gcc/lra.cc
+++ b/gcc/lra.cc
@@ -574,7 +574,7 @@ object_allocator<lra_insn_reg> lra_insn_reg_pool ("insn regs");
EARLY_CLOBBER_ALTS. */
static struct lra_insn_reg *
new_insn_reg (rtx_insn *insn, int regno, enum op_type type,
- machine_mode mode, bool subreg_p,
+ machine_mode mode, bool subreg_p, rtx op,
alternative_mask early_clobber_alts,
struct lra_insn_reg *next)
{
@@ -586,6 +586,7 @@ new_insn_reg (rtx_insn *insn, int regno, enum op_type type,
ir->subreg_p = subreg_p;
ir->early_clobber_alts = early_clobber_alts;
ir->regno = regno;
+ ir->op = op;
ir->next = next;
return ir;
}
@@ -926,7 +927,7 @@ collect_non_operand_hard_regs (rtx_insn *insn, rtx *x,
&& ! (FIRST_STACK_REG <= regno
&& regno <= LAST_STACK_REG));
#endif
- list = new_insn_reg (data->insn, regno, type, mode, subreg_p,
+ list = new_insn_reg (data->insn, regno, type, mode, subreg_p, *x,
early_clobber ? ALL_ALTERNATIVES : 0, list);
}
}
@@ -1484,6 +1485,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
code = GET_CODE (x);
mode = GET_MODE (x);
subreg_p = false;
+ rtx op = x;
if (GET_CODE (x) == SUBREG)
{
mode = wider_subreg_mode (x);
@@ -1501,7 +1503,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
if (bitmap_set_bit (&lra_reg_info[regno].insn_bitmap, INSN_UID (insn)))
{
data->regs = new_insn_reg (data->insn, regno, type, mode, subreg_p,
- early_clobber_alts, data->regs);
+ op, early_clobber_alts, data->regs);
return;
}
else
@@ -1513,7 +1515,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
/* The info cannot be integrated into the found
structure. */
data->regs = new_insn_reg (data->insn, regno, type, mode,
- subreg_p, early_clobber_alts,
+ subreg_p, op, early_clobber_alts,
data->regs);
else
{
--
2.25.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/4] df: Add DF_LIVE_SUBREG problem
2024-04-24 10:05 ` [PATCH 2/4] df: Add DF_LIVE_SUBREG problem Lehua Ding
@ 2024-04-25 20:56 ` Dimitar Dimitrov
0 siblings, 0 replies; 6+ messages in thread
From: Dimitar Dimitrov @ 2024-04-25 20:56 UTC (permalink / raw)
To: Lehua Ding
Cc: gcc-patches, juzhe.zhong, shuo.chen, jin.xia, vmakarov,
richard.sandiford
On Wed, Apr 24, 2024 at 06:05:03PM +0800, Lehua Ding wrote:
> This patch add a new DF problem, named DF_LIVE_SUBREG. This problem
> is extended from the DF_LR problem and support track the subreg liveness
> of multireg pseudo if these pseudo satisfy the following conditions:
>
> 1. the mode size greater than it's REGMODE_NATURAL_SIZE.
> 2. the reg is used in insns via subreg pattern.
>
> The main methods are as follows:
>
> 1. split bitmap in/out/def/use fileds to full_in/out/def/use and
> partial_in/out/def/use. If a pseudo need to be tracked it's subreg
> liveness, then it is recorded in partial_in/out/def/use fileds.
> Meantimes, there are range_in/out/def/use fileds which records the live
> range of the tracked pseudo.
> 2. in the df_live_subreg_finalize function, we move the tracked pseudo from
> the partial_in/out/def/use to full_in/out/def/use if the pseudo's live
> range is full.
Hi Lehua,
I'm not familiar with LRA, so my comments bellow could be totally off
point. Please treat them as mild suggestions.
>
> gcc/ChangeLog:
>
> * Makefile.in: Add subreg-live-range object file.
> * df-problems.cc (struct df_live_subreg_problem_data): Private struct
> for DF_LIVE_SUBREG problem.
> (df_live_subreg_get_bb_info): getting bb regs in/out data.
> (get_live_subreg_local_bb_info): getting bb regs use/def data.
> (multireg_p): checking is the regno a pseudo multireg.
> (need_track_subreg_p): checking is the regno need to be tracked.
> (init_range): getting the range of subreg rtx.
> (remove_subreg_range): removing use data for the reg/subreg rtx.
> (add_subreg_range): adding def/use data for the reg/subreg rtx.
> (df_live_subreg_free_bb_info): Free basic block df data.
> (df_live_subreg_alloc): Allocate and init df data.
> (df_live_subreg_reset): Reset the live in/out df data.
> (df_live_subreg_bb_local_compute): Compute basic block df data.
> (df_live_subreg_local_compute): Compute all basic blocks df data.
> (df_live_subreg_init): Init the in/out df data.
> (df_live_subreg_check_result): Assert the full and partial df data.
> (df_live_subreg_confluence_0): Confluence function for infinite loops.
> (df_live_subreg_confluence_n): Confluence function for normal edge.
> (df_live_subreg_transfer_function): Transfer function.
> (df_live_subreg_finalize): Finalize the all_in/all_out df data.
> (df_live_subreg_free): Free the df data.
> (df_live_subreg_top_dump): Dump top df data.
> (df_live_subreg_bottom_dump): Dump bottom df data.
> (df_live_subreg_add_problem): Add the DF_LIVE_SUBREG problem.
> * df.h (enum df_problem_id): Add DF_LIVE_SUBREG.
> (class subregs_live): Simple decalare.
> (class df_live_subreg_local_bb_info): New class for full/partial def/use
> df data.
> (class df_live_subreg_bb_info): New class for full/partial in/out
> df data.
> (df_live_subreg): getting the df_live_subreg data.
> (df_live_subreg_add_problem): Exported.
> (df_live_subreg_finalize): Ditto.
> (df_live_subreg_check_result): Ditto.
> (multireg_p): Ditto.
> (init_range): Ditto.
> (add_subreg_range): Ditto.
> (remove_subreg_range): Ditto.
> (df_get_subreg_live_in): Accessor the all_in df data.
> (df_get_subreg_live_out): Accessor the all_out df data.
> (df_get_subreg_live_full_in): Accessor the full_in df data.
> (df_get_subreg_live_full_out): Accessor the full_out df data.
> (df_get_subreg_live_partial_in): Accessor the partial_in df data.
> (df_get_subreg_live_partial_out): Accessor the partial_out df data.
> (df_get_subreg_live_range_in): Accessor the range_in df data.
> (df_get_subreg_live_range_out): Accessor the range_out df data.
> * regs.h (get_nblocks): Get the blocks of mode.
> * sbitmap.cc (bitmap_full_p): sbitmap predicator.
> (bitmap_same_p): sbitmap predicator.
> (test_full): test bitmap_full_p.
> (test_same): test bitmap_same_p.
> (sbitmap_cc_tests): Add test_full and test_same.
> * sbitmap.h (bitmap_full_p): Exported.
> (bitmap_same_p): Ditto.
> * timevar.def (TV_DF_LIVE_SUBREG): add DF_LIVE_SUBREG timevar.
> * subreg-live-range.cc: New file.
> * subreg-live-range.h: New file.
> ---
> gcc/Makefile.in | 1 +
> gcc/df-problems.cc | 855 ++++++++++++++++++++++++++++++++++++++-
> gcc/df.h | 155 +++++++
> gcc/regs.h | 5 +
> gcc/sbitmap.cc | 98 +++++
> gcc/sbitmap.h | 2 +
> gcc/subreg-live-range.cc | 53 +++
> gcc/subreg-live-range.h | 206 ++++++++++
> gcc/timevar.def | 1 +
> 9 files changed, 1375 insertions(+), 1 deletion(-)
> create mode 100644 gcc/subreg-live-range.cc
> create mode 100644 gcc/subreg-live-range.h
>
> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
> index a74761b7ab3..e195238f6ab 100644
> --- a/gcc/Makefile.in
> +++ b/gcc/Makefile.in
> @@ -1684,6 +1684,7 @@ OBJS = \
> store-motion.o \
> streamer-hooks.o \
> stringpool.o \
> + subreg-live-range.o \
> substring-locations.o \
> target-globals.o \
> targhooks.o \
> diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc
> index 88ee0dd67fc..93c5b52146c 100644
> --- a/gcc/df-problems.cc
> +++ b/gcc/df-problems.cc
> @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see
> #include "target.h"
> #include "rtl.h"
> #include "df.h"
> +#include "subreg-live-range.h"
> #include "memmodel.h"
> #include "tm_p.h"
> #include "insn-config.h"
> @@ -1344,8 +1345,860 @@ df_lr_verify_transfer_functions (void)
> bitmap_clear (&all_blocks);
> }
>
> +/*----------------------------------------------------------------------------
> + REGISTER AND SUBREGS LIVES
> + Like DF_LR, but include tracking subreg liveness. Currently used to provide
Nit: Should have two spaces after the dot, per GNU coding style. I'd suggest
to run the contrib/check_GNU_style.py script on your patches.
> + subreg liveness related information to the register allocator. The subreg
> + information is currently tracked for registers that satisfy the following
> + conditions:
> + 1. REG is a pseudo register
> + 2. MODE_SIZE > UNIT_SIZE
> + 3. MODE_SIZE is a multiple of UNIT_SIZE
> + 4. REG is used via subreg pattern
> + Assuming: MODE = the machine mode of the REG
> + MODE_SIZE = GET_MODE_SIZE (MODE)
> + UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE)
> + Condition 3 is currently strict, maybe it can be removed in the future, but
> + for now it is sufficient.
> + ----------------------------------------------------------------------------*/
> +
> +/* These two empty data are used as default data in case the user does not turn
> + * on the track-subreg-liveness feature. */
> +bitmap_head empty_bitmap;
> +subregs_live empty_live;
These names seem a bit too short for global variables. Perhaps tuck
them in a namespace?
Also, since these must remain empty, shouldn't they be declared as const?
namespace df {
const bitmap_head empty_bitmap;
const subregs_live empty_live;
}
> +
> +/* Private data for live_subreg problem. */
> +struct df_live_subreg_problem_data
> +{
> + /* Record registers that need to track subreg liveness. */
> + bitmap_head tracked_regs;
> + /* An obstack for the bitmaps we need for this problem. */
> + bitmap_obstack live_subreg_bitmaps;
> +};
> +
> +/* Helper functions */
> +
> +static df_live_subreg_bb_info *
> +df_live_subreg_get_bb_info (unsigned int index)
> +{
> + if (index < df_live_subreg->block_info_size)
> + return &(
> + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[index];
Please consider using C++ instead:
return &static_cast<df_live_subreg_bb_info *>(df_live_subreg->block_info)[index];
> + else
> + return NULL;
Use nullptr instead?
> +}
> +
> +static df_live_subreg_local_bb_info *
> +get_live_subreg_local_bb_info (unsigned int bb_index)
> +{
> + return df_live_subreg_get_bb_info (bb_index);
> +}
> +
> +/* Return true if regno is a multireg. */
> +bool
> +multireg_p (int regno)
> +{
> + if (regno < FIRST_PSEUDO_REGISTER)
> + return false;
> + rtx regno_rtx = regno_reg_rtx[regno];
> + machine_mode reg_mode = GET_MODE (regno_rtx);
> + poly_int64 total_size = GET_MODE_SIZE (reg_mode);
> + poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode);
> + return maybe_gt (total_size, natural_size)
> + && multiple_p (total_size, natural_size);
> +}
> +
> +/* Return true if the REGNO need be track with subreg liveness. */
> +
> +static bool
> +need_track_subreg_p (unsigned regno)
> +{
> + struct df_live_subreg_problem_data *problem_data
> + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
Here auto could be used:
auto problem_data = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
> + return bitmap_bit_p (&problem_data->tracked_regs, regno);
> +}
> +
> +/* Return the subreg_range of REF. */
Is this comment valid? I see neither a REF parameter, nor a returned
value. Did you perhaps mean:
/* Fill RANGE with the subreg range for OP in REGMODE_NATURAL_SIZE
granularity. */
> +void
> +init_range (rtx op, sbitmap range)
> +{
> + rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op;
> + machine_mode reg_mode = GET_MODE (reg);
> +
> + if (!read_modify_subreg_p (op))
> + {
> + bitmap_set_range (range, 0, get_nblocks (reg_mode));
> + return;
> + }
> +
> + rtx subreg = op;
> + machine_mode subreg_mode = GET_MODE (subreg);
> + poly_int64 offset = SUBREG_BYTE (subreg);
> + int nblocks = get_nblocks (reg_mode);
> + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode);
> + poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode);
> + poly_int64 left = offset + subreg_size;
> +
> + int subreg_start = -1;
> + int subreg_nblocks = -1;
> + for (int i = 0; i < nblocks; i += 1)
> + {
> + poly_int64 right = unit_size * (i + 1);
> + if (subreg_start < 0 && maybe_lt (offset, right))
> + subreg_start = i;
> + if (subreg_nblocks < 0 && maybe_le (left, right))
> + {
> + subreg_nblocks = i + 1 - subreg_start;
> + break;
> + }
> + }
> + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0);
> +
> + bitmap_set_range (range, subreg_start, subreg_nblocks);
> +}
> +
> +/* Remove RANGE from BB_INFO's use data. */
> +void
> +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
> + const_sbitmap range)
> +{
> + bitmap partial = &bb_info->partial_use;
> +
> + bb_info->range_use->remove_range (regno, range);
> + if (bb_info->range_use->empty_p (regno))
> + bitmap_clear_bit (partial, regno);
> +}
> +
> +/* Remove RANGE of REF from BB_INFO's use data. */
> +static void
> +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref)
> +{
> + unsigned int regno = DF_REF_REGNO (ref);
> + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
> +
> + if (need_track_subreg_p (regno))
> + {
> + auto_sbitmap range (get_nblocks (reg_mode));
> + init_range (DF_REF_REG (ref), range);
> + remove_subreg_range (bb_info, regno, range);
> + }
> + else
> + {
> + bitmap_clear_bit (&bb_info->full_use, regno);
> + gcc_assert (!bitmap_bit_p (&bb_info->partial_use, regno));
> + gcc_assert (!bitmap_bit_p (&bb_info->partial_def, regno));
> + }
> +}
> +
> +/* add RANGE to BB_INFO's def/use. If is_def is true, means for BB_INFO's def,
> + otherwise for BB_INFO's use. */
> +void
> +add_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
> + const_sbitmap range, bool is_def)
Would it be more clear to split into two separate functions
instead of using a boolean flag? E.g.:
add_subreg_range_to_def (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
const_sbitmap range)
add_subreg_range_to_use (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
const_sbitmap range)
> +{
> + bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use;
> + subregs_live *range_live = is_def ? bb_info->range_def : bb_info->range_use;
> +
> + bitmap_set_bit (partial, regno);
> + range_live->add_range (regno, range);
> +}
> +
> +/* add RANGE of REF to BB_INFO def/use. If is_def is true, means for BB_INFO's
> + def, otherwise for BB_INFO's use. */
> +static void
> +add_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref,
> + bool is_def)
> +{
> + unsigned int regno = DF_REF_REGNO (ref);
> + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
> +
> + if (need_track_subreg_p (regno))
> + {
> + auto_sbitmap range (get_nblocks (reg_mode));
> + init_range (DF_REF_REG (ref), range);
> + add_subreg_range (bb_info, regno, range, is_def);
> + }
> + else
> + {
> + bitmap full = is_def ? &bb_info->full_def : &bb_info->full_use;
> + bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use;
> + bitmap_set_bit (full, regno);
> + gcc_assert (!bitmap_bit_p (partial, regno));
> + }
> +}
> +
> +/* Free basic block info. */
> +
> +static void
> +df_live_subreg_free_bb_info (basic_block bb ATTRIBUTE_UNUSED, void *vbb_info)
> +{
> + df_live_subreg_bb_info *bb_info = (df_live_subreg_bb_info *) vbb_info;
Perhaps:
auto bb_info = static_cast<df_live_subreg_bb_info *> vbb_info;
> + if (bb_info)
> + {
> + delete bb_info->range_in;
> + bb_info->range_in = NULL;
> + delete bb_info->range_out;
> + bb_info->range_out = NULL;
> +
> + bitmap_clear (&bb_info->all_in);
> + bitmap_clear (&bb_info->full_in);
> + bitmap_clear (&bb_info->partial_in);
> + bitmap_clear (&bb_info->all_out);
> + bitmap_clear (&bb_info->full_out);
> + bitmap_clear (&bb_info->partial_out);
> + }
> +}
> +
> +/* Allocate or reset bitmaps for DF_LIVE_SUBREG blocks. The solution bits are
> + not touched unless the block is new. */
> +
> +static void
> +df_live_subreg_alloc (bitmap all_blocks ATTRIBUTE_UNUSED)
> +{
> + struct df_live_subreg_problem_data *problem_data;
> + df_grow_bb_info (df_live_subreg);
> + if (df_live_subreg->problem_data)
> + problem_data
> + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
> + else
> + {
> + problem_data = XNEW (struct df_live_subreg_problem_data);
> + df_live_subreg->problem_data = problem_data;
> +
> + bitmap_obstack_initialize (&problem_data->live_subreg_bitmaps);
> + bitmap_initialize (&problem_data->tracked_regs,
> + &problem_data->live_subreg_bitmaps);
> + }
> +
> + bitmap_clear (&problem_data->tracked_regs);
> +
> + basic_block bb;
> + FOR_EACH_BB_FN (bb, cfun)
> + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, bb->index);
> +
> + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, ENTRY_BLOCK);
> + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, EXIT_BLOCK);
> +
> + unsigned int bb_index;
> + bitmap_iterator bi;
> + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
> + bb_index, bi)
> + {
> + /* Find the regs which we need to track it's subreg liveness. */
> + rtx_insn *insn;
> + df_ref use;
> + FOR_BB_INSNS (bb, insn)
> + {
> + if (!NONDEBUG_INSN_P (insn))
> + continue;
> +
> + df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
> +
> + FOR_EACH_INSN_INFO_USE (use, insn_info)
> + {
> + unsigned int regno = DF_REF_REGNO (use);
> + /* A multireg which is used via subreg pattern. */
> + if (multireg_p (regno)
> + && DF_REF_FLAGS (use) & (DF_REF_SUBREG))
> + bitmap_set_bit (&problem_data->tracked_regs, regno);
> + }
> + }
> + }
> +
> + if (dump_file)
> + {
> + fprintf (dump_file, ";; regs need to be tracked subreg liveness: ");
> + df_print_regset (dump_file, &problem_data->tracked_regs);
> + }
> +
> + size_t n = bitmap_count_bits (&problem_data->tracked_regs);
> +
> + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
> + bb_index, bi)
> + {
> + /* Clean global infos. */
> + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
Here and for the rest of the call sites, I think auto is allowed to be used:
auto bb_info = df_live_subreg_get_bb_info (bb_index);
Regards,
Dimitar
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-04-25 20:56 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-24 10:05 [PATCH V2 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Lehua Ding
2024-04-24 10:05 ` [PATCH 1/4] df: Add -ftrack-subreg-liveness option Lehua Ding
2024-04-24 10:05 ` [PATCH 2/4] df: Add DF_LIVE_SUBREG problem Lehua Ding
2024-04-25 20:56 ` Dimitar Dimitrov
2024-04-24 10:05 ` [PATCH 3/4] ira: Apply DF_LIVE_SUBREG data Lehua Ding
2024-04-24 10:05 ` [PATCH 4/4] lra: " Lehua Ding
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).