* [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA
@ 2024-05-12 22:57 Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Juzhe-Zhong
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Juzhe-Zhong @ 2024-05-12 22:57 UTC (permalink / raw)
To: gcc-patches
Cc: vmakarov, richard.sandiford, jin.xia, dimitar, Juzhe-Zhong, Lehua Ding
V3: Address comments from Dimitar Dimitrov
V4: Move detailed function from subreg-live-range.h to subreg-live-range.cc.
These patches are used to add a new data flow DF_LIVE_SUBREG,
which will track subreg liveness and then apply it to IRA and LRA
passes (enabled via -O3 or -ftrack-subreg-liveness). These patches
are for GCC 15. And these codes are pushed to the devel/subreg-coalesce
branch. In addition, my colleague Shuo Chen will also be involved in some
of the remain work, thank you for your support.
These patches are separated from the subreg-coalesce patches submitted
a few months ago. I refactored the code according to comments. The next
patches will support subreg coalesce base on they. Here are some data
abot build time of SPEC INT 2017 (x86-64 target):
baseline baseline(+track-subreg-liveness)
specint2017 build time : 1892s 1883s
Regarding build times, I've run it a few times, but they all seem to take
much less time. Since the difference is small, it's possible that it's just
a change in environment. But it's theoretically possible, since supporting
subreg-liveness could have reduced the number of living regs.
For memory usage, I trided PR 69609 by valgrind, peak memory size grow from
2003910656 to 2003947520, very small increase.
Note that these patches don't enable register coalesce with subreg liveness in IRA/LRA,
so no performance change as expected.
And we will enable register coalsece with subreg liveness tracking in the followup patches.
Bootstrap and Regtested on x86-64 no regression.
Co-authored-by: Lehua Ding <lehua.ding@rivai.ai>
Juzhe-Zhong (4):
DF: Add -ftrack-subreg-liveness option
DF: Add DF_LIVE_SUBREG problem
IRA: Apply DF_LIVE_SUBREG data
LRA: Apply DF_LIVE_SUBREG data
gcc/Makefile.in | 1 +
gcc/common.opt | 4 +
gcc/common.opt.urls | 3 +
gcc/df-problems.cc | 886 ++++++++++++++++++++++++++++++++++++++-
gcc/df.h | 159 +++++++
gcc/doc/invoke.texi | 8 +
gcc/ira-build.cc | 7 +-
gcc/ira-color.cc | 8 +-
gcc/ira-emit.cc | 12 +-
gcc/ira-lives.cc | 7 +-
gcc/ira.cc | 19 +-
gcc/lra-coalesce.cc | 27 +-
gcc/lra-constraints.cc | 109 ++++-
gcc/lra-int.h | 4 +
gcc/lra-lives.cc | 357 ++++++++++++----
gcc/lra-remat.cc | 8 +-
gcc/lra-spills.cc | 27 +-
gcc/lra.cc | 10 +-
gcc/opts.cc | 1 +
gcc/regs.h | 5 +
gcc/sbitmap.cc | 98 +++++
gcc/sbitmap.h | 2 +
gcc/subreg-live-range.cc | 233 ++++++++++
gcc/subreg-live-range.h | 60 +++
gcc/timevar.def | 1 +
25 files changed, 1920 insertions(+), 136 deletions(-)
create mode 100644 gcc/subreg-live-range.cc
create mode 100644 gcc/subreg-live-range.h
--
2.36.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option
2024-05-12 22:57 [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Juzhe-Zhong
@ 2024-05-12 22:57 ` Juzhe-Zhong
2024-06-13 15:54 ` Richard Sandiford
2024-05-12 22:57 ` [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem Juzhe-Zhong
` (2 subsequent siblings)
3 siblings, 1 reply; 7+ messages in thread
From: Juzhe-Zhong @ 2024-05-12 22:57 UTC (permalink / raw)
To: gcc-patches; +Cc: vmakarov, richard.sandiford, jin.xia, dimitar, Juzhe-Zhong
---
gcc/common.opt | 4 ++++
gcc/common.opt.urls | 3 +++
gcc/doc/invoke.texi | 8 ++++++++
gcc/opts.cc | 1 +
4 files changed, 16 insertions(+)
diff --git a/gcc/common.opt b/gcc/common.opt
index 40cab3cb36a..5710e817abe 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -2163,6 +2163,10 @@ fira-share-spill-slots
Common Var(flag_ira_share_spill_slots) Init(1) Optimization
Share stack slots for spilled pseudo-registers.
+ftrack-subreg-liveness
+Common Var(flag_track_subreg_liveness) Init(0) Optimization
+Track subreg liveness information.
+
fira-verbose=
Common RejectNegative Joined UInteger Var(flag_ira_verbose) Init(5)
-fira-verbose=<number> Control IRA's level of diagnostic messages.
diff --git a/gcc/common.opt.urls b/gcc/common.opt.urls
index f71ed80a34b..59f27a6f7c6 100644
--- a/gcc/common.opt.urls
+++ b/gcc/common.opt.urls
@@ -880,6 +880,9 @@ UrlSuffix(gcc/Optimize-Options.html#index-fira-share-save-slots)
fira-share-spill-slots
UrlSuffix(gcc/Optimize-Options.html#index-fira-share-spill-slots)
+ftrack-subreg-liveness
+UrlSuffix(gcc/Optimize-Options.html#index-ftrack-subreg-liveness)
+
fira-verbose=
UrlSuffix(gcc/Developer-Options.html#index-fira-verbose)
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index ddcd5213f06..fbcde8aa745 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -13188,6 +13188,14 @@ Disable sharing of stack slots allocated for pseudo-registers. Each
pseudo-register that does not get a hard register gets a separate
stack slot, and as a result function stack frames are larger.
+@opindex ftrack-subreg-liveness
+@item -ftrack-subreg-liveness
+Enable tracking subreg liveness information. This infomation allows IRA
+and LRA to support subreg coalesce feature which can improve the quality
+of register allocation.
+
+This option is enabled at level @option{-O3} for all targets.
+
@opindex flra-remat
@item -flra-remat
Enable CFG-sensitive rematerialization in LRA. Instead of loading
diff --git a/gcc/opts.cc b/gcc/opts.cc
index 14d1767e48f..8fe3a213807 100644
--- a/gcc/opts.cc
+++ b/gcc/opts.cc
@@ -698,6 +698,7 @@ static const struct default_options default_options_table[] =
{ OPT_LEVELS_3_PLUS, OPT_funswitch_loops, NULL, 1 },
{ OPT_LEVELS_3_PLUS, OPT_fvect_cost_model_, NULL, VECT_COST_MODEL_DYNAMIC },
{ OPT_LEVELS_3_PLUS, OPT_fversion_loops_for_strides, NULL, 1 },
+ { OPT_LEVELS_3_PLUS, OPT_ftrack_subreg_liveness, NULL, 1 },
/* -O3 parameters. */
{ OPT_LEVELS_3_PLUS, OPT__param_max_inline_insns_auto_, NULL, 30 },
--
2.36.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem
2024-05-12 22:57 [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Juzhe-Zhong
@ 2024-05-12 22:57 ` Juzhe-Zhong
2024-06-13 15:57 ` Richard Sandiford
2024-05-12 22:57 ` [SUBREG V4 3/4] IRA: Apply DF_LIVE_SUBREG data Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 4/4] LRA: " Juzhe-Zhong
3 siblings, 1 reply; 7+ messages in thread
From: Juzhe-Zhong @ 2024-05-12 22:57 UTC (permalink / raw)
To: gcc-patches; +Cc: vmakarov, richard.sandiford, jin.xia, dimitar, Juzhe-Zhong
---
gcc/Makefile.in | 1 +
gcc/df-problems.cc | 886 ++++++++++++++++++++++++++++++++++++++-
gcc/df.h | 159 +++++++
gcc/regs.h | 5 +
gcc/sbitmap.cc | 98 +++++
gcc/sbitmap.h | 2 +
gcc/subreg-live-range.cc | 233 ++++++++++
gcc/subreg-live-range.h | 60 +++
gcc/timevar.def | 1 +
9 files changed, 1444 insertions(+), 1 deletion(-)
create mode 100644 gcc/subreg-live-range.cc
create mode 100644 gcc/subreg-live-range.h
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index a7f15694c34..67d2e3ca1bc 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1684,6 +1684,7 @@ OBJS = \
store-motion.o \
streamer-hooks.o \
stringpool.o \
+ subreg-live-range.o \
substring-locations.o \
target-globals.o \
targhooks.o \
diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc
index 88ee0dd67fc..01f1f850925 100644
--- a/gcc/df-problems.cc
+++ b/gcc/df-problems.cc
@@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see
#include "target.h"
#include "rtl.h"
#include "df.h"
+#include "subreg-live-range.h"
#include "memmodel.h"
#include "tm_p.h"
#include "insn-config.h"
@@ -1344,8 +1345,891 @@ df_lr_verify_transfer_functions (void)
bitmap_clear (&all_blocks);
}
+/*----------------------------------------------------------------------------
+ REGISTER AND SUBREGS LIVES
+ Like DF_LR, but include tracking subreg liveness. Currently used to provide
+ subreg liveness related information to the register allocator. The subreg
+ information is currently tracked for registers that satisfy the following
+ conditions:
+ 1. REG is a pseudo register
+ 2. MODE_SIZE > UNIT_SIZE
+ 3. MODE_SIZE is a multiple of UNIT_SIZE
+ 4. REG is used via subreg pattern
+ Assuming: MODE = the machine mode of the REG
+ MODE_SIZE = GET_MODE_SIZE (MODE)
+ UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE)
+ Condition 3 is currently strict, maybe it can be removed in the future, but
+ for now it is sufficient.
+----------------------------------------------------------------------------*/
+
+/* These two empty data are used as default data in case the user does not turn
+ * on the track-subreg-liveness feature. */
+bitmap_head df_subreg_empty_bitmap;
+subregs_live df_subreg_empty_live;
+
+/* Private data for live_subreg problem. */
+struct df_live_subreg_problem_data
+{
+ /* Record registers that need to track subreg liveness. */
+ bitmap_head tracked_regs;
+ /* An obstack for the bitmaps we need for this problem. */
+ bitmap_obstack live_subreg_bitmaps;
+};
+
+/* Helper functions. */
+
+static df_live_subreg_bb_info *
+df_live_subreg_get_bb_info (unsigned int index)
+{
+ if (index < df_live_subreg->block_info_size)
+ return &static_cast<df_live_subreg_bb_info *> (
+ df_live_subreg->block_info)[index];
+ else
+ return nullptr;
+}
+
+static df_live_subreg_local_bb_info *
+get_live_subreg_local_bb_info (unsigned int bb_index)
+{
+ return df_live_subreg_get_bb_info (bb_index);
+}
+
+/* Return true if regno is a multireg. */
+bool
+multireg_p (int regno)
+{
+ if (regno < FIRST_PSEUDO_REGISTER)
+ return false;
+ rtx regno_rtx = regno_reg_rtx[regno];
+ machine_mode reg_mode = GET_MODE (regno_rtx);
+ poly_int64 total_size = GET_MODE_SIZE (reg_mode);
+ poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode);
+ return maybe_gt (total_size, natural_size)
+ && multiple_p (total_size, natural_size);
+}
+
+/* Return true if the REGNO need be track with subreg liveness. */
+
+static bool
+need_track_subreg_p (unsigned regno)
+{
+ auto problem_data
+ = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ return bitmap_bit_p (&problem_data->tracked_regs, regno);
+}
+
+/* Fill RANGE with the subreg range for OP in REGMODE_NATURAL_SIZE granularity.
+ */
+void
+init_range (rtx op, sbitmap range)
+{
+ rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op;
+ machine_mode reg_mode = GET_MODE (reg);
+
+ if (!read_modify_subreg_p (op))
+ {
+ bitmap_set_range (range, 0, get_nblocks (reg_mode));
+ return;
+ }
+
+ rtx subreg = op;
+ machine_mode subreg_mode = GET_MODE (subreg);
+ poly_int64 offset = SUBREG_BYTE (subreg);
+ int nblocks = get_nblocks (reg_mode);
+ poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode);
+ poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode);
+ poly_int64 left = offset + subreg_size;
+
+ int subreg_start = -1;
+ int subreg_nblocks = -1;
+ for (int i = 0; i < nblocks; i += 1)
+ {
+ poly_int64 right = unit_size * (i + 1);
+ if (subreg_start < 0 && maybe_lt (offset, right))
+ subreg_start = i;
+ if (subreg_nblocks < 0 && maybe_le (left, right))
+ {
+ subreg_nblocks = i + 1 - subreg_start;
+ break;
+ }
+ }
+ gcc_assert (subreg_start >= 0 && subreg_nblocks > 0);
+
+ bitmap_set_range (range, subreg_start, subreg_nblocks);
+}
+
+/* Remove RANGE from BB_INFO's use data. */
+void
+remove_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno,
+ const_sbitmap range)
+{
+ bitmap partial = &bb_info->partial_use;
+
+ bb_info->range_use->remove_range (regno, range);
+ if (bb_info->range_use->empty_p (regno))
+ bitmap_clear_bit (partial, regno);
+}
+
+/* Remove RANGE of REF from BB_INFO's use data. */
+static void
+remove_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref)
+{
+ unsigned int regno = DF_REF_REGNO (ref);
+ machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
+
+ if (need_track_subreg_p (regno))
+ {
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (DF_REF_REG (ref), range);
+ remove_subreg_range (bb_info, regno, range);
+ }
+ else
+ {
+ bitmap_clear_bit (&bb_info->full_use, regno);
+ gcc_assert (!bitmap_bit_p (&bb_info->partial_use, regno));
+ gcc_assert (!bitmap_bit_p (&bb_info->partial_def, regno));
+ }
+}
+
+/* add RANGE to BB_INFO's def. */
+void
+add_subreg_range_to_def (df_live_subreg_local_bb_info *bb_info,
+ unsigned int regno, const_sbitmap range)
+{
+ bitmap partial = &bb_info->partial_def;
+ subregs_live *range_live = bb_info->range_def;
+
+ bitmap_set_bit (partial, regno);
+ range_live->add_range (regno, range);
+}
+
+/* add RANGE to BB_INFO's use. */
+void
+add_subreg_range_to_use (df_live_subreg_local_bb_info *bb_info,
+ unsigned int regno, const_sbitmap range)
+{
+ bitmap partial = &bb_info->partial_use;
+ subregs_live *range_live = bb_info->range_use;
+
+ bitmap_set_bit (partial, regno);
+ range_live->add_range (regno, range);
+}
+
+/* add RANGE of REF to BB_INFO def. */
+static void
+add_subreg_range_to_def (df_live_subreg_local_bb_info *bb_info, df_ref ref)
+{
+ unsigned int regno = DF_REF_REGNO (ref);
+ machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
+
+ if (need_track_subreg_p (regno))
+ {
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (DF_REF_REG (ref), range);
+ add_subreg_range_to_def (bb_info, regno, range);
+ }
+ else
+ {
+ bitmap full = &bb_info->full_def;
+ bitmap partial = &bb_info->partial_def;
+ bitmap_set_bit (full, regno);
+ gcc_assert (!bitmap_bit_p (partial, regno));
+ }
+}
+
+/* add RANGE of REF to BB_INFO use. */
+static void
+add_subreg_range_to_use (df_live_subreg_local_bb_info *bb_info, df_ref ref)
+{
+ unsigned int regno = DF_REF_REGNO (ref);
+ machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref));
+
+ if (need_track_subreg_p (regno))
+ {
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (DF_REF_REG (ref), range);
+ add_subreg_range_to_use (bb_info, regno, range);
+ }
+ else
+ {
+ bitmap full = &bb_info->full_use;
+ bitmap partial = &bb_info->partial_use;
+ bitmap_set_bit (full, regno);
+ gcc_assert (!bitmap_bit_p (partial, regno));
+ }
+}
+
+/* Free basic block info. */
+
+static void
+df_live_subreg_free_bb_info (basic_block bb ATTRIBUTE_UNUSED, void *vbb_info)
+{
+ df_live_subreg_bb_info *bb_info = (df_live_subreg_bb_info *) vbb_info;
+ if (bb_info)
+ {
+ delete bb_info->range_in;
+ bb_info->range_in = NULL;
+ delete bb_info->range_out;
+ bb_info->range_out = NULL;
+
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ }
+}
+
+/* Allocate or reset bitmaps for DF_LIVE_SUBREG blocks. The solution bits are
+ not touched unless the block is new. */
+
+static void
+df_live_subreg_alloc (bitmap all_blocks ATTRIBUTE_UNUSED)
+{
+ struct df_live_subreg_problem_data *problem_data;
+ df_grow_bb_info (df_live_subreg);
+ if (df_live_subreg->problem_data)
+ problem_data
+ = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ else
+ {
+ problem_data = XNEW (struct df_live_subreg_problem_data);
+ df_live_subreg->problem_data = problem_data;
+
+ bitmap_obstack_initialize (&problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&problem_data->tracked_regs,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ bitmap_clear (&problem_data->tracked_regs);
+
+ basic_block bb;
+ FOR_EACH_BB_FN (bb, cfun)
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, bb->index);
+
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, ENTRY_BLOCK);
+ bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, EXIT_BLOCK);
+
+ unsigned int bb_index;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ /* Find the regs which we need to track it's subreg liveness. */
+ rtx_insn *insn;
+ df_ref use;
+ FOR_BB_INSNS (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* A multireg which is used via subreg pattern. */
+ if (multireg_p (regno) && DF_REF_FLAGS (use) & (DF_REF_SUBREG))
+ bitmap_set_bit (&problem_data->tracked_regs, regno);
+ }
+ }
+ }
+
+ if (dump_file)
+ {
+ fprintf (dump_file, ";; regs need to be tracked subreg liveness: ");
+ df_print_regset (dump_file, &problem_data->tracked_regs);
+ }
+
+ size_t n = bitmap_count_bits (&problem_data->tracked_regs);
+
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ /* Clean global infos. */
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+
+ /* When bitmaps are already initialized, just clear them. */
+ if (bb_info->all_in.obstack)
+ {
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ }
+ else
+ {
+ bitmap_initialize (&bb_info->all_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->full_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->partial_in,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->all_out,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->full_out,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&bb_info->partial_out,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ if (bb_info->range_in)
+ {
+ bb_info->range_in->clear (n);
+ bb_info->range_out->clear (n);
+ }
+ else
+ {
+ bb_info->range_in = new subregs_live (n);
+ bb_info->range_out = new subregs_live (n);
+ }
+
+ /* Clean local infos. */
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ /* When bitmaps are already initialized, just clear them. */
+ if (local_bb_info->full_use.obstack)
+ {
+ bitmap_clear (&local_bb_info->full_def);
+ bitmap_clear (&local_bb_info->partial_def);
+ bitmap_clear (&local_bb_info->full_use);
+ bitmap_clear (&local_bb_info->partial_use);
+ }
+ else
+ {
+ bitmap_initialize (&local_bb_info->full_def,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->partial_def,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->full_use,
+ &problem_data->live_subreg_bitmaps);
+ bitmap_initialize (&local_bb_info->partial_use,
+ &problem_data->live_subreg_bitmaps);
+ }
+
+ if (local_bb_info->range_def)
+ {
+ local_bb_info->range_def->clear (n);
+ local_bb_info->range_use->clear (n);
+ }
+ else
+ {
+ local_bb_info->range_def = new subregs_live (n);
+ local_bb_info->range_use = new subregs_live (n);
+ }
+ }
+
+ df_live_subreg->optional_p = true;
+}
+
+/* Reset the global solution for recalculation. */
+
+static void
+df_live_subreg_reset (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ gcc_assert (bb_info);
+ bitmap_clear (&bb_info->all_in);
+ bitmap_clear (&bb_info->full_in);
+ bitmap_clear (&bb_info->partial_in);
+ bitmap_clear (&bb_info->all_out);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ bb_info->range_in->clear ();
+ bb_info->range_out->clear ();
+ }
+}
+
+/* Compute local live register info for basic block BB. */
+
+static void
+df_live_subreg_bb_local_compute (unsigned int bb_index)
+{
+ basic_block bb = BASIC_BLOCK_FOR_FN (cfun, bb_index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ rtx_insn *insn;
+ df_ref def, use;
+
+ /* Process the registers set in an exception handler. */
+ FOR_EACH_ARTIFICIAL_DEF (def, bb_index)
+ if ((DF_REF_FLAGS (def) & DF_REF_AT_TOP) == 0)
+ {
+ add_subreg_range_to_def (local_bb_info, def);
+ remove_subreg_range (local_bb_info, def);
+ }
+
+ /* Process the hardware registers that are always live. */
+ FOR_EACH_ARTIFICIAL_USE (use, bb_index)
+ if ((DF_REF_FLAGS (use) & DF_REF_AT_TOP) == 0)
+ add_subreg_range_to_use (local_bb_info, use);
+
+ FOR_BB_INSNS_REVERSE (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+ FOR_EACH_INSN_INFO_DEF (def, insn_info)
+ {
+ remove_subreg_range (local_bb_info, def);
+ add_subreg_range_to_def (local_bb_info, def);
+ }
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* Ignore the use of subreg which is used as dest operand. */
+ if (need_track_subreg_p (regno)
+ && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG))
+ continue;
+ add_subreg_range_to_use (local_bb_info, use);
+ }
+ }
+
+ /* Process the registers set in an exception handler or the hard
+ frame pointer if this block is the target of a non local
+ goto. */
+ FOR_EACH_ARTIFICIAL_DEF (def, bb_index)
+ if (DF_REF_FLAGS (def) & DF_REF_AT_TOP)
+ {
+ add_subreg_range_to_def (local_bb_info, def);
+ remove_subreg_range (local_bb_info, def);
+ }
+
+#ifdef EH_USES
+ /* Process the uses that are live into an exception handler. */
+ FOR_EACH_ARTIFICIAL_USE (use, bb_index)
+ /* Add use to set of uses in this BB. */
+ if (DF_REF_FLAGS (use) & DF_REF_AT_TOP)
+ add_subreg_range_to_use (local_bb_info, use);
+#endif
+}
+
+/* Compute local live register info for each basic block within BLOCKS. */
+
+static void
+df_live_subreg_local_compute (bitmap all_blocks ATTRIBUTE_UNUSED)
+{
+ unsigned int bb_index, i;
+ bitmap_iterator bi;
+
+ bitmap_clear (&df->hardware_regs_used);
+
+ /* The all-important stack pointer must always be live. */
+ bitmap_set_bit (&df->hardware_regs_used, STACK_POINTER_REGNUM);
+
+ /* Global regs are always live, too. */
+ for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
+ if (global_regs[i])
+ bitmap_set_bit (&df->hardware_regs_used, i);
+
+ /* Before reload, there are a few registers that must be forced
+ live everywhere -- which might not already be the case for
+ blocks within infinite loops. */
+ if (!reload_completed)
+ {
+ unsigned int pic_offset_table_regnum = PIC_OFFSET_TABLE_REGNUM;
+ /* Any reference to any pseudo before reload is a potential
+ reference of the frame pointer. */
+ bitmap_set_bit (&df->hardware_regs_used, FRAME_POINTER_REGNUM);
+
+ /* Pseudos with argument area equivalences may require
+ reloading via the argument pointer. */
+ if (FRAME_POINTER_REGNUM != ARG_POINTER_REGNUM
+ && fixed_regs[ARG_POINTER_REGNUM])
+ bitmap_set_bit (&df->hardware_regs_used, ARG_POINTER_REGNUM);
+
+ /* Any constant, or pseudo with constant equivalences, may
+ require reloading from memory using the pic register. */
+ if (pic_offset_table_regnum != INVALID_REGNUM
+ && fixed_regs[pic_offset_table_regnum])
+ bitmap_set_bit (&df->hardware_regs_used, pic_offset_table_regnum);
+ }
+
+ EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0,
+ bb_index, bi)
+ {
+ if (bb_index == EXIT_BLOCK)
+ {
+ /* The exit block is special for this problem and its bits are
+ computed from thin air. */
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (EXIT_BLOCK);
+ bitmap_copy (&local_bb_info->full_use, df->exit_block_uses);
+ }
+ else
+ df_live_subreg_bb_local_compute (bb_index);
+ }
+
+ bitmap_clear (df_live_subreg->out_of_date_transfer_functions);
+}
+
+/* Initialize the solution vectors. */
+
+static void
+df_live_subreg_init (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+ bitmap_copy (&bb_info->full_in, &local_bb_info->full_use);
+ bitmap_copy (&bb_info->partial_in, &local_bb_info->partial_use);
+ bb_info->range_in->copy_lives (*local_bb_info->range_use);
+ bitmap_clear (&bb_info->full_out);
+ bitmap_clear (&bb_info->partial_out);
+ bb_info->range_out->clear ();
+ }
+}
+
+/* Check that the status of the final result is correct. */
+
+void
+df_live_subreg_check_result (bitmap full, bitmap partial,
+ subregs_live *partial_live)
+{
+ unsigned int regno;
+ bitmap_iterator bi;
+ /* Make sure that full and partial do not overlap. */
+ gcc_assert (!bitmap_intersect_p (full, partial));
+ /* Ensure that registers belonging to full are not partially live. */
+ EXECUTE_IF_SET_IN_BITMAP (full, 0, regno, bi)
+ gcc_assert (partial_live->empty_p (regno));
+ /* Ensure that registers belonging to partial are partially live. */
+ EXECUTE_IF_SET_IN_BITMAP (partial, 0, regno, bi)
+ gcc_assert (!partial_live->empty_p (regno));
+}
+
+/* Confluence function that processes infinite loops. This might be a
+ noreturn function that throws. And even if it isn't, getting the
+ unwind info right helps debugging. */
+static void
+df_live_subreg_confluence_0 (basic_block bb)
+{
+ bitmap full_out = &df_live_subreg_get_bb_info (bb->index)->full_out;
+ if (bb != EXIT_BLOCK_PTR_FOR_FN (cfun))
+ bitmap_copy (full_out, &df->hardware_regs_used);
+}
+
+/* Confluence function that ignores fake edges. */
+
+static bool
+df_live_subreg_confluence_n (edge e)
+{
+ class df_live_subreg_bb_info *src_bb_info
+ = df_live_subreg_get_bb_info (e->src->index);
+ class df_live_subreg_bb_info *dest_bb_info
+ = df_live_subreg_get_bb_info (e->dest->index);
+
+ bool changed = false;
+
+ /* Call-clobbered registers die across exception and call edges.
+ Conservatively treat partially-clobbered registers as surviving
+ across the edges; they might or might not, depending on what
+ mode they have. */
+ /* ??? Abnormal call edges ignored for the moment, as this gets
+ confused by sibling call edges, which crashes reg-stack. */
+ if (e->flags & EDGE_EH)
+ {
+ bitmap_view<HARD_REG_SET> eh_kills (eh_edge_abi.full_reg_clobbers ());
+ changed = bitmap_ior_and_compl_into (&src_bb_info->full_out,
+ &dest_bb_info->full_in, eh_kills);
+ }
+ else
+ changed = bitmap_ior_into (&src_bb_info->full_out, &dest_bb_info->full_in);
+
+ changed |= bitmap_ior_into (&src_bb_info->full_out, &df->hardware_regs_used);
+
+ /* Handle partial live case. */
+ if (!bitmap_empty_p (&dest_bb_info->partial_in))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (&dest_bb_info->partial_in,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap dest_range = dest_bb_info->range_in->get_range (regno);
+ changed |= src_bb_info->range_out->add_range (regno, dest_range);
+ }
+ changed |= bitmap_ior_into (&src_bb_info->partial_out,
+ &dest_bb_info->partial_in);
+
+ df_live_subreg_check_result (&src_bb_info->full_out,
+ &src_bb_info->partial_out,
+ src_bb_info->range_out);
+ }
+
+ return changed;
+}
+
+/* Transfer function. IN = USE | (OUT & ~DEF). */
+
+static bool
+df_live_subreg_transfer_function (int bb_index)
+{
+ class df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index);
+ class df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb_index);
+
+ bool changed = false;
+
+ changed
+ |= bitmap_ior_and_compl (&bb_info->full_in, &local_bb_info->full_use,
+ &bb_info->full_out, &local_bb_info->full_def);
+
+ /* Handle partial live case. */
+ if (!bitmap_empty_p (&bb_info->partial_out)
+ || !bitmap_empty_p (&local_bb_info->partial_use))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ bitmap_head temp_partial_out;
+ subregs_live temp_range_out;
+
+ /* TEMP = (OUT & ~DEF) */
+ bitmap_initialize (&temp_partial_out, &bitmap_default_obstack);
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER,
+ regno, bi)
+ {
+ sbitmap out_range = bb_info->range_out->get_range (regno);
+ temp_range_out.add_range (regno, out_range);
+ if (bitmap_bit_p (&local_bb_info->partial_def, regno))
+ {
+ sbitmap def_range = local_bb_info->range_def->get_range (regno);
+ temp_range_out.remove_range (regno, def_range);
+ if (!temp_range_out.empty_p (regno))
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+ else
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+
+ /* TEMP = USE | TEMP. */
+ EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap use_range = local_bb_info->range_use->get_range (regno);
+ temp_range_out.add_range (regno, use_range);
+ }
+ bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use);
+
+ /* IN = TEMP. */
+ changed |= bb_info->range_in->copy_lives (temp_range_out);
+ bitmap_copy (&bb_info->partial_in, &temp_partial_out);
+ df_live_subreg_check_result (&bb_info->full_in, &bb_info->partial_in,
+ bb_info->range_in);
+ }
+ else if (!bitmap_empty_p (&bb_info->partial_in))
+ {
+ changed = true;
+ bitmap_clear (&bb_info->partial_in);
+ bb_info->range_in->clear ();
+ }
+
+ return changed;
+}
+
+/* Calculating ALL_IN from FULL_IN and PARTIAL_IN
+ and ALL_OUT from FULL_OUT and PARTIAL_OUT. */
+
+void
+df_live_subreg_finalize (bitmap all_blocks)
+{
+ unsigned int bb_index;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi)
+ {
+ class df_live_subreg_bb_info *bb_info
+ = df_live_subreg_get_bb_info (bb_index);
+ gcc_assert (bb_info);
+ bitmap_ior (&bb_info->all_in, &bb_info->full_in, &bb_info->partial_in);
+ bitmap_ior (&bb_info->all_out, &bb_info->full_out, &bb_info->partial_out);
+
+ /* Move full live reg in partial_in/partial_out to full_in/full_out. */
+ unsigned int regno;
+ bitmap_iterator ri;
+ bool changed = false;
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_in, FIRST_PSEUDO_REGISTER,
+ regno, ri)
+ {
+ if (bb_info->range_in->full_p (regno))
+ {
+ bitmap_set_bit (&bb_info->full_in, regno);
+ bb_info->range_in->remove_range (regno);
+ changed = true;
+ }
+ }
+ if (changed)
+ bitmap_and_compl_into (&bb_info->partial_in, &bb_info->full_in);
+
+ changed = false;
+ EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER,
+ regno, ri)
+ {
+ if (bb_info->range_out->full_p (regno))
+ {
+ bitmap_set_bit (&bb_info->full_out, regno);
+ bb_info->range_out->remove_range (regno);
+ changed = true;
+ }
+ }
+ if (changed)
+ bitmap_and_compl_into (&bb_info->partial_out, &bb_info->full_out);
+ }
+}
+
+/* Free all storage associated with the problem. */
+
+static void
+df_live_subreg_free (void)
+{
+ df_live_subreg_problem_data *problem_data
+ = (df_live_subreg_problem_data *) df_live_subreg->problem_data;
+ if (df_live_subreg->block_info)
+ {
+ bitmap_obstack_release (&problem_data->live_subreg_bitmaps);
+ basic_block bb;
+ FOR_ALL_BB_FN (bb, cfun)
+ {
+ df_live_subreg_bb_info *bb_info
+ = df_live_subreg_get_bb_info (bb->index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb->index);
+ delete bb_info->range_in;
+ delete bb_info->range_out;
+ delete local_bb_info->range_def;
+ delete local_bb_info->range_use;
+ }
+ free (problem_data);
+ df_live_subreg->problem_data = NULL;
+
+ df_live_subreg->block_info_size = 0;
+ free (df_live_subreg->block_info);
+ df_live_subreg->block_info = NULL;
+ }
+
+ BITMAP_FREE (df_live_subreg->out_of_date_transfer_functions);
+ free (df_live_subreg);
+}
+
+/* Debugging info at top of bb. */
+
+static void
+df_live_subreg_top_dump (basic_block bb, FILE *file)
+{
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index);
+ df_live_subreg_local_bb_info *local_bb_info
+ = get_live_subreg_local_bb_info (bb->index);
+
+ if (!bb_info)
+ return;
+
+ fprintf (file, ";; subreg live all in \t");
+ df_print_regset (file, &bb_info->all_in);
+ fprintf (file, ";; subreg live full in \t");
+ df_print_regset (file, &bb_info->full_in);
+ fprintf (file, ";; subreg live partial in \t");
+ df_print_regset (file, &bb_info->partial_in);
+ fprintf (file, ";; subreg live range in \t");
+ bb_info->range_in->dump (file, "");
+
+ fprintf (file, "\n;; subreg live full use \t");
+ df_print_regset (file, &local_bb_info->full_use);
+ fprintf (file, ";; subreg live partial use \t");
+ df_print_regset (file, &local_bb_info->partial_use);
+ fprintf (file, ";; subreg live range use \t");
+ local_bb_info->range_use->dump (file, "");
+
+ fprintf (file, "\n;; subreg live full def \t");
+ df_print_regset (file, &local_bb_info->full_def);
+ fprintf (file, ";; subreg live partial def \t");
+ df_print_regset (file, &local_bb_info->partial_def);
+ fprintf (file, ";; subreg live range def \t");
+ local_bb_info->range_def->dump (file, "");
+}
+
+/* Debugging info at bottom of bb. */
+
+static void
+df_live_subreg_bottom_dump (basic_block bb, FILE *file)
+{
+ df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index);
+ if (!bb_info)
+ return;
+
+ fprintf (file, ";; subreg live all out \t");
+ df_print_regset (file, &bb_info->all_out);
+ fprintf (file, ";; subreg live full out \t");
+ df_print_regset (file, &bb_info->full_out);
+ fprintf (file, ";; subreg live partial out \t");
+ df_print_regset (file, &bb_info->partial_out);
+ fprintf (file, ";; subreg live range out \t");
+ bb_info->range_out->dump (file, "");
+}
+
+/* All of the information associated with every instance of the problem. */
+
+static const struct df_problem problem_LIVE_SUBREG = {
+ DF_LIVE_SUBREG, /* Problem id. */
+ DF_BACKWARD, /* Direction. */
+ df_live_subreg_alloc, /* Allocate the problem data. */
+ df_live_subreg_reset, /* Reset global information. */
+ df_live_subreg_free_bb_info, /* Free basic block info. */
+ df_live_subreg_local_compute, /* Local compute function. */
+ df_live_subreg_init, /* Init the solution data. */
+ df_worklist_dataflow, /* Worklist solver. */
+ df_live_subreg_confluence_0, /* Confluence operator 0. */
+ df_live_subreg_confluence_n, /* Confluence operator n. */
+ df_live_subreg_transfer_function, /* Transfer function. */
+ df_live_subreg_finalize, /* Finalize function. */
+ df_live_subreg_free, /* Free problem information. */
+ df_live_subreg_free, /* Remove this problem from the stack
+ of dataflow problem. */
+ NULL, /* Debugging. */
+ df_live_subreg_top_dump, /* Debugging start block. */
+ df_live_subreg_bottom_dump, /* Debugging end block. */
+ NULL, /* Debugging start insn. */
+ NULL, /* Debugging end insn. */
+ NULL, /* Incremental solution start. */
+ NULL, /* Incremental solution end. */
+ &problem_LR, /* Dependent problem. */
+ sizeof (df_live_subreg_bb_info), /* Size of entry of block_info array. */
+ TV_DF_LIVE_SUBREG, /* Timing variable. */
+ false /* Reset blocks on dropping out of blocks_to_analyze. */
+};
+
+/* Create a new DATAFLOW instance and add it to an existing instance
+ of DF. The returned structure is what is used to get at the
+ solution. */
+
+void
+df_live_subreg_add_problem (void)
+{
+ gcc_assert (flag_track_subreg_liveness);
+
+ df_add_problem (&problem_LIVE_SUBREG);
+
+ /* These will be initialized when df_scan_blocks processes each
+ block. */
+ df_live_subreg->out_of_date_transfer_functions
+ = BITMAP_ALLOC (&df_bitmap_obstack);
+}
-\f
/*----------------------------------------------------------------------------
LIVE AND MAY-INITIALIZED REGISTERS.
diff --git a/gcc/df.h b/gcc/df.h
index 84e5aa8b524..7a3876dec8c 100644
--- a/gcc/df.h
+++ b/gcc/df.h
@@ -47,6 +47,7 @@ enum df_problem_id
{
DF_SCAN,
DF_LR, /* Live Registers backward. */
+ DF_LIVE_SUBREG, /* Live Regs and Subregs. */
DF_LIVE, /* Live Registers & Uninitialized Registers */
DF_RD, /* Reaching Defs. */
DF_CHAIN, /* Def-Use and/or Use-Def Chains. */
@@ -878,6 +879,33 @@ public:
bitmap_head out; /* At the bottom of the block. */
};
+class subregs_live; /* Defined in subreg-range.{h,cc} */
+
+/* Local live info in basic block. Use by live_subreg problem and LRA pass. */
+class df_live_subreg_local_bb_info
+{
+public:
+ bitmap_head full_def;
+ bitmap_head full_use;
+ /* Only for pseudo registers. */
+ bitmap_head partial_def;
+ bitmap_head partial_use;
+ subregs_live *range_def = nullptr;
+ subregs_live *range_use = nullptr;
+};
+
+/* Live in/out infos of each basic. */
+class df_live_subreg_bb_info : public df_live_subreg_local_bb_info
+{
+public:
+ bitmap_head all_in, full_in;
+ bitmap_head all_out, full_out;
+ /* Only for pseudo registers. */
+ bitmap_head partial_in;
+ bitmap_head partial_out;
+ subregs_live *range_in = nullptr;
+ subregs_live *range_out = nullptr;
+};
/* Uninitialized registers. All bitmaps are referenced by the
register number. Anded results of the forwards and backward live
@@ -940,6 +968,7 @@ extern class df_d *df;
#define df_scan (df->problems_by_index[DF_SCAN])
#define df_rd (df->problems_by_index[DF_RD])
#define df_lr (df->problems_by_index[DF_LR])
+#define df_live_subreg (df->problems_by_index[DF_LIVE_SUBREG])
#define df_live (df->problems_by_index[DF_LIVE])
#define df_chain (df->problems_by_index[DF_CHAIN])
#define df_word_lr (df->problems_by_index[DF_WORD_LR])
@@ -1031,6 +1060,8 @@ extern void df_lr_add_problem (void);
extern void df_lr_verify_transfer_functions (void);
extern void df_live_verify_transfer_functions (void);
extern void df_live_add_problem (void);
+extern void df_live_subreg_add_problem ();
+extern void df_live_subreg_finalize (bitmap all_blocks);
extern void df_live_set_all_dirty (void);
extern void df_chain_add_problem (unsigned int);
extern void df_word_lr_add_problem (void);
@@ -1059,6 +1090,19 @@ extern bool can_move_insns_across (rtx_insn *, rtx_insn *,
rtx_insn *, rtx_insn *,
basic_block, regset,
regset, rtx_insn **);
+extern void
+df_live_subreg_check_result (bitmap, bitmap, subregs_live *);
+extern bool multireg_p (int);
+extern void init_range (rtx, sbitmap);
+extern void
+add_subreg_range_to_def (df_live_subreg_local_bb_info *, unsigned int,
+ const_sbitmap);
+extern void
+add_subreg_range_to_use (df_live_subreg_local_bb_info *, unsigned int,
+ const_sbitmap);
+extern void
+remove_subreg_range (df_live_subreg_local_bb_info *, unsigned int,
+ const_sbitmap);
/* Functions defined in df-scan.cc. */
extern void df_scan_alloc (bitmap);
@@ -1192,6 +1236,121 @@ df_get_live_in (basic_block bb)
return DF_LR_IN (bb);
}
+/* Get the subreg live at in set for BB. The live set include full and partial
+ * live. We only track and use subreg liveness when -ftrack-subreg-liveness,
+ * otherwise use DF_LR_IN. This function is used by the register allocators.
+ */
+
+inline bitmap
+df_get_subreg_live_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->all_in);
+ }
+ return DF_LR_IN (bb);
+}
+
+/* Get the subreg live at out set for BB. The live set include full and
+ * partial live. We only track and use subreg liveness when
+ * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by
+ * the register allocators. */
+
+inline bitmap
+df_get_subreg_live_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->all_out);
+ }
+ return DF_LR_OUT (bb);
+}
+
+/* Get the subreg live at in set for BB. The live set only include full and
+ * partial live. We only track and use subreg liveness when
+ * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by
+ * the register allocators. */
+
+inline bitmap
+df_get_subreg_live_full_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->full_in);
+ }
+ return DF_LR_IN (bb);
+}
+
+inline bitmap
+df_get_subreg_live_full_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->full_out);
+ }
+ return DF_LR_OUT (bb);
+}
+
+/* Define in df-problems.cc, used when disable track-subreg-liveness. */
+extern bitmap_head df_subreg_empty_bitmap;
+extern subregs_live df_subreg_empty_live;
+
+inline bitmap
+df_get_subreg_live_partial_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->partial_in);
+ }
+ return &df_subreg_empty_bitmap;
+}
+
+inline bitmap
+df_get_subreg_live_partial_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return &(bb_info->partial_out);
+ }
+ return &df_subreg_empty_bitmap;
+}
+
+inline subregs_live *
+df_get_subreg_live_range_in (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return bb_info->range_in;
+ }
+ return &df_subreg_empty_live;
+}
+
+inline subregs_live *
+df_get_subreg_live_range_out (basic_block bb)
+{
+ if (flag_track_subreg_liveness)
+ {
+ df_live_subreg_bb_info *bb_info = &(
+ (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index];
+ return bb_info->range_out;
+ }
+ return &df_subreg_empty_live;
+}
+
/* Get basic block info. */
/* Get the artificial defs for a basic block. */
diff --git a/gcc/regs.h b/gcc/regs.h
index 16189c58fd1..b0e9fd0c344 100644
--- a/gcc/regs.h
+++ b/gcc/regs.h
@@ -389,4 +389,9 @@ range_in_hard_reg_set_p (const_hard_reg_set set, unsigned regno, int nregs)
return true;
}
+/* Return the number of blocks the MODE overlap. One block equal to mode's
+ natural size. */
+#define get_nblocks(mode) \
+ (exact_div (GET_MODE_SIZE (mode), REGMODE_NATURAL_SIZE (mode)).to_constant ())
+
#endif /* GCC_REGS_H */
diff --git a/gcc/sbitmap.cc b/gcc/sbitmap.cc
index 03bb2c6d44b..9e35d6f6736 100644
--- a/gcc/sbitmap.cc
+++ b/gcc/sbitmap.cc
@@ -208,6 +208,29 @@ bitmap_empty_p (const_sbitmap bmap)
return true;
}
+/* Return true if the bitmap is full, i.e. all bits are set. */
+
+bool
+bitmap_full_p (const_sbitmap bmap)
+{
+ unsigned int end_word = bmap->n_bits / SBITMAP_ELT_BITS;
+ unsigned int i = 0;
+ for (; i < end_word; i++)
+ if (bmap->elms[i] != (SBITMAP_ELT_TYPE) -1)
+ return false;
+
+ unsigned int end_bitno = bmap->n_bits % SBITMAP_ELT_BITS;
+
+ if (end_bitno == 0)
+ return true;
+
+ gcc_checking_assert (i + 1 == bmap->size);
+
+ SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1;
+ /* Make sure the tail bits are set. */
+ return (bmap->elms[i] & mask) == mask;
+}
+
/* Clear COUNT bits from START in BMAP. */
void
@@ -662,6 +685,36 @@ bitmap_subset_p (const_sbitmap a, const_sbitmap b)
return true;
}
+/* Return true if the bits in A and B are set the same and the number of bits is
+ the same. */
+
+bool
+bitmap_same_p (const_sbitmap a, const_sbitmap b)
+{
+ if (a->n_bits != b->n_bits)
+ return false;
+
+ gcc_checking_assert (a->size == b->size);
+
+ unsigned int end_word = a->n_bits / SBITMAP_ELT_BITS;
+ unsigned int i = 0;
+ for (; i < end_word; i++)
+ if (a->elms[i] != b->elms[i])
+ return false;
+
+ unsigned int end_bitno = a->n_bits % SBITMAP_ELT_BITS;
+
+ if (end_bitno == 0)
+ return true;
+
+ gcc_checking_assert (i + 1 == a->size);
+
+ SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1;
+
+ /* Make sure the tail bits are same. */
+ return (a->elms[i] & mask) == (b->elms[i] & mask);
+}
+
/* Set DST to be (A or (B and C)).
Return nonzero if any change is made. */
@@ -994,6 +1047,49 @@ test_bit_in_range ()
sbitmap_free (s);
}
+/* Verify bitmap_full_p functions for sbitmap. */
+
+static void
+test_full ()
+{
+ sbitmap s = sbitmap_alloc (193);
+
+ bitmap_clear (s);
+ ASSERT_FALSE (bitmap_full_p (s));
+
+ bitmap_ones (s);
+ ASSERT_TRUE (bitmap_full_p (s));
+
+ bitmap_clear_bit (s, 192);
+ ASSERT_FALSE (bitmap_full_p (s));
+
+ bitmap_ones (s);
+ bitmap_clear_bit (s, 17);
+ ASSERT_FALSE (bitmap_full_p (s));
+}
+
+/* Verify bitmap_same_p functions for sbitmap. */
+
+static void
+test_same ()
+{
+ sbitmap s1 = sbitmap_alloc (193);
+ sbitmap s2 = sbitmap_alloc (193);
+ sbitmap s3 = sbitmap_alloc (192);
+
+ ASSERT_FALSE (bitmap_same_p (s1, s3));
+
+ bitmap_clear (s1);
+ bitmap_clear (s2);
+ ASSERT_TRUE (bitmap_same_p (s1, s2));
+
+ bitmap_set_bit (s2, 192);
+ ASSERT_FALSE (bitmap_same_p (s1, s2));
+
+ bitmap_set_bit (s1, 192);
+ ASSERT_TRUE (bitmap_same_p (s1, s2));
+}
+
/* Run all of the selftests within this file. */
void
@@ -1001,6 +1097,8 @@ sbitmap_cc_tests ()
{
test_set_range ();
test_bit_in_range ();
+ test_full ();
+ test_same ();
}
} // namespace selftest
diff --git a/gcc/sbitmap.h b/gcc/sbitmap.h
index da6116ce925..71cfded9fb2 100644
--- a/gcc/sbitmap.h
+++ b/gcc/sbitmap.h
@@ -267,6 +267,7 @@ extern void bitmap_copy (sbitmap, const_sbitmap);
extern bool bitmap_equal_p (const_sbitmap, const_sbitmap);
extern unsigned int bitmap_count_bits (const_sbitmap);
extern bool bitmap_empty_p (const_sbitmap);
+extern bool bitmap_full_p (const_sbitmap);
extern void bitmap_clear (sbitmap);
extern void bitmap_clear_range (sbitmap, unsigned, unsigned);
extern void bitmap_set_range (sbitmap, unsigned, unsigned);
@@ -287,6 +288,7 @@ extern bool bitmap_and (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_ior (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_xor (sbitmap, const_sbitmap, const_sbitmap);
extern bool bitmap_subset_p (const_sbitmap, const_sbitmap);
+extern bool bitmap_same_p (const_sbitmap, const_sbitmap);
extern bool bitmap_bit_in_range_p (const_sbitmap, unsigned int, unsigned int);
extern int bitmap_first_set_bit (const_sbitmap);
diff --git a/gcc/subreg-live-range.cc b/gcc/subreg-live-range.cc
new file mode 100644
index 00000000000..4181463eefa
--- /dev/null
+++ b/gcc/subreg-live-range.cc
@@ -0,0 +1,233 @@
+/* SUBREG liveness tracking classes for DF & IRA & LRA.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ Contributed by RiVAI Technologies Ltd.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#include "subreg-live-range.h"
+
+subregs_live::~subregs_live ()
+{
+ for (auto &kv : lives)
+ sbitmap_free (kv.second);
+}
+
+void
+subregs_live::clear ()
+{
+ for (auto &kv : lives)
+ sbitmap_free (kv.second);
+ lives.clear ();
+}
+
+void
+subregs_live::clear (size_t n)
+{
+ clear ();
+ lives.rehash (n);
+}
+
+bool
+subregs_live::find_p (unsigned int regno) const
+{
+ return lives.find (regno) != lives.end ();
+}
+
+sbitmap
+subregs_live::get_range (unsigned int regno)
+{
+ gcc_assert (find_p (regno));
+ return lives.at (regno);
+}
+
+const_sbitmap
+subregs_live::get_range (unsigned int regno) const
+{
+ gcc_assert (find_p (regno));
+ return lives.at (regno);
+}
+
+/* Added RANGE to regno's ranges. Return true if leads to a change. */
+bool
+subregs_live::add_range (unsigned int regno, const_sbitmap range)
+{
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ gcc_assert (range->n_bits == curr->n_bits);
+ return bitmap_ior (curr, curr, range);
+ }
+ else
+ {
+ sbitmap a = sbitmap_alloc (range->n_bits);
+ lives.insert ({regno, a});
+ sbitmap curr = get_range (regno);
+ bitmap_copy (curr, range);
+ return !bitmap_empty_p (range);
+ }
+}
+
+/* Removed RANGE from regno's ranges. Return true if leads to a change. */
+bool
+subregs_live::remove_range (unsigned int regno, const_sbitmap range)
+{
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ bitmap_check_sizes (curr, range);
+ if (bitmap_subset_p (curr, range))
+ return remove_range (regno);
+
+ auto_sbitmap a (range->n_bits);
+ bitmap_not (a, range);
+ return bitmap_and (curr, curr, a);
+ }
+ return false;
+}
+
+/* Removed the whole range of REGNO. Return true if leads to a change. */
+bool
+subregs_live::remove_range (unsigned int regno)
+{
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ bool changed = !bitmap_empty_p (curr);
+ sbitmap_free (curr);
+ lives.erase (regno);
+ return changed;
+ }
+ return false;
+}
+
+/* Replace the range of REGNO with RANGE. Return true if leads to a change.
+ */
+bool
+subregs_live::replace_range (unsigned int regno, const_sbitmap range)
+{
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ if (!bitmap_same_p (curr, range))
+ {
+ bitmap_copy (curr, range);
+ return true;
+ }
+ else
+ return false;
+ }
+ else
+ return add_range (regno, range);
+}
+
+/* Copy subregs_live SL. Return true if leads to a change. */
+bool
+subregs_live::copy_lives (const subregs_live &sl)
+{
+ bool changed = false;
+ for (auto &kv : sl.lives)
+ {
+ unsigned int regno = kv.first;
+ const_sbitmap range = kv.second;
+ if (bitmap_empty_p (range))
+ continue;
+ if (find_p (regno))
+ changed |= replace_range (regno, range);
+ else
+ changed |= add_range (regno, range);
+ }
+
+ for (auto it = lives.cbegin (); it != lives.cend ();)
+ {
+ unsigned int regno = it->first;
+ auto prev_it = it;
+ it++;
+ if (sl.empty_p (regno))
+ {
+ changed |= bitmap_empty_p (it->second);
+ lives.erase (prev_it);
+ }
+ }
+
+ return changed;
+}
+
+/* Added subregs_live SL. Return true if leads to a change. */
+bool
+subregs_live::add_lives (const subregs_live &sl)
+{
+ bool changed = false;
+ for (auto &kv : sl.lives)
+ {
+ unsigned int regno = kv.first;
+ const_sbitmap range = kv.second;
+ if (find_p (regno))
+ {
+ sbitmap curr = get_range (regno);
+ changed |= bitmap_ior (curr, curr, range);
+ }
+ else
+ changed |= add_range (regno, range);
+ }
+ return changed;
+}
+
+/* Return true if regno's live range is full. */
+bool
+subregs_live::full_p (unsigned int regno) const
+{
+ return find_p (regno) && bitmap_full_p (get_range (regno));
+}
+
+/* Return true if regno's live range is empty. */
+bool
+subregs_live::empty_p (unsigned int regno) const
+{
+ return !find_p (regno) || bitmap_empty_p (get_range (regno));
+}
+
+void
+subregs_live::dump (FILE *file, const char *indent) const
+{
+ if (lives.empty ())
+ {
+ fprintf (file, "%sempty\n", indent);
+ return;
+ }
+ fprintf (file, "%s", indent);
+ for (auto &kv : lives)
+ {
+ const_sbitmap range = kv.second;
+ if (bitmap_empty_p (range))
+ continue;
+ fprintf (file, "%d: ", kv.first);
+ if (!bitmap_full_p (range))
+ {
+ dump_bitmap_file (file, range);
+ fprintf (file, ", ");
+ }
+ else
+ fprintf (file, "full, ");
+ }
+ fprintf (file, "\n");
+}
+
+DEBUG_FUNCTION void
+debug (const subregs_live &l)
+{
+ l.dump (stderr, "");
+}
diff --git a/gcc/subreg-live-range.h b/gcc/subreg-live-range.h
new file mode 100644
index 00000000000..f51d20c43fe
--- /dev/null
+++ b/gcc/subreg-live-range.h
@@ -0,0 +1,60 @@
+/* SUBREG liveness tracking classes for DF & IRA & LRA.
+ Copyright (C) 2024 Free Software Foundation, Inc.
+ Contributed by RiVAI Technologies Ltd.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_SUBREG_LIVE_RANGE_H
+#define GCC_SUBREG_LIVE_RANGE_H
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include <unordered_map>
+#include "sbitmap.h"
+
+/* class subregs_live record the live subreg_ranges of registers. */
+class subregs_live
+{
+public:
+ /* The key is usually the register's regno. */
+ std::unordered_map<unsigned int, sbitmap> lives;
+
+ subregs_live () : lives () {}
+
+ subregs_live (size_t n) : lives (n) {}
+
+ ~subregs_live ();
+ void clear ();
+ void clear (size_t);
+ bool find_p (unsigned int) const;
+ sbitmap get_range (unsigned int);
+ const_sbitmap get_range (unsigned int) const;
+ bool add_range (unsigned int, const_sbitmap);
+ bool remove_range (unsigned int, const_sbitmap);
+ bool remove_range (unsigned int);
+ bool replace_range (unsigned int, const_sbitmap);
+ bool copy_lives (const subregs_live &);
+ bool add_lives (const subregs_live &);
+ bool full_p (unsigned int) const;
+ bool empty_p (unsigned int) const;
+
+ /* Debug helper. */
+ void dump (FILE *file, const char *indent = ";; ") const;
+};
+
+#endif /* GCC_SUBREG_LIVE_RANGE_H */
diff --git a/gcc/timevar.def b/gcc/timevar.def
index 8e2168e0817..337a2b7225b 100644
--- a/gcc/timevar.def
+++ b/gcc/timevar.def
@@ -120,6 +120,7 @@ DEFTIMEVAR (TV_DF_SCAN , "df scan insns")
DEFTIMEVAR (TV_DF_MD , "df multiple defs")
DEFTIMEVAR (TV_DF_RD , "df reaching defs")
DEFTIMEVAR (TV_DF_LR , "df live regs")
+DEFTIMEVAR (TV_DF_LIVE_SUBREG , "df live subregs")
DEFTIMEVAR (TV_DF_LIVE , "df live&initialized regs")
DEFTIMEVAR (TV_DF_MIR , "df must-initialized regs")
DEFTIMEVAR (TV_DF_CHAIN , "df use-def / def-use chains")
--
2.36.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [SUBREG V4 3/4] IRA: Apply DF_LIVE_SUBREG data
2024-05-12 22:57 [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem Juzhe-Zhong
@ 2024-05-12 22:57 ` Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 4/4] LRA: " Juzhe-Zhong
3 siblings, 0 replies; 7+ messages in thread
From: Juzhe-Zhong @ 2024-05-12 22:57 UTC (permalink / raw)
To: gcc-patches; +Cc: vmakarov, richard.sandiford, jin.xia, dimitar, Juzhe-Zhong
---
gcc/ira-build.cc | 7 ++++---
gcc/ira-color.cc | 8 ++++----
gcc/ira-emit.cc | 12 ++++++------
gcc/ira-lives.cc | 7 ++++---
gcc/ira.cc | 19 ++++++++++++-------
5 files changed, 30 insertions(+), 23 deletions(-)
diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc
index ea593d5a087..283ff36d3dd 100644
--- a/gcc/ira-build.cc
+++ b/gcc/ira-build.cc
@@ -1921,7 +1921,8 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node)
create_insn_allocnos (PATTERN (insn), NULL, false);
/* It might be a allocno living through from one subloop to
another. */
- EXECUTE_IF_SET_IN_REG_SET (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, i, bi)
+ EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ i, bi)
if (ira_curr_regno_allocno_map[i] == NULL)
ira_create_allocno (i, false, ira_curr_loop_tree_node);
}
@@ -1937,9 +1938,9 @@ create_loop_allocnos (edge e)
bitmap_iterator bi;
ira_loop_tree_node_t parent;
- live_in_regs = df_get_live_in (e->dest);
+ live_in_regs = df_get_subreg_live_in (e->dest);
border_allocnos = ira_curr_loop_tree_node->border_allocnos;
- EXECUTE_IF_SET_IN_REG_SET (df_get_live_out (e->src),
+ EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_out (e->src),
FIRST_PSEUDO_REGISTER, i, bi)
if (bitmap_bit_p (live_in_regs, i))
{
diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc
index b9ae32d1b4d..bfebc48ef83 100644
--- a/gcc/ira-color.cc
+++ b/gcc/ira-color.cc
@@ -2786,8 +2786,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p)
FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds)
if (e->src != loop_node->loop->latch
&& (regno < 0
- || (bitmap_bit_p (df_get_live_out (e->src), regno)
- && bitmap_bit_p (df_get_live_in (e->dest), regno))))
+ || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno)
+ && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno))))
freq += EDGE_FREQUENCY (e);
}
else
@@ -2795,8 +2795,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p)
auto_vec<edge> edges = get_loop_exit_edges (loop_node->loop);
FOR_EACH_VEC_ELT (edges, i, e)
if (regno < 0
- || (bitmap_bit_p (df_get_live_out (e->src), regno)
- && bitmap_bit_p (df_get_live_in (e->dest), regno)))
+ || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno)
+ && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno)))
freq += EDGE_FREQUENCY (e);
}
diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc
index d347f11fa02..8075b082e36 100644
--- a/gcc/ira-emit.cc
+++ b/gcc/ira-emit.cc
@@ -510,8 +510,8 @@ generate_edge_moves (edge e)
return;
src_map = src_loop_node->regno_allocno_map;
dest_map = dest_loop_node->regno_allocno_map;
- regs_live_in_dest = df_get_live_in (e->dest);
- regs_live_out_src = df_get_live_out (e->src);
+ regs_live_in_dest = df_get_subreg_live_in (e->dest);
+ regs_live_out_src = df_get_subreg_live_out (e->src);
EXECUTE_IF_SET_IN_REG_SET (regs_live_in_dest,
FIRST_PSEUDO_REGISTER, regno, bi)
if (bitmap_bit_p (regs_live_out_src, regno))
@@ -1229,16 +1229,16 @@ add_ranges_and_copies (void)
destination block) to use for searching allocnos by their
regnos because of subsequent IR flattening. */
node = IRA_BB_NODE (bb)->parent;
- bitmap_copy (live_through, df_get_live_in (bb));
+ bitmap_copy (live_through, df_get_subreg_live_in (bb));
add_range_and_copies_from_move_list
(at_bb_start[bb->index], node, live_through, REG_FREQ_FROM_BB (bb));
- bitmap_copy (live_through, df_get_live_out (bb));
+ bitmap_copy (live_through, df_get_subreg_live_out (bb));
add_range_and_copies_from_move_list
(at_bb_end[bb->index], node, live_through, REG_FREQ_FROM_BB (bb));
FOR_EACH_EDGE (e, ei, bb->succs)
{
- bitmap_and (live_through,
- df_get_live_in (e->dest), df_get_live_out (bb));
+ bitmap_and (live_through, df_get_subreg_live_in (e->dest),
+ df_get_subreg_live_out (bb));
add_range_and_copies_from_move_list
((move_t) e->aux, node, live_through,
REG_FREQ_FROM_EDGE_FREQ (EDGE_FREQUENCY (e)));
diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc
index e07d3dc3e89..7641184069d 100644
--- a/gcc/ira-lives.cc
+++ b/gcc/ira-lives.cc
@@ -1254,7 +1254,8 @@ process_out_of_region_eh_regs (basic_block bb)
if (! eh_p)
return;
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, i, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER,
+ i, bi)
{
ira_allocno_t a = ira_curr_regno_allocno_map[i];
for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--)
@@ -1288,7 +1289,7 @@ add_conflict_from_region_landing_pads (eh_region region, ira_object_t obj,
if ((landing_label = lp->landing_pad) != NULL
&& (landing_bb = BLOCK_FOR_INSN (landing_label)) != NULL
&& (region->type != ERT_CLEANUP
- || bitmap_bit_p (df_get_live_in (landing_bb),
+ || bitmap_bit_p (df_get_subreg_live_in (landing_bb),
ALLOCNO_REGNO (a))))
{
HARD_REG_SET new_conflict_regs
@@ -1325,7 +1326,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node)
high_pressure_start_point[ira_pressure_classes[i]] = -1;
}
curr_bb_node = loop_tree_node;
- reg_live_out = df_get_live_out (bb);
+ reg_live_out = df_get_subreg_live_out (bb);
sparseset_clear (objects_live);
REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out);
hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs);
diff --git a/gcc/ira.cc b/gcc/ira.cc
index 5642aea3caa..bebf6bb3e9b 100644
--- a/gcc/ira.cc
+++ b/gcc/ira.cc
@@ -4735,8 +4735,8 @@ find_moveable_pseudos (void)
bitmap_initialize (local, 0);
bitmap_initialize (transp, 0);
bitmap_initialize (moveable, 0);
- bitmap_copy (live, df_get_live_out (bb));
- bitmap_and_into (live, df_get_live_in (bb));
+ bitmap_copy (live, df_get_subreg_live_out (bb));
+ bitmap_and_into (live, df_get_subreg_live_in (bb));
bitmap_copy (transp, live);
bitmap_clear (moveable);
bitmap_clear (live);
@@ -5036,7 +5036,8 @@ interesting_dest_for_shprep_1 (rtx set, basic_block call_dom)
rtx dest = SET_DEST (set);
if (!REG_P (src) || !HARD_REGISTER_P (src)
|| !REG_P (dest) || HARD_REGISTER_P (dest)
- || (call_dom && !bitmap_bit_p (df_get_live_in (call_dom), REGNO (dest))))
+ || (call_dom
+ && !bitmap_bit_p (df_get_subreg_live_in (call_dom), REGNO (dest))))
return NULL;
return dest;
}
@@ -5514,10 +5515,12 @@ allocate_initial_values (void)
/* Update global register liveness information. */
FOR_EACH_BB_FN (bb, cfun)
{
- if (REGNO_REG_SET_P (df_get_live_in (bb), regno))
- SET_REGNO_REG_SET (df_get_live_in (bb), new_regno);
- if (REGNO_REG_SET_P (df_get_live_out (bb), regno))
- SET_REGNO_REG_SET (df_get_live_out (bb), new_regno);
+ if (REGNO_REG_SET_P (df_get_subreg_live_in (bb), regno))
+ SET_REGNO_REG_SET (df_get_subreg_live_in (bb),
+ new_regno);
+ if (REGNO_REG_SET_P (df_get_subreg_live_out (bb), regno))
+ SET_REGNO_REG_SET (df_get_subreg_live_out (bb),
+ new_regno);
}
}
}
@@ -5679,6 +5682,8 @@ ira (FILE *f)
if (optimize > 1)
df_remove_problem (df_live);
gcc_checking_assert (df_live == NULL);
+ if (flag_track_subreg_liveness)
+ df_live_subreg_add_problem ();
if (flag_checking)
df->changeable_flags |= DF_VERIFY_SCHEDULED;
--
2.36.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* [SUBREG V4 4/4] LRA: Apply DF_LIVE_SUBREG data
2024-05-12 22:57 [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Juzhe-Zhong
` (2 preceding siblings ...)
2024-05-12 22:57 ` [SUBREG V4 3/4] IRA: Apply DF_LIVE_SUBREG data Juzhe-Zhong
@ 2024-05-12 22:57 ` Juzhe-Zhong
3 siblings, 0 replies; 7+ messages in thread
From: Juzhe-Zhong @ 2024-05-12 22:57 UTC (permalink / raw)
To: gcc-patches; +Cc: vmakarov, richard.sandiford, jin.xia, dimitar, Juzhe-Zhong
---
gcc/lra-coalesce.cc | 27 +++-
gcc/lra-constraints.cc | 109 ++++++++++---
gcc/lra-int.h | 4 +
gcc/lra-lives.cc | 357 ++++++++++++++++++++++++++++++++---------
gcc/lra-remat.cc | 8 +-
gcc/lra-spills.cc | 27 +++-
gcc/lra.cc | 10 +-
7 files changed, 430 insertions(+), 112 deletions(-)
diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc
index a9b5b51cb3f..9416775a009 100644
--- a/gcc/lra-coalesce.cc
+++ b/gcc/lra-coalesce.cc
@@ -186,19 +186,28 @@ static bitmap_head used_pseudos_bitmap;
/* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info
bitmap). */
static void
-update_live_info (bitmap lr_bitmap)
+update_live_info (bitmap all, bitmap full, bitmap partial)
{
unsigned int j;
bitmap_iterator bi;
bitmap_clear (&used_pseudos_bitmap);
- EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap,
+ EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all,
FIRST_PSEUDO_REGISTER, j, bi)
bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]);
- if (! bitmap_empty_p (&used_pseudos_bitmap))
+ if (!bitmap_empty_p (&used_pseudos_bitmap))
{
- bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap);
- bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap);
+ bitmap_and_compl_into (all, &coalesced_pseudos_bitmap);
+ bitmap_ior_into (all, &used_pseudos_bitmap);
+
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_and_compl_into (full, &coalesced_pseudos_bitmap);
+ bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial);
+
+ bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap);
+ bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full);
+ }
}
}
@@ -301,8 +310,12 @@ lra_coalesce (void)
bitmap_initialize (&used_pseudos_bitmap, ®_obstack);
FOR_EACH_BB_FN (bb, cfun)
{
- update_live_info (df_get_live_in (bb));
- update_live_info (df_get_live_out (bb));
+ update_live_info (df_get_subreg_live_in (bb),
+ df_get_subreg_live_full_in (bb),
+ df_get_subreg_live_partial_in (bb));
+ update_live_info (df_get_subreg_live_out (bb),
+ df_get_subreg_live_full_out (bb),
+ df_get_subreg_live_partial_out (bb));
FOR_BB_INSNS_SAFE (bb, insn, next)
if (INSN_P (insn)
&& bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn)))
diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc
index e945a4da451..c9246e6be58 100644
--- a/gcc/lra-constraints.cc
+++ b/gcc/lra-constraints.cc
@@ -6565,34 +6565,86 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail)
{
if (prev_bb != NULL)
{
- /* Update df_get_live_in (prev_bb): */
+ /* Update subreg live (prev_bb): */
+ bitmap subreg_all_in = df_get_subreg_live_in (prev_bb);
+ bitmap subreg_full_in = df_get_subreg_live_full_in (prev_bb);
+ bitmap subreg_partial_in = df_get_subreg_live_partial_in (prev_bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (prev_bb);
EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi)
if (bitmap_bit_p (&live_regs, j))
- bitmap_set_bit (df_get_live_in (prev_bb), j);
- else
- bitmap_clear_bit (df_get_live_in (prev_bb), j);
+ {
+ bitmap_set_bit (subreg_all_in, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_set_bit (subreg_full_in, j);
+ if (bitmap_bit_p (subreg_partial_in, j))
+ {
+ bitmap_clear_bit (subreg_partial_in, j);
+ range_in->remove_range (j);
+ }
+ }
+ }
+ else if (bitmap_bit_p (subreg_all_in, j))
+ {
+ bitmap_clear_bit (subreg_all_in, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (subreg_full_in, j);
+ if (bitmap_bit_p (subreg_partial_in, j))
+ {
+ bitmap_clear_bit (subreg_partial_in, j);
+ range_in->remove_range (j);
+ }
+ }
+ }
}
+ bitmap subreg_all_out = df_get_subreg_live_out (curr_bb);
if (curr_bb != last_bb)
{
- /* Update df_get_live_out (curr_bb): */
+ /* Update subreg live (curr_bb): */
+ bitmap subreg_full_out = df_get_subreg_live_full_out (curr_bb);
+ bitmap subreg_partial_out = df_get_subreg_live_partial_out (curr_bb);
+ subregs_live *range_out = df_get_subreg_live_range_out (curr_bb);
EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi)
{
live_p = bitmap_bit_p (&live_regs, j);
if (! live_p)
FOR_EACH_EDGE (e, ei, curr_bb->succs)
- if (bitmap_bit_p (df_get_live_in (e->dest), j))
+ if (bitmap_bit_p (df_get_subreg_live_in (e->dest), j))
{
live_p = true;
break;
}
if (live_p)
- bitmap_set_bit (df_get_live_out (curr_bb), j);
- else
- bitmap_clear_bit (df_get_live_out (curr_bb), j);
+ {
+ bitmap_set_bit (subreg_all_out, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_set_bit (subreg_full_out, j);
+ if (bitmap_bit_p (subreg_partial_out, j))
+ {
+ bitmap_clear_bit (subreg_partial_out, j);
+ range_out->remove_range (j);
+ }
+ }
+ }
+ else if (bitmap_bit_p (subreg_all_out, j))
+ {
+ bitmap_clear_bit (subreg_all_out, j);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (subreg_full_out, j);
+ if (bitmap_bit_p (subreg_partial_out, j))
+ {
+ bitmap_clear_bit (subreg_partial_out, j);
+ range_out->remove_range (j);
+ }
+ }
+ }
}
}
prev_bb = curr_bb;
- bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb));
+ bitmap_and (&live_regs, &check_only_regs, subreg_all_out);
}
if (! NONDEBUG_INSN_P (curr_insn))
continue;
@@ -6709,7 +6761,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res)
bitmap_clear (res);
FOR_EACH_EDGE (e, ei, from->succs)
if (e->dest != to)
- bitmap_ior_into (res, df_get_live_in (e->dest));
+ bitmap_ior_into (res, df_get_subreg_live_in (e->dest));
last = get_last_insertion_point (from);
if (! JUMP_P (last))
return;
@@ -6781,7 +6833,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail)
{
/* We are at the end of BB. Add qualified living
pseudos for potential splitting. */
- to_process = df_get_live_out (curr_bb);
+ to_process = df_get_subreg_live_out (curr_bb);
if (last_processed_bb != NULL)
{
/* We are somewhere in the middle of EBB. */
@@ -7153,7 +7205,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail)
{
/* We reached the beginning of the current block -- do
rest of spliting in the current BB. */
- to_process = df_get_live_in (curr_bb);
+ to_process = df_get_subreg_live_in (curr_bb);
if (BLOCK_FOR_INSN (head) != curr_bb)
{
/* We are somewhere in the middle of EBB. */
@@ -7230,7 +7282,7 @@ lra_inheritance (void)
fprintf (lra_dump_file, "EBB");
/* Form a EBB starting with BB. */
bitmap_clear (&ebb_global_regs);
- bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb));
+ bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_in (bb));
for (;;)
{
if (lra_dump_file != NULL)
@@ -7246,7 +7298,7 @@ lra_inheritance (void)
break;
bb = bb->next_bb;
}
- bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb));
+ bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_out (bb));
if (lra_dump_file != NULL)
fprintf (lra_dump_file, "\n");
if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb)))
@@ -7275,15 +7327,26 @@ int lra_undo_inheritance_iter;
/* Fix BB live info LIVE after removing pseudos created on pass doing
inheritance/split which are REMOVED_PSEUDOS. */
static void
-fix_bb_live_info (bitmap live, bitmap removed_pseudos)
+fix_bb_live_info (bitmap all, bitmap full, bitmap partial,
+ bitmap removed_pseudos)
{
unsigned int regno;
bitmap_iterator bi;
EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi)
- if (bitmap_clear_bit (live, regno)
- && REG_P (lra_reg_info[regno].restore_rtx))
- bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx));
+ {
+ if (bitmap_clear_bit (all, regno)
+ && REG_P (lra_reg_info[regno].restore_rtx))
+ {
+ bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx));
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_bit (full, regno);
+ bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx));
+ gcc_assert (!bitmap_bit_p (partial, regno));
+ }
+ }
+ }
}
/* Return regno of the (subreg of) REG. Otherwise, return a negative
@@ -7349,8 +7412,12 @@ remove_inheritance_pseudos (bitmap remove_pseudos)
constraint pass. */
FOR_EACH_BB_FN (bb, cfun)
{
- fix_bb_live_info (df_get_live_in (bb), remove_pseudos);
- fix_bb_live_info (df_get_live_out (bb), remove_pseudos);
+ fix_bb_live_info (df_get_subreg_live_in (bb),
+ df_get_subreg_live_full_in (bb),
+ df_get_subreg_live_partial_in (bb), remove_pseudos);
+ fix_bb_live_info (df_get_subreg_live_out (bb),
+ df_get_subreg_live_full_out (bb),
+ df_get_subreg_live_partial_out (bb), remove_pseudos);
FOR_BB_INSNS_REVERSE (bb, curr_insn)
{
if (! INSN_P (curr_insn))
diff --git a/gcc/lra-int.h b/gcc/lra-int.h
index 5f605c3ae41..d73ff4e31b5 100644
--- a/gcc/lra-int.h
+++ b/gcc/lra-int.h
@@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see
#ifndef GCC_LRA_INT_H
#define GCC_LRA_INT_H
+#include "subreg-live-range.h"
+
#define lra_assert(c) gcc_checking_assert (c)
/* The parameter used to prevent infinite reloading for an insn. Each
@@ -161,6 +163,8 @@ struct lra_insn_reg
int regno;
/* Next reg info of the same insn. */
struct lra_insn_reg *next;
+ /* The corresponding reg or subreg RTX. */
+ rtx op;
};
/* Static part (common info for insns with the same ICODE) of LRA
diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc
index 66c6577e5d6..27897fdd3e1 100644
--- a/gcc/lra-lives.cc
+++ b/gcc/lra-lives.cc
@@ -272,8 +272,34 @@ update_pseudo_point (int regno, int point, enum point_type type)
}
}
-/* The corresponding bitmaps of BB currently being processed. */
-static bitmap bb_killed_pseudos, bb_gen_pseudos;
+/* Structure describing local BB data used for pseudo
+ live-analysis. */
+class bb_data_pseudos : public df_live_subreg_local_bb_info
+{
+public:
+ /* Basic block about which the below data are. */
+ basic_block bb;
+};
+
+/* Array for all BB data. Indexed by the corresponding BB index. */
+typedef class bb_data_pseudos *bb_data_t;
+
+/* All basic block data are referred through the following array. */
+static bb_data_t bb_data;
+
+/* The corresponding df local data of BB currently being processed. */
+static bb_data_t curr_bb_info;
+
+/* The regs which need to be tracked it's subreg liveness. */
+static bitmap_head tracked_regs;
+
+/* Return true if the REGNO need be track with subreg liveness. */
+
+static bool
+need_track_subreg_p (unsigned regno)
+{
+ return bitmap_bit_p (&tracked_regs, regno);
+}
/* Record hard register REGNO as now being live. It updates
living hard regs and START_LIVING. */
@@ -287,7 +313,7 @@ make_hard_regno_live (int regno)
SET_HARD_REG_BIT (hard_regs_live, regno);
sparseset_set_bit (start_living, regno);
if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno))
- bitmap_set_bit (bb_gen_pseudos, regno);
+ bitmap_set_bit (&curr_bb_info->full_use, regno);
}
/* Process the definition of hard register REGNO. This updates
@@ -310,8 +336,8 @@ make_hard_regno_dead (int regno)
sparseset_set_bit (start_dying, regno);
if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno))
{
- bitmap_clear_bit (bb_gen_pseudos, regno);
- bitmap_set_bit (bb_killed_pseudos, regno);
+ bitmap_clear_bit (&curr_bb_info->full_use, regno);
+ bitmap_set_bit (&curr_bb_info->full_def, regno);
}
}
@@ -343,7 +369,7 @@ mark_pseudo_dead (int regno)
/* Mark register REGNO (pseudo or hard register) in MODE as being live
and update BB_GEN_PSEUDOS. */
static void
-mark_regno_live (int regno, machine_mode mode)
+mark_regno_live (int regno, machine_mode mode, struct lra_insn_reg *reg)
{
int last;
@@ -355,15 +381,23 @@ mark_regno_live (int regno, machine_mode mode)
else
{
mark_pseudo_live (regno);
- bitmap_set_bit (bb_gen_pseudos, regno);
+ if (!need_track_subreg_p (regno))
+ bitmap_set_bit (&curr_bb_info->full_use, regno);
+ else
+ {
+ machine_mode reg_mode
+ = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op);
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (reg->op, range);
+ add_subreg_range_to_use (curr_bb_info, regno, range);
+ }
}
}
-
/* Mark register REGNO (pseudo or hard register) in MODE as being dead
and update BB_GEN_PSEUDOS and BB_KILLED_PSEUDOS. */
static void
-mark_regno_dead (int regno, machine_mode mode)
+mark_regno_dead (int regno, machine_mode mode, struct lra_insn_reg *reg)
{
int last;
@@ -375,8 +409,20 @@ mark_regno_dead (int regno, machine_mode mode)
else
{
mark_pseudo_dead (regno);
- bitmap_clear_bit (bb_gen_pseudos, regno);
- bitmap_set_bit (bb_killed_pseudos, regno);
+ if (!need_track_subreg_p (regno))
+ {
+ bitmap_clear_bit (&curr_bb_info->full_use, regno);
+ bitmap_set_bit (&curr_bb_info->full_def, regno);
+ }
+ else
+ {
+ machine_mode reg_mode
+ = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op);
+ auto_sbitmap range (get_nblocks (reg_mode));
+ init_range (reg->op, range);
+ remove_subreg_range (curr_bb_info, regno, range);
+ add_subreg_range_to_def (curr_bb_info, regno, range);
+ }
}
}
@@ -387,23 +433,6 @@ mark_regno_dead (int regno, machine_mode mode)
border. That might be a consequence of some global transformations
in LRA, e.g. PIC pseudo reuse or rematerialization. */
-/* Structure describing local BB data used for pseudo
- live-analysis. */
-class bb_data_pseudos
-{
-public:
- /* Basic block about which the below data are. */
- basic_block bb;
- bitmap_head killed_pseudos; /* pseudos killed in the BB. */
- bitmap_head gen_pseudos; /* pseudos generated in the BB. */
-};
-
-/* Array for all BB data. Indexed by the corresponding BB index. */
-typedef class bb_data_pseudos *bb_data_t;
-
-/* All basic block data are referred through the following array. */
-static bb_data_t bb_data;
-
/* Two small functions for access to the bb data. */
static inline bb_data_t
get_bb_data (basic_block bb)
@@ -429,14 +458,73 @@ static bitmap_head all_hard_regs_bitmap;
static bool
live_trans_fun (int bb_index)
{
- basic_block bb = get_bb_data_by_index (bb_index)->bb;
- bitmap bb_liveout = df_get_live_out (bb);
- bitmap bb_livein = df_get_live_in (bb);
- bb_data_t bb_info = get_bb_data (bb);
-
- bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap);
- return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos,
- &temp_bitmap, &bb_info->killed_pseudos);
+ bb_data_t local_bb_info = get_bb_data_by_index (bb_index);
+ bitmap full_in = df_get_subreg_live_full_in (local_bb_info->bb);
+ bitmap full_out = df_get_subreg_live_full_out (local_bb_info->bb);
+
+ bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap);
+ bool changed = bitmap_ior_and_compl (full_in, &local_bb_info->full_use,
+ &temp_bitmap, &local_bb_info->full_def);
+
+ /* Handle partial live case. */
+ if (flag_track_subreg_liveness)
+ {
+ bitmap partial_in = df_get_subreg_live_partial_in (local_bb_info->bb);
+ bitmap partial_out = df_get_subreg_live_partial_out (local_bb_info->bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (local_bb_info->bb);
+ subregs_live *range_out
+ = df_get_subreg_live_range_out (local_bb_info->bb);
+
+ if (!bitmap_empty_p (partial_out)
+ || !bitmap_empty_p (&local_bb_info->partial_use))
+ {
+ unsigned int regno;
+ bitmap_iterator bi;
+ bitmap_head temp_partial_out;
+ subregs_live temp_range_out;
+
+ /* TEMP = (OUT & ~DEF) */
+ bitmap_initialize (&temp_partial_out, &bitmap_default_obstack);
+ EXECUTE_IF_SET_IN_BITMAP (partial_out, FIRST_PSEUDO_REGISTER, regno,
+ bi)
+ {
+ sbitmap out_range = range_out->get_range (regno);
+ temp_range_out.add_range (regno, out_range);
+ if (bitmap_bit_p (&local_bb_info->partial_def, regno))
+ {
+ sbitmap def_range
+ = local_bb_info->range_def->get_range (regno);
+ temp_range_out.remove_range (regno, def_range);
+ if (!temp_range_out.empty_p (regno))
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+ else
+ bitmap_set_bit (&temp_partial_out, regno);
+ }
+
+ /* TEMP = USE | TEMP */
+ EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ {
+ sbitmap use_range = local_bb_info->range_use->get_range (regno);
+ temp_range_out.add_range (regno, use_range);
+ }
+ bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use);
+
+ /* IN = TEMP */
+ changed |= range_in->copy_lives (temp_range_out);
+ bitmap_copy (partial_in, &temp_partial_out);
+ df_live_subreg_check_result (full_in, partial_in, range_in);
+ }
+ else if (!bitmap_empty_p (partial_in))
+ {
+ changed = true;
+ bitmap_clear (partial_in);
+ range_in->clear ();
+ }
+ }
+
+ return changed;
}
/* The confluence function used by the DF equation solver to set up
@@ -444,7 +532,9 @@ live_trans_fun (int bb_index)
static void
live_con_fun_0 (basic_block bb)
{
- bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap);
+ bitmap_and_into (df_get_subreg_live_out (bb), &all_hard_regs_bitmap);
+ if (flag_track_subreg_liveness)
+ bitmap_and_into (df_get_subreg_live_full_out (bb), &all_hard_regs_bitmap);
}
/* The confluence function used by the DF equation solver to propagate
@@ -456,13 +546,37 @@ live_con_fun_0 (basic_block bb)
static bool
live_con_fun_n (edge e)
{
- basic_block bb = e->src;
- basic_block dest = e->dest;
- bitmap bb_liveout = df_get_live_out (bb);
- bitmap dest_livein = df_get_live_in (dest);
+ bitmap src_full_out = df_get_subreg_live_full_out (e->src);
+ bitmap dest_full_in = df_get_subreg_live_full_in (e->dest);
+
+ bool changed = bitmap_ior_and_compl_into (src_full_out, dest_full_in,
+ &all_hard_regs_bitmap);
+ /* Handle partial live case. */
+ if (flag_track_subreg_liveness)
+ {
+ bitmap src_partial_out = df_get_subreg_live_partial_out (e->src);
+ subregs_live *src_range_out = df_get_subreg_live_range_out (e->src);
+ bitmap dest_partial_in = df_get_subreg_live_partial_in (e->dest);
+ subregs_live *dest_range_in = df_get_subreg_live_range_in (e->dest);
+
+ if (bitmap_empty_p (dest_partial_in))
+ return changed;
+
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_SET_IN_BITMAP (dest_partial_in, FIRST_PSEUDO_REGISTER, regno,
+ bi)
+ {
+ sbitmap dest_range = dest_range_in->get_range (regno);
+ changed |= src_range_out->add_range (regno, dest_range);
+ }
+ changed |= bitmap_ior_into (src_partial_out, dest_partial_in);
- return bitmap_ior_and_compl_into (bb_liveout,
- dest_livein, &all_hard_regs_bitmap);
+ df_live_subreg_check_result (src_full_out, src_partial_out,
+ src_range_out);
+ }
+
+ return changed;
}
/* Indexes of all function blocks. */
@@ -479,12 +593,47 @@ initiate_live_solver (void)
bitmap_initialize (&all_blocks, ®_obstack);
basic_block bb;
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_initialize (&tracked_regs, ®_obstack);
+ FOR_ALL_BB_FN (bb, cfun)
+ {
+ rtx_insn *insn;
+ df_ref use;
+ FOR_BB_INSNS (bb, insn)
+ {
+ if (!NONDEBUG_INSN_P (insn))
+ continue;
+
+ df_insn_info *insn_info = DF_INSN_INFO_GET (insn);
+
+ FOR_EACH_INSN_INFO_USE (use, insn_info)
+ {
+ unsigned int regno = DF_REF_REGNO (use);
+ /* A multireg which is used via subreg pattern. */
+ if (multireg_p (regno)
+ && DF_REF_FLAGS (use) & (DF_REF_SUBREG))
+ bitmap_set_bit (&tracked_regs, regno);
+ }
+ }
+ }
+ }
+
+
FOR_ALL_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
bb_info->bb = bb;
- bitmap_initialize (&bb_info->killed_pseudos, ®_obstack);
- bitmap_initialize (&bb_info->gen_pseudos, ®_obstack);
+ bitmap_initialize (&bb_info->full_def, ®_obstack);
+ bitmap_initialize (&bb_info->full_use, ®_obstack);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_initialize (&bb_info->partial_def, ®_obstack);
+ bitmap_initialize (&bb_info->partial_use, ®_obstack);
+ size_t num_regs = bitmap_count_bits (&tracked_regs);
+ bb_info->range_def = new subregs_live (num_regs);
+ bb_info->range_use = new subregs_live (num_regs);
+ }
bitmap_set_bit (&all_blocks, bb->index);
}
}
@@ -496,11 +645,19 @@ finish_live_solver (void)
basic_block bb;
bitmap_clear (&all_blocks);
+ bitmap_clear (&tracked_regs);
FOR_ALL_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
- bitmap_clear (&bb_info->killed_pseudos);
- bitmap_clear (&bb_info->gen_pseudos);
+ bitmap_clear (&bb_info->full_def);
+ bitmap_clear (&bb_info->full_use);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear (&bb_info->partial_def);
+ bitmap_clear (&bb_info->partial_use);
+ delete bb_info->range_def;
+ delete bb_info->range_use;
+ }
}
free (bb_data);
bitmap_clear (&all_hard_regs_bitmap);
@@ -663,7 +820,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
/* Only has a meaningful value once we've seen a call. */
function_abi last_call_abi = default_function_abi;
- reg_live_out = df_get_live_out (bb);
+ reg_live_out = df_get_subreg_live_out (bb);
sparseset_clear (pseudos_live);
sparseset_clear (pseudos_live_through_calls);
sparseset_clear (pseudos_live_through_setjumps);
@@ -675,10 +832,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
mark_pseudo_live (j);
}
- bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos;
- bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos;
- bitmap_clear (bb_gen_pseudos);
- bitmap_clear (bb_killed_pseudos);
+ curr_bb_info = get_bb_data (bb);
+ bitmap_clear (&curr_bb_info->full_use);
+ bitmap_clear (&curr_bb_info->full_def);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear (&curr_bb_info->partial_use);
+ bitmap_clear (&curr_bb_info->partial_def);
+ curr_bb_info->range_use->clear ();
+ curr_bb_info->range_def->clear ();
+ }
freq = REG_FREQ_FROM_BB (bb);
if (lra_dump_file != NULL)
@@ -860,7 +1023,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
if (reg->type != OP_IN)
{
update_pseudo_point (reg->regno, curr_point, USE_POINT);
- mark_regno_live (reg->regno, reg->biggest_mode);
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
/* ??? Should be a no-op for unused registers. */
check_pseudos_live_through_calls (reg->regno, last_call_abi);
}
@@ -886,7 +1049,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_OUT)
update_pseudo_point (reg->regno, curr_point, DEF_POINT);
- mark_regno_dead (reg->regno, reg->biggest_mode);
+ mark_regno_dead (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -945,8 +1108,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_IN)
update_pseudo_point (reg->regno, curr_point, USE_POINT);
- mark_regno_live (reg->regno, reg->biggest_mode);
check_pseudos_live_through_calls (reg->regno, last_call_abi);
+ /* Ignore the use of subreg which is used as dest operand. */
+ if (need_track_subreg_p (reg->regno) && reg->subreg_p
+ && reg->type == OP_INOUT)
+ continue;
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -970,12 +1137,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
{
if (reg->type == OP_OUT)
update_pseudo_point (reg->regno, curr_point, DEF_POINT);
- mark_regno_dead (reg->regno, reg->biggest_mode);
+ mark_regno_dead (reg->regno, reg->biggest_mode, reg);
/* We're done processing inputs, so make sure early clobber
operands that are both inputs and outputs are still live. */
if (reg->type == OP_INOUT)
- mark_regno_live (reg->regno, reg->biggest_mode);
+ mark_regno_live (reg->regno, reg->biggest_mode, reg);
}
for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next)
@@ -1099,8 +1266,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
bool live_change_p = false;
/* Check if bb border live info was changed. */
unsigned int live_pseudos_num = 0;
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb),
- FIRST_PSEUDO_REGISTER, j, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ j, bi)
{
live_pseudos_num++;
if (! sparseset_bit_p (pseudos_live, j))
@@ -1118,7 +1285,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
live_change_p = true;
if (lra_dump_file != NULL)
EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j)
- if (! bitmap_bit_p (df_get_live_in (bb), j))
+ if (! bitmap_bit_p (df_get_subreg_live_in (bb), j))
fprintf (lra_dump_file,
" r%d is added to live at bb%d start\n", j, bb->index);
}
@@ -1133,7 +1300,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
mark_pseudo_dead (i);
}
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
+ j, bi)
{
if (sparseset_cardinality (pseudos_live_through_calls) == 0)
break;
@@ -1149,7 +1317,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i))
continue;
- if (bitmap_bit_p (df_get_live_in (bb), i))
+ if (bitmap_bit_p (df_get_subreg_live_in (bb), i))
continue;
live_change_p = true;
@@ -1157,7 +1325,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p)
fprintf (lra_dump_file,
" hard reg r%d is added to live at bb%d start\n", i,
bb->index);
- bitmap_set_bit (df_get_live_in (bb), i);
+ bitmap_set_bit (df_get_subreg_live_in (bb), i);
+ if (flag_track_subreg_liveness)
+ bitmap_set_bit (df_get_subreg_live_full_in (bb), i);
}
if (need_curr_point_incr)
@@ -1421,12 +1591,30 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
{
/* We need to clear pseudo live info as some pseudos can
disappear, e.g. pseudos with used equivalences. */
- FOR_EACH_BB_FN (bb, cfun)
+ FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun),
+ EXIT_BLOCK_PTR_FOR_FN (cfun), next_bb)
{
- bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER,
+ bitmap_clear_range (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER,
max_regno - FIRST_PSEUDO_REGISTER);
- bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER,
+ bitmap_clear_range (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER,
max_regno - FIRST_PSEUDO_REGISTER);
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_clear_range (df_get_subreg_live_full_in (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_partial_in (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_full_out (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ bitmap_clear_range (df_get_subreg_live_partial_out (bb),
+ FIRST_PSEUDO_REGISTER,
+ max_regno - FIRST_PSEUDO_REGISTER);
+ df_get_subreg_live_range_in (bb)->clear ();
+ df_get_subreg_live_range_out (bb)->clear ();
+ }
}
/* As we did not change CFG since LRA start we can use
DF-infrastructure solver to solve live data flow problem. */
@@ -1439,6 +1627,10 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
(DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n,
live_trans_fun, &all_blocks,
df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD));
+
+ if (flag_track_subreg_liveness)
+ df_live_subreg_finalize (&all_blocks);
+
if (lra_dump_file != NULL)
{
fprintf (lra_dump_file,
@@ -1447,16 +1639,33 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p)
FOR_EACH_BB_FN (bb, cfun)
{
bb_data_t bb_info = get_bb_data (bb);
- bitmap bb_livein = df_get_live_in (bb);
- bitmap bb_liveout = df_get_live_out (bb);
fprintf (lra_dump_file, "\nBB %d:\n", bb->index);
- lra_dump_bitmap_with_title (" gen:",
- &bb_info->gen_pseudos, bb->index);
- lra_dump_bitmap_with_title (" killed:",
- &bb_info->killed_pseudos, bb->index);
- lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index);
- lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index);
+ lra_dump_bitmap_with_title (" full use", &bb_info->full_use,
+ bb->index);
+ lra_dump_bitmap_with_title (" full def", &bb_info->full_def,
+ bb->index);
+ lra_dump_bitmap_with_title (" live in full",
+ df_get_subreg_live_full_in (bb),
+ bb->index);
+ lra_dump_bitmap_with_title (" live out full",
+ df_get_subreg_live_full_out (bb),
+ bb->index);
+ if (flag_track_subreg_liveness)
+ {
+ lra_dump_bitmap_with_title (" partial use",
+ &bb_info->partial_use, bb->index);
+ lra_dump_bitmap_with_title (" partial def",
+ &bb_info->partial_def, bb->index);
+ lra_dump_bitmap_with_title (" live in partial",
+ df_get_subreg_live_partial_in (
+ bb),
+ bb->index);
+ lra_dump_bitmap_with_title (" live out partial",
+ df_get_subreg_live_partial_out (
+ bb),
+ bb->index);
+ }
}
}
}
diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc
index c84bf3c9938..ef0157513b0 100644
--- a/gcc/lra-remat.cc
+++ b/gcc/lra-remat.cc
@@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void)
fprintf (lra_dump_file, "\nBB %d:\n", bb->index);
/* Livein */
fprintf (lra_dump_file, " register live in:");
- dump_regset (df_get_live_in (bb), lra_dump_file);
+ dump_regset (df_get_subreg_live_in (bb), lra_dump_file);
putc ('\n', lra_dump_file);
/* Liveout */
fprintf (lra_dump_file, " register live out:");
- dump_regset (df_get_live_out (bb), lra_dump_file);
+ dump_regset (df_get_subreg_live_out (bb), lra_dump_file);
putc ('\n', lra_dump_file);
/* Changed/dead regs: */
fprintf (lra_dump_file, " changed regs:");
@@ -727,7 +727,7 @@ calculate_livein_cands (void)
FOR_EACH_BB_FN (bb, cfun)
{
- bitmap livein_regs = df_get_live_in (bb);
+ bitmap livein_regs = df_get_subreg_live_in (bb);
bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands;
for (unsigned int i = 0; i < cands_num; i++)
{
@@ -1064,7 +1064,7 @@ do_remat (void)
FOR_EACH_BB_FN (bb, cfun)
{
CLEAR_HARD_REG_SET (live_hard_regs);
- EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi)
+ EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), 0, regno, bi)
{
int hard_regno = regno < FIRST_PSEUDO_REGISTER
? regno
diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc
index 8b1a945d632..08ec247bc00 100644
--- a/gcc/lra-spills.cc
+++ b/gcc/lra-spills.cc
@@ -566,8 +566,31 @@ spill_pseudos (void)
"Debug insn #%u is reset because it referenced "
"removed pseudo\n", INSN_UID (insn));
}
- bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos);
- bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos);
+ bitmap_and_compl_into (df_get_subreg_live_in (bb), spilled_pseudos);
+ bitmap_and_compl_into (df_get_subreg_live_out (bb), spilled_pseudos);
+
+ if (flag_track_subreg_liveness)
+ {
+ bitmap_and_compl_into (df_get_subreg_live_full_in (bb),
+ spilled_pseudos);
+ bitmap partial_in = df_get_subreg_live_partial_in (bb);
+ subregs_live *range_in = df_get_subreg_live_range_in (bb);
+ unsigned int regno;
+ bitmap_iterator bi;
+ EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ range_in->remove_range (regno);
+ bitmap_and_compl_into (partial_in, spilled_pseudos);
+
+ bitmap_and_compl_into (df_get_subreg_live_full_out (bb),
+ spilled_pseudos);
+ bitmap partial_out = df_get_subreg_live_partial_out (bb);
+ subregs_live *range_out = df_get_subreg_live_range_out (bb);
+ EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos,
+ FIRST_PSEUDO_REGISTER, regno, bi)
+ range_out->remove_range (regno);
+ bitmap_and_compl_into (partial_out, spilled_pseudos);
+ }
}
}
}
diff --git a/gcc/lra.cc b/gcc/lra.cc
index fb32e134004..a6fabe7d118 100644
--- a/gcc/lra.cc
+++ b/gcc/lra.cc
@@ -574,7 +574,7 @@ object_allocator<lra_insn_reg> lra_insn_reg_pool ("insn regs");
EARLY_CLOBBER_ALTS. */
static struct lra_insn_reg *
new_insn_reg (rtx_insn *insn, int regno, enum op_type type,
- machine_mode mode, bool subreg_p,
+ machine_mode mode, bool subreg_p, rtx op,
alternative_mask early_clobber_alts,
struct lra_insn_reg *next)
{
@@ -586,6 +586,7 @@ new_insn_reg (rtx_insn *insn, int regno, enum op_type type,
ir->subreg_p = subreg_p;
ir->early_clobber_alts = early_clobber_alts;
ir->regno = regno;
+ ir->op = op;
ir->next = next;
return ir;
}
@@ -926,7 +927,7 @@ collect_non_operand_hard_regs (rtx_insn *insn, rtx *x,
&& ! (FIRST_STACK_REG <= regno
&& regno <= LAST_STACK_REG));
#endif
- list = new_insn_reg (data->insn, regno, type, mode, subreg_p,
+ list = new_insn_reg (data->insn, regno, type, mode, subreg_p, *x,
early_clobber ? ALL_ALTERNATIVES : 0, list);
}
}
@@ -1484,6 +1485,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
code = GET_CODE (x);
mode = GET_MODE (x);
subreg_p = false;
+ rtx op = x;
if (GET_CODE (x) == SUBREG)
{
mode = wider_subreg_mode (x);
@@ -1501,7 +1503,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
if (bitmap_set_bit (&lra_reg_info[regno].insn_bitmap, INSN_UID (insn)))
{
data->regs = new_insn_reg (data->insn, regno, type, mode, subreg_p,
- early_clobber_alts, data->regs);
+ op, early_clobber_alts, data->regs);
return;
}
else
@@ -1513,7 +1515,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x,
/* The info cannot be integrated into the found
structure. */
data->regs = new_insn_reg (data->insn, regno, type, mode,
- subreg_p, early_clobber_alts,
+ subreg_p, op, early_clobber_alts,
data->regs);
else
{
--
2.36.3
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option
2024-05-12 22:57 ` [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Juzhe-Zhong
@ 2024-06-13 15:54 ` Richard Sandiford
0 siblings, 0 replies; 7+ messages in thread
From: Richard Sandiford @ 2024-06-13 15:54 UTC (permalink / raw)
To: Juzhe-Zhong; +Cc: gcc-patches, vmakarov, jin.xia, dimitar
Juzhe-Zhong <juzhe.zhong@rivai.ai> writes:
> ---
> gcc/common.opt | 4 ++++
> gcc/common.opt.urls | 3 +++
> gcc/doc/invoke.texi | 8 ++++++++
> gcc/opts.cc | 1 +
> 4 files changed, 16 insertions(+)
>
> diff --git a/gcc/common.opt b/gcc/common.opt
> index 40cab3cb36a..5710e817abe 100644
> --- a/gcc/common.opt
> +++ b/gcc/common.opt
> @@ -2163,6 +2163,10 @@ fira-share-spill-slots
> Common Var(flag_ira_share_spill_slots) Init(1) Optimization
> Share stack slots for spilled pseudo-registers.
>
> +ftrack-subreg-liveness
> +Common Var(flag_track_subreg_liveness) Init(0) Optimization
> +Track subreg liveness information.
> +
> fira-verbose=
> Common RejectNegative Joined UInteger Var(flag_ira_verbose) Init(5)
> -fira-verbose=<number> Control IRA's level of diagnostic messages.
> diff --git a/gcc/common.opt.urls b/gcc/common.opt.urls
> index f71ed80a34b..59f27a6f7c6 100644
> --- a/gcc/common.opt.urls
> +++ b/gcc/common.opt.urls
> @@ -880,6 +880,9 @@ UrlSuffix(gcc/Optimize-Options.html#index-fira-share-save-slots)
> fira-share-spill-slots
> UrlSuffix(gcc/Optimize-Options.html#index-fira-share-spill-slots)
>
> +ftrack-subreg-liveness
> +UrlSuffix(gcc/Optimize-Options.html#index-ftrack-subreg-liveness)
> +
> fira-verbose=
> UrlSuffix(gcc/Developer-Options.html#index-fira-verbose)
>
> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index ddcd5213f06..fbcde8aa745 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -13188,6 +13188,14 @@ Disable sharing of stack slots allocated for pseudo-registers. Each
> pseudo-register that does not get a hard register gets a separate
> stack slot, and as a result function stack frames are larger.
>
> +@opindex ftrack-subreg-liveness
> +@item -ftrack-subreg-liveness
> +Enable tracking subreg liveness information. This infomation allows IRA
> +and LRA to support subreg coalesce feature which can improve the quality
> +of register allocation.
> +
> +This option is enabled at level @option{-O3} for all targets.
> +
This is a good description, but some of these GCC terms might not be
familiar to users. How about something like:
------------------------------------------------------------
Enable a more precise form of dataflow analysis. This analysis focuses
on values that occupy multiple consecutive machine registers; examples
of such values include complex numbers and small tuples of vectors.
The analysis detects which parts of a value are in use at a given time
and which parts are free to be reused for other things. Enabling the
analysis can improve the quality of register allocation.
This option is enabled at level @option{-O3} for all targets.
------------------------------------------------------------
It might be worth enabling at -O2 and above eventually, but I agree
it makes sense to start with -O3.
OK with that change if you agree and if there are no countersuggestions
from others.
Thanks,
Richard
> @opindex flra-remat
> @item -flra-remat
> Enable CFG-sensitive rematerialization in LRA. Instead of loading
> diff --git a/gcc/opts.cc b/gcc/opts.cc
> index 14d1767e48f..8fe3a213807 100644
> --- a/gcc/opts.cc
> +++ b/gcc/opts.cc
> @@ -698,6 +698,7 @@ static const struct default_options default_options_table[] =
> { OPT_LEVELS_3_PLUS, OPT_funswitch_loops, NULL, 1 },
> { OPT_LEVELS_3_PLUS, OPT_fvect_cost_model_, NULL, VECT_COST_MODEL_DYNAMIC },
> { OPT_LEVELS_3_PLUS, OPT_fversion_loops_for_strides, NULL, 1 },
> + { OPT_LEVELS_3_PLUS, OPT_ftrack_subreg_liveness, NULL, 1 },
>
> /* -O3 parameters. */
> { OPT_LEVELS_3_PLUS, OPT__param_max_inline_insns_auto_, NULL, 30 },
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem
2024-05-12 22:57 ` [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem Juzhe-Zhong
@ 2024-06-13 15:57 ` Richard Sandiford
0 siblings, 0 replies; 7+ messages in thread
From: Richard Sandiford @ 2024-06-13 15:57 UTC (permalink / raw)
To: Juzhe-Zhong; +Cc: gcc-patches, vmakarov, jin.xia, dimitar
Thanks for the update!
In terms of high-level comments:
- It looks like the code processes normal pseudos as well as subregs.
Is that necessary? The handling of single-register pseudos should
be fully redundant with the normal liveness tracking.
I think if possible we should just process pseudos for which
need_track_subreg_p is true.
- It looks like the information uses a map from register numbers to
sbitmaps. That seems like quite an expensive structure.
The way I'd imagined it working, we'd instead precompute a mapping
from pseudo registers to a zero-based "subreg id". A pseudo that
has N blocks would have N consecutive subreg ids. We could then
have a single bitmap (rather than sbitmap) for all subreg ids.
For example, if we have:
P == FIRST_PSEUDO_REGISTER
reg P: 2 subregisters: subreg ids [0, 1]
reg P+1: 1 subregister: ignored
reg P+2: 4 subregisters: subreg ids [2, 5]
reg P+3: 1 subregister: ignored
The map from pseudo registers to subreg ids could be linear array or
a hash map. I don't have a good intuition for which would be better,
but a linear array sounds like the best starting point.
Some individual comments below:
Juzhe-Zhong <juzhe.zhong@rivai.ai> writes:
> ---
> gcc/Makefile.in | 1 +
> gcc/df-problems.cc | 886 ++++++++++++++++++++++++++++++++++++++-
> gcc/df.h | 159 +++++++
> gcc/regs.h | 5 +
> gcc/sbitmap.cc | 98 +++++
> gcc/sbitmap.h | 2 +
> gcc/subreg-live-range.cc | 233 ++++++++++
> gcc/subreg-live-range.h | 60 +++
> gcc/timevar.def | 1 +
> 9 files changed, 1444 insertions(+), 1 deletion(-)
> create mode 100644 gcc/subreg-live-range.cc
> create mode 100644 gcc/subreg-live-range.h
>
> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
> index a7f15694c34..67d2e3ca1bc 100644
> --- a/gcc/Makefile.in
> +++ b/gcc/Makefile.in
> @@ -1684,6 +1684,7 @@ OBJS = \
> store-motion.o \
> streamer-hooks.o \
> stringpool.o \
> + subreg-live-range.o \
> substring-locations.o \
> target-globals.o \
> targhooks.o \
> diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc
> index 88ee0dd67fc..01f1f850925 100644
> --- a/gcc/df-problems.cc
> +++ b/gcc/df-problems.cc
> @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see
> #include "target.h"
> #include "rtl.h"
> #include "df.h"
> +#include "subreg-live-range.h"
> #include "memmodel.h"
> #include "tm_p.h"
> #include "insn-config.h"
> @@ -1344,8 +1345,891 @@ df_lr_verify_transfer_functions (void)
> bitmap_clear (&all_blocks);
> }
>
> +/*----------------------------------------------------------------------------
> + REGISTER AND SUBREGS LIVES
> + Like DF_LR, but include tracking subreg liveness. Currently used to provide
> + subreg liveness related information to the register allocator. The subreg
> + information is currently tracked for registers that satisfy the following
> + conditions:
> + 1. REG is a pseudo register
> + 2. MODE_SIZE > UNIT_SIZE
> + 3. MODE_SIZE is a multiple of UNIT_SIZE
> + 4. REG is used via subreg pattern
> + Assuming: MODE = the machine mode of the REG
> + MODE_SIZE = GET_MODE_SIZE (MODE)
> + UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE)
> + Condition 3 is currently strict, maybe it can be removed in the future, but
> + for now it is sufficient.
> +----------------------------------------------------------------------------*/
> +
> +/* These two empty data are used as default data in case the user does not turn
> + * on the track-subreg-liveness feature. */
Nit: should be no leading "*" on this line.
Maybe:
/* Data for an empty subreg problem, for cases in which subreg tracking
is not enabled. */
> +bitmap_head df_subreg_empty_bitmap;
> +subregs_live df_subreg_empty_live;
> +
> +/* Private data for live_subreg problem. */
> +struct df_live_subreg_problem_data
> +{
> + /* Record registers that need to track subreg liveness. */
Maybe:
/* The set of pseudo registers to track. */
But with the linear array described above, it would be simpler to check
whether the subreg id >= 0.
> + bitmap_head tracked_regs;
> + /* An obstack for the bitmaps we need for this problem. */
> + bitmap_obstack live_subreg_bitmaps;
> +};
> +
> +/* Helper functions. */
> +
> +static df_live_subreg_bb_info *
> +df_live_subreg_get_bb_info (unsigned int index)
> +{
> + if (index < df_live_subreg->block_info_size)
> + return &static_cast<df_live_subreg_bb_info *> (
> + df_live_subreg->block_info)[index];
> + else
> + return nullptr;
> +}
> +
> +static df_live_subreg_local_bb_info *
> +get_live_subreg_local_bb_info (unsigned int bb_index)
> +{
> + return df_live_subreg_get_bb_info (bb_index);
> +}
> +
> +/* Return true if regno is a multireg. */
> +bool
> +multireg_p (int regno)
> +{
> + if (regno < FIRST_PSEUDO_REGISTER)
> + return false;
> + rtx regno_rtx = regno_reg_rtx[regno];
> + machine_mode reg_mode = GET_MODE (regno_rtx);
> + poly_int64 total_size = GET_MODE_SIZE (reg_mode);
> + poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode);
> + return maybe_gt (total_size, natural_size)
> + && multiple_p (total_size, natural_size);
> +}
How about replacing this and get_nblocks with the following two new routines:
/* Return the number of hard registers that are normally used to store
a value of mode MODE. */
unsigned int
regmode_natural_nregs (machine_mode mode)
{
poly_uint64 total_size = GET_MODE_SIZE (mode);
poly_uint64 natural_size = REGMODE_NATURAL_SIZE (mode);
unsigned int nregs;
/* REGMODE_NATURAL_SIZE must be defined such that total_size and
natural_size are ordered. */
if (!can_div_away_from_zero_p (total_size, natural_size, &nregs))
gcc_unreachable ();
return nregs;
}
/* If register REGNO is a pseudo register, return the number of hard registers
that are normally used to store it. Return 1 otherwise. */
unsigned int
regno_natural_nregs (unsigned int regno)
{
if (regno < FIRST_PSEUDO_REGISTER)
return 1;
return regmode_natural_nregs (PSEUDO_REGNO_MODE (regno));
}
I think this ought to go in rtlanal.cc, perhaps after read_modify_subreg_p.
Then multireg_p (regno) can be replaced by regno_natural_nregs (regno) > 1.
> +
> +/* Return true if the REGNO need be track with subreg liveness. */
> +
> +static bool
> +need_track_subreg_p (unsigned regno)
> +{
> + auto problem_data
> + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data;
> + return bitmap_bit_p (&problem_data->tracked_regs, regno);
> +}
> +
> +/* Fill RANGE with the subreg range for OP in REGMODE_NATURAL_SIZE granularity.
> + */
> +void
> +init_range (rtx op, sbitmap range)
> +{
> + rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op;
> + machine_mode reg_mode = GET_MODE (reg);
> +
> + if (!read_modify_subreg_p (op))
> + {
> + bitmap_set_range (range, 0, get_nblocks (reg_mode));
> + return;
> + }
> +
> + rtx subreg = op;
> + machine_mode subreg_mode = GET_MODE (subreg);
> + poly_int64 offset = SUBREG_BYTE (subreg);
> + int nblocks = get_nblocks (reg_mode);
> + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode);
> + poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode);
> + poly_int64 left = offset + subreg_size;
> +
> + int subreg_start = -1;
> + int subreg_nblocks = -1;
> + for (int i = 0; i < nblocks; i += 1)
> + {
> + poly_int64 right = unit_size * (i + 1);
> + if (subreg_start < 0 && maybe_lt (offset, right))
> + subreg_start = i;
> + if (subreg_nblocks < 0 && maybe_le (left, right))
> + {
> + subreg_nblocks = i + 1 - subreg_start;
> + break;
> + }
> + }
> + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0);
> +
> + bitmap_set_range (range, subreg_start, subreg_nblocks);
I might be wrong, but I think this can be written:
/* Verified by validate_subreg. */
if (!can_div_trunc_p (offset, unit_size, &subreg_start)
|| !can_div_away_from_zero_p (offset + subreg_size, unit_size,
&subreg_end))
gcc_unreachable ();
bitmap_set_range (range, subreg_start, subreg_end - subreg_start);
Thanks,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-06-13 15:57 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-05-12 22:57 [SUBREG V4 0/4] Add DF_LIVE_SUBREG data and apply to IRA and LRA Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 1/4] DF: Add -ftrack-subreg-liveness option Juzhe-Zhong
2024-06-13 15:54 ` Richard Sandiford
2024-05-12 22:57 ` [SUBREG V4 2/4] DF: Add DF_LIVE_SUBREG problem Juzhe-Zhong
2024-06-13 15:57 ` Richard Sandiford
2024-05-12 22:57 ` [SUBREG V4 3/4] IRA: Apply DF_LIVE_SUBREG data Juzhe-Zhong
2024-05-12 22:57 ` [SUBREG V4 4/4] LRA: " Juzhe-Zhong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).