From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpbg151.qq.com (smtpbg151.qq.com [18.169.211.239]) by sourceware.org (Postfix) with ESMTPS id B83B3385770F for ; Wed, 8 Nov 2023 03:48:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B83B3385770F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org B83B3385770F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=18.169.211.239 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699415295; cv=none; b=XaKLq8h19YEaGPOEZRTB3tsffyFBJvr8FV7JFhD+TXQulPbq+OfmKuyoiEPz7oViopOicCqrNf4WisdGy5zbzeeFHOZ3Uf9h5eaMnu4/G0pevAUFnZsi1m9FgqPO7znzMeIQ07E05e3jxkfqFdz78GMiZ9Pi82X3wi5dkd+9SN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699415295; c=relaxed/simple; bh=QDLKqw5pxd6ZInC0pMUbSJ39KCHm8xGXRLguKnyjv3w=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=q4TXttylJDN6yoJ/FJHfGL4MdXKHH1eItti7HRGPCmV8+i9XyOnQ2EMKVgVyFlVvlSJgqe1JHpxpPIpkSEUHD23+iP8ZXBY+XnbRe31EFtZUhUtWVdLC63/2LuvbuDgWeWmRHQVUvBGy77QhYP2kswu3exXVspMELxVz8G1GQk0= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp81t1699415281t86jul3l Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 08 Nov 2023 11:48:01 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: U4uNK7LTfdqUua7ceeKWuZx+jKyOIN/aUg9v08DWLXXxDzyNMqy82eod0M20Y I1qUg+EWROxCbj2aGdOr8FFSZEBso6GT7kjXuOg/OfnPn/Wu/KO0YKfYi+LWbT6Ovl0IEZT o8WTaAJD6nqgDkQroppqP+hnmsssVYy5+am4LfiIWoZaKrpM3PMNyV3y7+/NpUjikCWyZrC oBlcauxOfTSGbJIoLhUWfj+YHz4JDA7hyf9mWSHSocGsHM185Ay1G0P42I1ag2wOa7uwfKD KSnZAEILl6ZdBbOL89M9pQWq7SDbpaC+U426Ja3L9tp07DBpGahR9/b+7QGBUyE5/qvwNGr hX1UEpORfPsxRKhR5Lya7pFYX/rA/VJmAKjoB28RsisqWh96fyYC0a9b4FY16U/9oMp7i+i X-QQ-GoodBg: 2 X-BIZMAIL-ID: 2027587207807180183 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 6/7] lra: Apply live_subreg df_problem to lra pass Date: Wed, 8 Nov 2023 11:47:39 +0800 Message-Id: <20231108034740.834590-7-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20231108034740.834590-1-lehua.ding@rivai.ai> References: <20231108034740.834590-1-lehua.ding@rivai.ai> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00,GIT_PATCH_0,KAM_DMARC_STATUS,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This patch change the use of old live data to the new live_subreg data. gcc/ChangeLog: * lra-coalesce.cc (update_live_info): Update. (lra_coalesce): Update. * lra-constraints.cc (update_ebb_live_info): Update. (get_live_on_other_edges): Update. (inherit_in_ebb): Update. (lra_inheritance): Update. (fix_bb_live_info): Update. (remove_inheritance_pseudos): Update. * lra-lives.cc (make_hard_regno_live): Update. (make_hard_regno_dead): Update. (mark_regno_live): Update. (mark_regno_dead): Update. (class bb_data_pseudos): Update. (live_trans_fun): Update. (live_con_fun_0): Update. (live_con_fun_n): Update. (initiate_live_solver): Update. (finish_live_solver): Update. (process_bb_lives): Update. (lra_create_live_ranges_1): Update. * lra-remat.cc (dump_candidates_and_remat_bb_data): Update. (calculate_livein_cands): Update. (do_remat): Update. * lra-spills.cc (spill_pseudos): Update. --- gcc/lra-coalesce.cc | 20 ++- gcc/lra-constraints.cc | 93 ++++++++++--- gcc/lra-lives.cc | 308 ++++++++++++++++++++++++++++++++--------- gcc/lra-remat.cc | 13 +- gcc/lra-spills.cc | 22 ++- 5 files changed, 354 insertions(+), 102 deletions(-) diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc index 04a5bbd714b..abfc54f1cc2 100644 --- a/gcc/lra-coalesce.cc +++ b/gcc/lra-coalesce.cc @@ -188,19 +188,25 @@ static bitmap_head used_pseudos_bitmap; /* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info bitmap). */ static void -update_live_info (bitmap lr_bitmap) +update_live_info (bitmap all, bitmap full, bitmap partial) { unsigned int j; bitmap_iterator bi; bitmap_clear (&used_pseudos_bitmap); - EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap, + EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all, FIRST_PSEUDO_REGISTER, j, bi) bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]); if (! bitmap_empty_p (&used_pseudos_bitmap)) { - bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap); - bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap); + bitmap_and_compl_into (all, &coalesced_pseudos_bitmap); + bitmap_ior_into (all, &used_pseudos_bitmap); + + bitmap_and_compl_into (full, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial); + + bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full); } } @@ -303,8 +309,10 @@ lra_coalesce (void) bitmap_initialize (&used_pseudos_bitmap, ®_obstack); FOR_EACH_BB_FN (bb, cfun) { - update_live_info (df_get_live_in (bb)); - update_live_info (df_get_live_out (bb)); + update_live_info (DF_LIVE_SUBREG_IN (bb), DF_LIVE_SUBREG_FULL_IN (bb), + DF_LIVE_SUBREG_PARTIAL_IN (bb)); + update_live_info (DF_LIVE_SUBREG_OUT (bb), DF_LIVE_SUBREG_FULL_OUT (bb), + DF_LIVE_SUBREG_PARTIAL_OUT (bb)); FOR_BB_INSNS_SAFE (bb, insn, next) if (INSN_P (insn) && bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn))) diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc index 0607c8be7cb..c3ad846b97b 100644 --- a/gcc/lra-constraints.cc +++ b/gcc/lra-constraints.cc @@ -6571,34 +6571,75 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail) { if (prev_bb != NULL) { - /* Update df_get_live_in (prev_bb): */ + /* Update subreg live (prev_bb): */ + bitmap subreg_all_in = DF_LIVE_SUBREG_IN (prev_bb); + bitmap subreg_full_in = DF_LIVE_SUBREG_FULL_IN (prev_bb); + bitmap subreg_partial_in = DF_LIVE_SUBREG_PARTIAL_IN (prev_bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (prev_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) if (bitmap_bit_p (&live_regs, j)) - bitmap_set_bit (df_get_live_in (prev_bb), j); - else - bitmap_clear_bit (df_get_live_in (prev_bb), j); + { + bitmap_set_bit (subreg_all_in, j); + bitmap_set_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_live (j); + } + } + else if (bitmap_bit_p (subreg_all_in, j)) + { + bitmap_clear_bit (subreg_all_in, j); + bitmap_clear_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_live (j); + } + } } + bitmap subreg_all_out = DF_LIVE_SUBREG_OUT (curr_bb); if (curr_bb != last_bb) { - /* Update df_get_live_out (curr_bb): */ + /* Update subreg live (curr_bb): */ + bitmap subreg_all_out = DF_LIVE_SUBREG_OUT (curr_bb); + bitmap subreg_full_out = DF_LIVE_SUBREG_FULL_OUT (curr_bb); + bitmap subreg_partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (curr_bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (curr_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) { live_p = bitmap_bit_p (&live_regs, j); if (! live_p) FOR_EACH_EDGE (e, ei, curr_bb->succs) - if (bitmap_bit_p (df_get_live_in (e->dest), j)) + if (bitmap_bit_p (DF_LIVE_SUBREG_IN (e->dest), j)) { live_p = true; break; } if (live_p) - bitmap_set_bit (df_get_live_out (curr_bb), j); - else - bitmap_clear_bit (df_get_live_out (curr_bb), j); + { + bitmap_set_bit (subreg_all_out, j); + bitmap_set_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_live (j); + } + } + else if (bitmap_bit_p (subreg_all_out, j)) + { + bitmap_clear_bit (subreg_all_out, j); + bitmap_clear_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_live (j); + } + } } } prev_bb = curr_bb; - bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb)); + bitmap_and (&live_regs, &check_only_regs, subreg_all_out); } if (! NONDEBUG_INSN_P (curr_insn)) continue; @@ -6715,7 +6756,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res) bitmap_clear (res); FOR_EACH_EDGE (e, ei, from->succs) if (e->dest != to) - bitmap_ior_into (res, df_get_live_in (e->dest)); + bitmap_ior_into (res, DF_LIVE_SUBREG_IN (e->dest)); last = get_last_insertion_point (from); if (! JUMP_P (last)) return; @@ -6787,7 +6828,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We are at the end of BB. Add qualified living pseudos for potential splitting. */ - to_process = df_get_live_out (curr_bb); + to_process = DF_LIVE_SUBREG_OUT (curr_bb); if (last_processed_bb != NULL) { /* We are somewhere in the middle of EBB. */ @@ -7159,7 +7200,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We reached the beginning of the current block -- do rest of spliting in the current BB. */ - to_process = df_get_live_in (curr_bb); + to_process = DF_LIVE_SUBREG_IN (curr_bb); if (BLOCK_FOR_INSN (head) != curr_bb) { /* We are somewhere in the middle of EBB. */ @@ -7236,7 +7277,7 @@ lra_inheritance (void) fprintf (lra_dump_file, "EBB"); /* Form a EBB starting with BB. */ bitmap_clear (&ebb_global_regs); - bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb)); + bitmap_ior_into (&ebb_global_regs, DF_LIVE_SUBREG_IN (bb)); for (;;) { if (lra_dump_file != NULL) @@ -7252,7 +7293,7 @@ lra_inheritance (void) break; bb = bb->next_bb; } - bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb)); + bitmap_ior_into (&ebb_global_regs, DF_LIVE_SUBREG_OUT (bb)); if (lra_dump_file != NULL) fprintf (lra_dump_file, "\n"); if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb))) @@ -7281,15 +7322,23 @@ int lra_undo_inheritance_iter; /* Fix BB live info LIVE after removing pseudos created on pass doing inheritance/split which are REMOVED_PSEUDOS. */ static void -fix_bb_live_info (bitmap live, bitmap removed_pseudos) +fix_bb_live_info (bitmap all, bitmap full, bitmap partial, + bitmap removed_pseudos) { unsigned int regno; bitmap_iterator bi; EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi) - if (bitmap_clear_bit (live, regno) - && REG_P (lra_reg_info[regno].restore_rtx)) - bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx)); + { + if (bitmap_clear_bit (all, regno) + && REG_P (lra_reg_info[regno].restore_rtx)) + { + bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx)); + bitmap_clear_bit (full, regno); + bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx)); + gcc_assert (!bitmap_bit_p (partial, regno)); + } + } } /* Return regno of the (subreg of) REG. Otherwise, return a negative @@ -7355,8 +7404,10 @@ remove_inheritance_pseudos (bitmap remove_pseudos) constraint pass. */ FOR_EACH_BB_FN (bb, cfun) { - fix_bb_live_info (df_get_live_in (bb), remove_pseudos); - fix_bb_live_info (df_get_live_out (bb), remove_pseudos); + fix_bb_live_info (DF_LIVE_SUBREG_IN (bb), DF_LIVE_SUBREG_FULL_IN (bb), + DF_LIVE_SUBREG_PARTIAL_IN (bb), remove_pseudos); + fix_bb_live_info (DF_LIVE_SUBREG_OUT (bb), DF_LIVE_SUBREG_FULL_OUT (bb), + DF_LIVE_SUBREG_PARTIAL_OUT (bb), remove_pseudos); FOR_BB_INSNS_REVERSE (bb, curr_insn) { if (! INSN_P (curr_insn)) diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc index f60e564da82..477b82786cf 100644 --- a/gcc/lra-lives.cc +++ b/gcc/lra-lives.cc @@ -287,7 +287,7 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); sparseset_set_bit (start_living, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); } /* Process the definition of hard register REGNO. This updates @@ -310,8 +310,8 @@ make_hard_regno_dead (int regno) sparseset_set_bit (start_dying, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) { - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); } } @@ -355,7 +355,9 @@ mark_regno_live (int regno, machine_mode mode) else { mark_pseudo_live (regno); - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_def, regno)); } } @@ -375,8 +377,10 @@ mark_regno_dead (int regno, machine_mode mode) else { mark_pseudo_dead (regno); - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&curr_bb_info->partial_def, regno)); } } @@ -387,23 +391,6 @@ mark_regno_dead (int regno, machine_mode mode) border. That might be a consequence of some global transformations in LRA, e.g. PIC pseudo reuse or rematerialization. */ -/* Structure describing local BB data used for pseudo - live-analysis. */ -class bb_data_pseudos -{ -public: - /* Basic block about which the below data are. */ - basic_block bb; - bitmap_head killed_pseudos; /* pseudos killed in the BB. */ - bitmap_head gen_pseudos; /* pseudos generated in the BB. */ -}; - -/* Array for all BB data. Indexed by the corresponding BB index. */ -typedef class bb_data_pseudos *bb_data_t; - -/* All basic block data are referred through the following array. */ -static bb_data_t bb_data; - /* Two small functions for access to the bb data. */ static inline bb_data_t get_bb_data (basic_block bb) @@ -430,13 +417,93 @@ static bool live_trans_fun (int bb_index) { basic_block bb = get_bb_data_by_index (bb_index)->bb; - bitmap bb_liveout = df_get_live_out (bb); - bitmap bb_livein = df_get_live_in (bb); + bitmap full_out = DF_LIVE_SUBREG_FULL_OUT (bb); + bitmap full_in = DF_LIVE_SUBREG_FULL_IN (bb); + bitmap partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (bb); + bitmap partial_in = DF_LIVE_SUBREG_PARTIAL_IN (bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (bb); bb_data_t bb_info = get_bb_data (bb); - bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap); - return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos, - &temp_bitmap, &bb_info->killed_pseudos); + if (!has_subreg_live_p) + { + bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap); + return bitmap_ior_and_compl (full_in, &bb_info->full_use, &temp_bitmap, + &bb_info->full_def); + } + + /* If there has subreg live need be tracked. */ + unsigned int regno; + bitmap_iterator bi; + bool changed = false; + bitmap_head temp_full_out; + bitmap_head temp_partial_out; + bitmap_head temp_partial_be_full_out; + bitmap_head all_def; + subregs_live temp_range_out; + bitmap_initialize (&temp_full_out, ®_obstack); + bitmap_initialize (&temp_partial_out, ®_obstack); + bitmap_initialize (&temp_partial_be_full_out, ®_obstack); + bitmap_initialize (&all_def, ®_obstack); + + bitmap_and_compl (&temp_full_out, full_out, &all_hard_regs_bitmap); + + bitmap_ior (&all_def, &bb_info->full_def, &bb_info->partial_def); + + bitmap_and (&temp_partial_out, &temp_full_out, &bb_info->partial_def); + EXECUTE_IF_SET_IN_BITMAP (&temp_partial_out, FIRST_PSEUDO_REGISTER, regno, bi) + { + subreg_ranges temp (bb_info->range_def->lives.at (regno).max); + temp.make_full (); + temp.remove_ranges (bb_info->range_def->lives.at (regno)); + temp_range_out.add_ranges (regno, temp); + } + bitmap_ior_and_compl_into (&temp_partial_out, partial_out, &all_def); + EXECUTE_IF_AND_COMPL_IN_BITMAP (partial_out, &all_def, FIRST_PSEUDO_REGISTER, + regno, bi) + { + temp_range_out.add_ranges (regno, range_out->lives.at (regno)); + } + EXECUTE_IF_AND_IN_BITMAP (partial_out, &bb_info->partial_def, 0, regno, bi) + { + subreg_ranges temp = range_out->lives.at (regno); + temp.remove_ranges (bb_info->range_def->lives.at (regno)); + if (!temp.empty_p ()) + { + bitmap_set_bit (&temp_partial_out, regno); + temp_range_out.add_ranges (regno, temp); + } + } + + temp_range_out.add_lives (*bb_info->range_use); + EXECUTE_IF_AND_IN_BITMAP (&temp_partial_out, &bb_info->partial_use, 0, regno, + bi) + { + subreg_ranges temp = temp_range_out.lives.at (regno); + temp.add_ranges (bb_info->range_use->lives.at (regno)); + if (temp.full_p ()) + { + bitmap_set_bit (&temp_partial_be_full_out, regno); + temp_range_out.remove_live (regno); + } + } + + bitmap_ior_and_compl_into (&temp_partial_be_full_out, &temp_full_out, + &all_def); + changed + |= bitmap_ior (full_in, &temp_partial_be_full_out, &bb_info->full_use); + + bitmap_ior_into (&temp_partial_out, &bb_info->partial_use); + changed |= bitmap_and_compl (partial_in, &temp_partial_out, + &temp_partial_be_full_out); + changed |= range_in->copy_lives (temp_range_out); + + bitmap_clear (&temp_full_out); + bitmap_clear (&temp_partial_out); + bitmap_clear (&temp_partial_be_full_out); + bitmap_clear (&all_def); + + return changed; } /* The confluence function used by the DF equation solver to set up @@ -444,7 +511,9 @@ live_trans_fun (int bb_index) static void live_con_fun_0 (basic_block bb) { - bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_OUT (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_FULL_OUT (bb), &all_hard_regs_bitmap); + bitmap_and_into (DF_LIVE_SUBREG_PARTIAL_OUT (bb), &all_hard_regs_bitmap); } /* The confluence function used by the DF equation solver to propagate @@ -456,13 +525,77 @@ live_con_fun_0 (basic_block bb) static bool live_con_fun_n (edge e) { - basic_block bb = e->src; - basic_block dest = e->dest; - bitmap bb_liveout = df_get_live_out (bb); - bitmap dest_livein = df_get_live_in (dest); + class df_live_subreg_bb_info *src_bb_info + = df_live_subreg_get_bb_info (e->src->index); + class df_live_subreg_bb_info *dest_bb_info + = df_live_subreg_get_bb_info (e->dest->index); - return bitmap_ior_and_compl_into (bb_liveout, - dest_livein, &all_hard_regs_bitmap); + if (!has_subreg_live_p) + { + return bitmap_ior_and_compl_into (&src_bb_info->full_out, + &dest_bb_info->full_in, + &all_hard_regs_bitmap); + } + + /* If there has subreg live need be tracked. Calculation formula: + temp_full mean: + 1. partial in out/in, full in other in/out + 2. partial in out and in, and mrege range is full + temp_range mean: + the range of regno which partial live + src_bb_info->partial_out = (src_bb_info->partial_out | + dest_bb_info->partial_in) & ~temp_full src_bb_info->range_out = copy + (temp_range) src_bb_info->full_out |= dest_bb_info->full_in | temp_full + */ + subregs_live temp_range; + temp_range.add_lives (*src_bb_info->range_out); + temp_range.add_lives (*dest_bb_info->range_in); + + bitmap_head temp_partial_all; + bitmap_initialize (&temp_partial_all, &bitmap_default_obstack); + bitmap_ior (&temp_partial_all, &src_bb_info->partial_out, + &dest_bb_info->partial_in); + + bitmap_head temp_full; + bitmap_initialize (&temp_full, &bitmap_default_obstack); + + /* Collect regno that become full after merge src_bb_info->partial_out + and dest_bb_info->partial_in. */ + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (&temp_partial_all, FIRST_PSEUDO_REGISTER, regno, bi) + { + if (bitmap_bit_p (&src_bb_info->full_out, regno) + || bitmap_bit_p (&dest_bb_info->full_in, regno)) + { + bitmap_set_bit (&temp_full, regno); + temp_range.remove_live (regno); + continue; + } + else if (!bitmap_bit_p (&src_bb_info->partial_out, regno) + || !bitmap_bit_p (&dest_bb_info->partial_in, regno)) + continue; + + subreg_ranges temp = src_bb_info->range_out->lives.at (regno); + temp.add_ranges (dest_bb_info->range_in->lives.at (regno)); + if (temp.full_p ()) + { + bitmap_set_bit (&temp_full, regno); + temp_range.remove_live (regno); + } + } + + /* Calculating src_bb_info->partial_out and src_bb_info->range_out. */ + bool changed = bitmap_and_compl (&src_bb_info->partial_out, &temp_partial_all, + &temp_full); + changed |= src_bb_info->range_out->copy_lives (temp_range); + + /* Calculating src_bb_info->full_out. */ + bitmap_ior_and_compl_into (&temp_full, &dest_bb_info->full_in, + &all_hard_regs_bitmap); + changed |= bitmap_ior_into (&src_bb_info->full_out, &temp_full); + + return changed; } /* Indexes of all function blocks. */ @@ -483,8 +616,12 @@ initiate_live_solver (void) { bb_data_t bb_info = get_bb_data (bb); bb_info->bb = bb; - bitmap_initialize (&bb_info->killed_pseudos, ®_obstack); - bitmap_initialize (&bb_info->gen_pseudos, ®_obstack); + bitmap_initialize (&bb_info->full_def, ®_obstack); + bitmap_initialize (&bb_info->partial_def, ®_obstack); + bitmap_initialize (&bb_info->full_use, ®_obstack); + bitmap_initialize (&bb_info->partial_use, ®_obstack); + bb_info->range_def = new subregs_live (); + bb_info->range_use = new subregs_live (); bitmap_set_bit (&all_blocks, bb->index); } } @@ -499,8 +636,12 @@ finish_live_solver (void) FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap_clear (&bb_info->killed_pseudos); - bitmap_clear (&bb_info->gen_pseudos); + bitmap_clear (&bb_info->full_def); + bitmap_clear (&bb_info->partial_def); + bitmap_clear (&bb_info->full_use); + bitmap_clear (&bb_info->partial_use); + delete bb_info->range_def; + delete bb_info->range_use; } free (bb_data); bitmap_clear (&all_hard_regs_bitmap); @@ -663,7 +804,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) /* Only has a meaningful value once we've seen a call. */ function_abi last_call_abi = default_function_abi; - reg_live_out = df_get_live_out (bb); + reg_live_out = DF_LIVE_SUBREG_OUT (bb); + bitmap reg_live_partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (bb); sparseset_clear (pseudos_live); sparseset_clear (pseudos_live_through_calls); sparseset_clear (pseudos_live_through_setjumps); @@ -675,10 +818,13 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_live (j); } - bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos; - bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos; - bitmap_clear (bb_gen_pseudos); - bitmap_clear (bb_killed_pseudos); + curr_bb_info = get_bb_data (bb); + bitmap_clear (&curr_bb_info->full_use); + bitmap_clear (&curr_bb_info->partial_use); + bitmap_clear (&curr_bb_info->full_def); + bitmap_clear (&curr_bb_info->partial_def); + curr_bb_info->range_use->clear (); + curr_bb_info->range_def->clear (); freq = REG_FREQ_FROM_BB (bb); if (lra_dump_file != NULL) @@ -1101,16 +1247,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) bool live_change_p = false; /* Check if bb border live info was changed. */ unsigned int live_pseudos_num = 0; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), - FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, j, + bi) { live_pseudos_num++; - if (! sparseset_bit_p (pseudos_live, j)) + if (!sparseset_bit_p (pseudos_live, j)) { live_change_p = true; if (lra_dump_file != NULL) - fprintf (lra_dump_file, - " r%d is removed as live at bb%d start\n", j, bb->index); + fprintf (lra_dump_file, " r%d is removed as live at bb%d start\n", + j, bb->index); break; } } @@ -1120,9 +1266,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) live_change_p = true; if (lra_dump_file != NULL) EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j) - if (! bitmap_bit_p (df_get_live_in (bb), j)) - fprintf (lra_dump_file, - " r%d is added to live at bb%d start\n", j, bb->index); + if (!bitmap_bit_p (DF_LIVE_SUBREG_IN (bb), j)) + fprintf (lra_dump_file, " r%d is added to live at bb%d start\n", j, + bb->index); } /* See if we'll need an increment at the end of this basic block. An increment is needed if the PSEUDOS_LIVE set is not empty, @@ -1135,8 +1281,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_dead (i); } - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi) - { + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, j, + bi) + { if (sparseset_cardinality (pseudos_live_through_calls) == 0) break; if (sparseset_bit_p (pseudos_live_through_calls, j)) @@ -1151,7 +1298,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i)) continue; - if (bitmap_bit_p (df_get_live_in (bb), i)) + if (bitmap_bit_p (DF_LIVE_SUBREG_IN (bb), i)) continue; live_change_p = true; @@ -1159,7 +1306,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) fprintf (lra_dump_file, " hard reg r%d is added to live at bb%d start\n", i, bb->index); - bitmap_set_bit (df_get_live_in (bb), i); + bitmap_set_bit (DF_LIVE_SUBREG_IN (bb), i); + bitmap_set_bit (DF_LIVE_SUBREG_FULL_IN (bb), i); } if (need_curr_point_incr) @@ -1425,10 +1573,24 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) disappear, e.g. pseudos with used equivalences. */ FOR_EACH_BB_FN (bb, cfun) { - bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (DF_LIVE_SUBREG_IN (bb), FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_FULL_IN (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_PARTIAL_IN (bb), + FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); - bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (DF_LIVE_SUBREG_OUT (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_FULL_OUT (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + DF_LIVE_SUBREG_RANGE_IN (bb)->clear (); + DF_LIVE_SUBREG_RANGE_OUT (bb)->clear (); } /* As we did not change CFG since LRA start we can use DF-infrastructure solver to solve live data flow problem. */ @@ -1441,6 +1603,8 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) (DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n, live_trans_fun, &all_blocks, df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD)); + df_live_subreg_finalize (&all_blocks); + if (lra_dump_file != NULL) { fprintf (lra_dump_file, @@ -1449,16 +1613,28 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) FOR_EACH_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap bb_livein = df_get_live_in (bb); - bitmap bb_liveout = df_get_live_out (bb); fprintf (lra_dump_file, "\nBB %d:\n", bb->index); - lra_dump_bitmap_with_title (" gen:", - &bb_info->gen_pseudos, bb->index); - lra_dump_bitmap_with_title (" killed:", - &bb_info->killed_pseudos, bb->index); - lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index); - lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index); + lra_dump_bitmap_with_title (" full use", &bb_info->full_use, + bb->index); + lra_dump_bitmap_with_title (" partial use", + &bb_info->partial_use, bb->index); + lra_dump_bitmap_with_title (" full def", &bb_info->full_def, + bb->index); + lra_dump_bitmap_with_title (" partial def", + &bb_info->partial_def, bb->index); + lra_dump_bitmap_with_title (" live in full", + DF_LIVE_SUBREG_FULL_IN (bb), + bb->index); + lra_dump_bitmap_with_title (" live in partial", + DF_LIVE_SUBREG_PARTIAL_IN (bb), + bb->index); + lra_dump_bitmap_with_title (" live out full", + DF_LIVE_SUBREG_FULL_OUT (bb), + bb->index); + lra_dump_bitmap_with_title (" live out partial", + DF_LIVE_SUBREG_PARTIAL_OUT (bb), + bb->index); } } } diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc index 681dcf36331..26d3da07b00 100644 --- a/gcc/lra-remat.cc +++ b/gcc/lra-remat.cc @@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void) fprintf (lra_dump_file, "\nBB %d:\n", bb->index); /* Livein */ fprintf (lra_dump_file, " register live in:"); - dump_regset (df_get_live_in (bb), lra_dump_file); + dump_regset (DF_LIVE_SUBREG_IN (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Liveout */ fprintf (lra_dump_file, " register live out:"); - dump_regset (df_get_live_out (bb), lra_dump_file); + dump_regset (DF_LIVE_SUBREG_OUT (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Changed/dead regs: */ fprintf (lra_dump_file, " changed regs:"); @@ -727,7 +727,7 @@ calculate_livein_cands (void) FOR_EACH_BB_FN (bb, cfun) { - bitmap livein_regs = df_get_live_in (bb); + bitmap livein_regs = DF_LIVE_SUBREG_IN (bb); bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands; for (unsigned int i = 0; i < cands_num; i++) { @@ -1064,11 +1064,10 @@ do_remat (void) FOR_EACH_BB_FN (bb, cfun) { CLEAR_HARD_REG_SET (live_hard_regs); - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_IN (bb), 0, regno, bi) { - int hard_regno = regno < FIRST_PSEUDO_REGISTER - ? regno - : reg_renumber[regno]; + int hard_regno + = regno < FIRST_PSEUDO_REGISTER ? regno : reg_renumber[regno]; if (hard_regno >= 0) SET_HARD_REG_BIT (live_hard_regs, hard_regno); } diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc index a663a1931e3..d38a2ffe2a7 100644 --- a/gcc/lra-spills.cc +++ b/gcc/lra-spills.cc @@ -566,8 +566,26 @@ spill_pseudos (void) "Debug insn #%u is reset because it referenced " "removed pseudo\n", INSN_UID (insn)); } - bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos); - bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos); + unsigned int regno; + bitmap_iterator bi; + + bitmap_and_compl_into (DF_LIVE_SUBREG_IN (bb), spilled_pseudos); + bitmap_and_compl_into (DF_LIVE_SUBREG_FULL_IN (bb), spilled_pseudos); + bitmap partial_in = DF_LIVE_SUBREG_PARTIAL_IN (bb); + subregs_live *range_in = DF_LIVE_SUBREG_RANGE_IN (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_in->remove_live (regno); + bitmap_and_compl_into (partial_in, spilled_pseudos); + + bitmap_and_compl_into (DF_LIVE_SUBREG_OUT (bb), spilled_pseudos); + bitmap_and_compl_into (DF_LIVE_SUBREG_FULL_OUT (bb), spilled_pseudos); + bitmap partial_out = DF_LIVE_SUBREG_PARTIAL_OUT (bb); + subregs_live *range_out = DF_LIVE_SUBREG_RANGE_OUT (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_out->remove_live (regno); + bitmap_and_compl_into (partial_out, spilled_pseudos); } } } -- 2.36.3