[PATCH] Speedup DF dataflow solver

[PATCH] Speedup DF dataflow solver

[Richard Biener]

The following makes sure to process blocks that follow the current
block in the iteration order in the same iteration and only postpone
blocks that would be visited earlier to the next iteration.

For the all.i testcase in PR26854 at -O2 this shaves off 50% of
the time to solve the DF RD problem, other problems also improve
but not as drastically.

Bootstrapped on x86_64-unknown-linux-gnu, testing in progress.

OK for trunk?

Thanks,
Richard.

	PR middle-end/26854
	* df-core.cc (df_worklist_propagate_forward): Put later
	blocks on worklist and only earlier blocks on pending.
	(df_worklist_propagate_backward): Likewise.
	(df_worklist_dataflow_doublequeue): Change the iteration
	to process new blocks in the same iteration if that
	maintains the iteration order.
---
 gcc/df-core.cc | 54 ++++++++++++++++++++++++++++++++------------------
 1 file changed, 35 insertions(+), 19 deletions(-)

diff --git a/gcc/df-core.cc b/gcc/df-core.cc
index e5ae9ab9348..38f69ac5743 100644
--- a/gcc/df-core.cc
+++ b/gcc/df-core.cc
@@ -874,7 +874,8 @@ make_pass_df_finish (gcc::context *ctxt)
 /* Helper function for df_worklist_dataflow.
    Propagate the dataflow forward.
    Given a BB_INDEX, do the dataflow propagation
-   and set bits on for successors in PENDING
+   and set bits on for successors in PENDING for earlier
+   and WORKLIST for later in bbindex_to_postorder
    if the out set of the dataflow has changed.
 
    AGE specify time when BB was visited last time.
@@ -890,10 +891,11 @@ make_pass_df_finish (gcc::context *ctxt)
 
 static bool
 df_worklist_propagate_forward (struct dataflow *dataflow,
-                               unsigned bb_index,
-                               unsigned *bbindex_to_postorder,
-                               bitmap pending,
-                               sbitmap considered,
+			       unsigned bb_index,
+			       unsigned *bbindex_to_postorder,
+			       bitmap worklist,
+			       bitmap pending,
+			       sbitmap considered,
 			       vec<int> &last_change_age,
 			       int age)
 {
@@ -924,7 +926,13 @@ df_worklist_propagate_forward (struct dataflow *dataflow,
           unsigned ob_index = e->dest->index;
 
           if (bitmap_bit_p (considered, ob_index))
-            bitmap_set_bit (pending, bbindex_to_postorder[ob_index]);
+	    {
+	      if (bbindex_to_postorder[bb_index]
+		  < bbindex_to_postorder[ob_index])
+		bitmap_set_bit (worklist, bbindex_to_postorder[ob_index]);
+	      else
+		bitmap_set_bit (pending, bbindex_to_postorder[ob_index]);
+	    }
         }
       return true;
     }
@@ -937,10 +945,11 @@ df_worklist_propagate_forward (struct dataflow *dataflow,
 
 static bool
 df_worklist_propagate_backward (struct dataflow *dataflow,
-                                unsigned bb_index,
-                                unsigned *bbindex_to_postorder,
-                                bitmap pending,
-                                sbitmap considered,
+				unsigned bb_index,
+				unsigned *bbindex_to_postorder,
+				bitmap worklist,
+				bitmap pending,
+				sbitmap considered,
 				vec<int> &last_change_age,
 				int age)
 {
@@ -971,7 +980,13 @@ df_worklist_propagate_backward (struct dataflow *dataflow,
           unsigned ob_index = e->src->index;
 
           if (bitmap_bit_p (considered, ob_index))
-            bitmap_set_bit (pending, bbindex_to_postorder[ob_index]);
+	    {
+	      if (bbindex_to_postorder[bb_index]
+		  < bbindex_to_postorder[ob_index])
+		bitmap_set_bit (worklist, bbindex_to_postorder[ob_index]);
+	      else
+		bitmap_set_bit (pending, bbindex_to_postorder[ob_index]);
+	    }
         }
       return true;
     }
@@ -1021,36 +1036,37 @@ df_worklist_dataflow_doublequeue (struct dataflow *dataflow,
      and pending is for the next. */
   while (!bitmap_empty_p (pending))
     {
-      bitmap_iterator bi;
-      unsigned int index;
-
       std::swap (pending, worklist);
 
-      EXECUTE_IF_SET_IN_BITMAP (worklist, 0, index, bi)
+      do
 	{
+	  unsigned index = bitmap_first_set_bit (worklist);
+	  bitmap_clear_bit (worklist, index);
+
 	  unsigned bb_index;
 	  dcount++;
 
-	  bitmap_clear_bit (pending, index);
 	  bb_index = blocks_in_postorder[index];
 	  prev_age = last_visit_age[index];
 	  if (dir == DF_FORWARD)
 	    changed = df_worklist_propagate_forward (dataflow, bb_index,
 						     bbindex_to_postorder,
-						     pending, considered,
+						     worklist, pending,
+						     considered,
 						     last_change_age,
 						     prev_age);
 	  else
 	    changed = df_worklist_propagate_backward (dataflow, bb_index,
 						      bbindex_to_postorder,
-						      pending, considered,
+						      worklist, pending,
+						      considered,
 						      last_change_age,
 						      prev_age);
 	  last_visit_age[index] = ++age;
 	  if (changed)
 	    last_change_age[index] = age;
 	}
-      bitmap_clear (worklist);
+      while (!bitmap_empty_p (worklist));
     }
 
   BITMAP_FREE (worklist);
-- 
2.35.3

Re: [PATCH] Speedup DF dataflow solver

[Jakub Jelinek]

On Tue, Feb 14, 2023 at 03:21:53PM +0100, Richard Biener wrote:
> The following makes sure to process blocks that follow the current
> block in the iteration order in the same iteration and only postpone
> blocks that would be visited earlier to the next iteration.
> 
> For the all.i testcase in PR26854 at -O2 this shaves off 50% of
> the time to solve the DF RD problem, other problems also improve
> but not as drastically.
> 
> Bootstrapped on x86_64-unknown-linux-gnu, testing in progress.
> 
> OK for trunk?
> 
> Thanks,
> Richard.
> 
> 	PR middle-end/26854
> 	* df-core.cc (df_worklist_propagate_forward): Put later
> 	blocks on worklist and only earlier blocks on pending.
> 	(df_worklist_propagate_backward): Likewise.
> 	(df_worklist_dataflow_doublequeue): Change the iteration
> 	to process new blocks in the same iteration if that
> 	maintains the iteration order.

LGTM.

	Jakub