[PATCH] Maintain LC SSA when doing SVE vectorization

[PATCH] Maintain LC SSA when doing SVE vectorization

[Richard Biener]

The final loop IV use after the loop has that not in LC SSA
(and inserts not simplified _2 = _3 - 0 stmts).  In particular
since it splits the exit edge when there's a virtual PHI in the
destination it breaks virtual LC SSA form (but likely also
non-virtual).

The following properly inserts LC PHIs instead.

Bootstrap & regtest pending on x86_64-unknown-linux-gnu.

OK if that succeeds?

Thanks,
Richard.

2022-07-04  Richard Biener  <rguenther@suse.de>

	* tree-vect-loop-manip.cc (vect_set_loop_condition_normal):
	Maintain LC SSA.
---
 gcc/tree-vect-loop-manip.cc | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 895957472c9..eba71c50428 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -920,9 +920,22 @@ vect_set_loop_condition_normal (class loop *loop, tree niters, tree step,
 
   if (final_iv)
     {
-      gassign *assign = gimple_build_assign (final_iv, MINUS_EXPR,
-					     indx_after_incr, init);
-      gsi_insert_on_edge_immediate (single_exit (loop), assign);
+      gassign *assign;
+      edge exit = single_exit (loop);
+      gcc_assert (single_pred_p (exit->dest));
+      tree phi_dest
+	= integer_zerop (init) ? final_iv : copy_ssa_name (indx_after_incr);
+      /* Make sure to maintain LC SSA form here and elide the subtraction
+	 if the value is zero.  */
+      gphi *phi = create_phi_node (phi_dest, exit->dest);
+      add_phi_arg (phi, indx_after_incr, exit, UNKNOWN_LOCATION);
+      if (!integer_zerop (init))
+	{
+	  assign = gimple_build_assign (final_iv, MINUS_EXPR,
+					phi_dest, init);
+	  gimple_stmt_iterator gsi = gsi_after_labels (exit->dest);
+	  gsi_insert_before (&gsi, assign, GSI_SAME_STMT);
+	}
     }
 
   return cond_stmt;
-- 
2.35.3

Re: [PATCH] Maintain LC SSA when doing SVE vectorization

[Richard Sandiford]

Richard Biener <rguenther@suse.de> writes:
> The final loop IV use after the loop has that not in LC SSA
> (and inserts not simplified _2 = _3 - 0 stmts).  In particular
> since it splits the exit edge when there's a virtual PHI in the
> destination it breaks virtual LC SSA form (but likely also
> non-virtual).
>
> The following properly inserts LC PHIs instead.
>
> Bootstrap & regtest pending on x86_64-unknown-linux-gnu.
>
> OK if that succeeds?

LGTM, thanks.

Richard

>
> Thanks,
> Richard.
>
> 2022-07-04  Richard Biener  <rguenther@suse.de>
>
> 	* tree-vect-loop-manip.cc (vect_set_loop_condition_normal):
> 	Maintain LC SSA.
> ---
>  gcc/tree-vect-loop-manip.cc | 19 ++++++++++++++++---
>  1 file changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index 895957472c9..eba71c50428 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -920,9 +920,22 @@ vect_set_loop_condition_normal (class loop *loop, tree niters, tree step,
>  
>    if (final_iv)
>      {
> -      gassign *assign = gimple_build_assign (final_iv, MINUS_EXPR,
> -					     indx_after_incr, init);
> -      gsi_insert_on_edge_immediate (single_exit (loop), assign);
> +      gassign *assign;
> +      edge exit = single_exit (loop);
> +      gcc_assert (single_pred_p (exit->dest));
> +      tree phi_dest
> +	= integer_zerop (init) ? final_iv : copy_ssa_name (indx_after_incr);
> +      /* Make sure to maintain LC SSA form here and elide the subtraction
> +	 if the value is zero.  */
> +      gphi *phi = create_phi_node (phi_dest, exit->dest);
> +      add_phi_arg (phi, indx_after_incr, exit, UNKNOWN_LOCATION);
> +      if (!integer_zerop (init))
> +	{
> +	  assign = gimple_build_assign (final_iv, MINUS_EXPR,
> +					phi_dest, init);
> +	  gimple_stmt_iterator gsi = gsi_after_labels (exit->dest);
> +	  gsi_insert_before (&gsi, assign, GSI_SAME_STMT);
> +	}
>      }
>  
>    return cond_stmt;