[gcc r11-8211] vectorizer: Remove dead scalar .COND_* calls from vectorized loops [PR99767]

[gcc r11-8211] vectorizer: Remove dead scalar .COND_* calls from vectorized loops [PR99767]

[Jakub Jelinek]

https://gcc.gnu.org/g:b44ab138b7d4aaa5a9ee7956121ffc94777f6a42

commit r11-8211-gb44ab138b7d4aaa5a9ee7956121ffc94777f6a42
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Fri Apr 16 11:44:04 2021 +0200

    vectorizer: Remove dead scalar .COND_* calls from vectorized loops [PR99767]
    
    The following testcase ICEs because disabling of DCE means there are dead
    stmts in the loop (though, in theory they could become dead only shortly
    before if-conv through some optimization), ifcvt which goes through all
    stmts in the loop if-converts them into .COND_DIV etc. internal fn calls
    in the copy of the loop meant for vectorization only, the loop is
    successfully vectorized but the particular .COND_* call is not because
    it isn't a live statement and the scalar .COND_* remains in the IL until
    expansion where it ICEs because these ifns only support vectors and not
    scalars.
    
    These ifns are similar to .MASK_{LOAD,STORE} in this behavior.
    
    One possible fix could be to expand scalar versions of them during
    expansion, basically undoing what if-conv did to create them, i.e.
    expand them as the lhs = else; if (mask) { lhs = statement; } or so.
    
    For .MASK_LOAD we have code to replace them in vect_transform_loop already
    though (not needed for .MASK_STORE, as stores should be always live
    and thus always vectorized), so this patch instead replaces .COND_*
    similarly to .MASK_LOAD in that loop, with the small difference
    that lhs = .MASK_LOAD (...); is replaced by lhs = 0; while
    lhs = .COND_* (..., else_arg); is replaced by lhs = else_arg.
    The statement must be dead, otherwise it would be vectorized, so I think
    it is not a big deal we don't turn it back into multiple basic blocks etc.
    (and it might be not possible to do that at that point).
    
    2021-04-16  Jakub Jelinek  <jakub@redhat.com>
    
            PR target/99767
            * tree-vect-loop.c (vect_transform_loop): Don't remove just
            dead scalar .MASK_LOAD calls, but also dead .COND_* calls - replace
            them by their last argument.
    
            * gcc.target/aarch64/pr99767.c: New test.

Diff:
---
 gcc/testsuite/gcc.target/aarch64/pr99767.c | 16 ++++++++++++++++
 gcc/tree-vect-loop.c                       | 16 +++++++++++++++-
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/gcc/testsuite/gcc.target/aarch64/pr99767.c b/gcc/testsuite/gcc.target/aarch64/pr99767.c
new file mode 100644
index 00000000000..5bbfcbdcd69
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/pr99767.c
@@ -0,0 +1,16 @@
+/* PR target/99767 */
+/* { dg-do compile } */
+/* { dg-options " -O1 -fopenmp-simd -fno-tree-dce -march=armv8-a+sve" } */
+
+int a[1024], b[1024];
+
+void
+foo (void)
+{
+  #pragma omp simd
+  for (int i = 0; i < 1024; i++)
+    if (b[i] > 23) {
+      a[i] = b[i] + 1;
+      int v = 1 / 0;	/* { dg-warning "division by zero" } */
+    }
+}
diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index 93fa2928e00..2aba503fef7 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -9676,7 +9676,10 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call)
 	   !gsi_end_p (gsi); gsi_next (&gsi))
 	{
 	  gcall *call = dyn_cast <gcall *> (gsi_stmt (gsi));
-	  if (call && gimple_call_internal_p (call, IFN_MASK_LOAD))
+	  if (!call || !gimple_call_internal_p (call))
+	    continue;
+	  internal_fn ifn = gimple_call_internal_fn (call);
+	  if (ifn == IFN_MASK_LOAD)
 	    {
 	      tree lhs = gimple_get_lhs (call);
 	      if (!VECTOR_TYPE_P (TREE_TYPE (lhs)))
@@ -9686,6 +9689,17 @@ vect_transform_loop (loop_vec_info loop_vinfo, gimple *loop_vectorized_call)
 		  gsi_replace (&gsi, new_stmt, true);
 		}
 	    }
+	  else if (conditional_internal_fn_code (ifn) != ERROR_MARK)
+	    {
+	      tree lhs = gimple_get_lhs (call);
+	      if (!VECTOR_TYPE_P (TREE_TYPE (lhs)))
+		{
+		  tree else_arg
+		    = gimple_call_arg (call, gimple_call_num_args (call) - 1);
+		  gimple *new_stmt = gimple_build_assign (lhs, else_arg);
+		  gsi_replace (&gsi, new_stmt, true);
+		}
+	    }
 	}
     }				/* BBs in loop */