[PATCH] Punt on vectorizing sign-changing conditional reduction (PR tree-optimization/91665)

[PATCH] Punt on vectorizing sign-changing conditional reduction (PR tree-optimization/91665)

[Jakub Jelinek]

Hi!

We ICE on the following testcase, because we don't handle conditional
expression where base has incompatible type (the iv bumped in unsigned type,
but is signed otherwise, as happens for C/C++ signed char/short).
This patch just punts on it, during x86_64-linux and i686-linux
bootstrap/regtest it made a difference only on the testcase added by the
patch.

Preapproved by Richi on IRC, committed to trunk so far, queued for backporting.

2019-09-07  Jakub Jelinek  <jakub@redhat.com>

	PR tree-optimization/91665
	* tree-vect-loop.c (vectorizable_reduction): Punt if base has type
	incompatible with the type of PHI result.

	* gcc.dg/vect/pr91665.c: New test.

--- gcc/tree-vect-loop.c.jj	2019-09-05 15:29:20.570627230 +0200
+++ gcc/tree-vect-loop.c	2019-09-06 19:39:18.777405377 +0200
@@ -6656,10 +6656,13 @@ vectorizable_reduction (stmt_vec_info st
 	  gcc_assert (TREE_CODE (base) == INTEGER_CST
 		      && TREE_CODE (step) == INTEGER_CST);
 	  cond_reduc_val = NULL_TREE;
+	  tree res = PHI_RESULT (STMT_VINFO_STMT (cond_stmt_vinfo));
+	  if (!types_compatible_p (TREE_TYPE (res), TREE_TYPE (base)))
+	    ;
 	  /* Find a suitable value, for MAX_EXPR below base, for MIN_EXPR
 	     above base; punt if base is the minimum value of the type for
 	     MAX_EXPR or maximum value of the type for MIN_EXPR for now.  */
-	  if (tree_int_cst_sgn (step) == -1)
+	  else if (tree_int_cst_sgn (step) == -1)
 	    {
 	      cond_reduc_op_code = MIN_EXPR;
 	      if (tree_int_cst_sgn (base) == -1)
--- gcc/testsuite/gcc.dg/vect/pr91665.c.jj	2019-09-05 14:20:01.986459651 +0200
+++ gcc/testsuite/gcc.dg/vect/pr91665.c	2019-09-05 13:54:52.944132072 +0200
@@ -0,0 +1,15 @@
+/* PR tree-optimization/91665 */
+/* { dg-do compile } */
+/* { dg-additional-options "-Ofast" } */
+
+short int v;
+
+void
+foo (short int x, short int y)
+{
+  short int *p = &v;
+
+  x = 1;
+  while (x != 0)
+    x += ++y || (*p = x);
+}

	Jakub