[Bug tree-optimization/109176] [13 Regression] internal compiler error: in to_constant, at poly-int.h:504

["rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109176

--- Comment #7 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #5)
> Before veclower2 we had/have
>   _7 = ba_5(D) < a_6(D);
>   _8 = svnand_b_z (_7, _7, _7);
>   _9 = VEC_COND_EXPR <_7, _8, _7>;
> where _7/_8/_9 are all __SVBool_t.
> Before the r11-1445 changes, the VEC_COND_EXPR would be left untouched,
> because
> expand_vec_cond_expr_p (__SVBool_t, __SVBool_t, SSA_NAME) is true on aarch64.
> Now it doesn't, because _7 def_stmt is a comparison, and so
> expand_vec_cond_expr_p (__SVBool_t, VNx16QI, LT_EXPR).
> --- gcc/tree-vect-generic.cc.jj	2023-03-12 22:36:06.356178068 +0100
> +++ gcc/tree-vect-generic.cc	2023-03-20 16:31:03.321831866 +0100
> @@ -1063,6 +1063,12 @@ expand_vector_condition (gimple_stmt_ite
>        return true;
>      }
>  
> +  if (!TYPE_VECTOR_SUBPARTS (type).is_constant ()
> +      && VECTOR_BOOLEAN_TYPE_P (type)
> +      && VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (a))
> +      && expand_vec_cond_expr_p (type, TREE_TYPE (a), TREE_CODE (a)))
> +    return true;
> +
>    /* Handle vector boolean types with bitmasks.  If there is a comparison
>       and we can expand the comparison into the vector boolean bitmask,
>       or otherwise if it is compatible with type, we can transform
> basically restores the behaviour here for the SVE boolean types and makes it
> compile.
> Because otherwise, unless this would be solvable by the AVX512 "Handle
> vector boolean types with bitmasks." code the non-constant vector elts cases
> will always ICE in the following code,   int nunits =
> nunits_for_known_piecewise_op (type);
> requires constant TYPE_VECTOR_SUBPARTS and the code really isn't prepared to
> handle anything non-constant.

OK, so expand_vec_cond_expr_p does

bool
expand_vec_cond_expr_p (tree value_type, tree cmp_op_type, enum tree_code code)
{
  machine_mode value_mode = TYPE_MODE (value_type);
  machine_mode cmp_op_mode = TYPE_MODE (cmp_op_type);
  if (VECTOR_BOOLEAN_TYPE_P (cmp_op_type)
      && get_vcond_mask_icode (TYPE_MODE (value_type),
                               TYPE_MODE (cmp_op_type)) != CODE_FOR_nothing)
    return true;

first.  I think vector lowering should always check the original 'a'
if it is a mask, like with the following, there's no good reason to
special-case this just for VL vectors

diff --git a/gcc/tree-vect-generic.cc b/gcc/tree-vect-generic.cc
index 519a824ec72..c957c9aa7a9 100644
--- a/gcc/tree-vect-generic.cc
+++ b/gcc/tree-vect-generic.cc
@@ -1040,6 +1040,10 @@ expand_vector_condition (gimple_stmt_iterator *gsi,
bitmap dce_ssa_names)
   tree_code code = TREE_CODE (a);
   gassign *assign = NULL;

+  if (VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (a))
+      && expand_vec_cond_expr_p (type, TREE_TYPE (a), ERROR_MARK))
+    return true;
+
   if (code == SSA_NAME)
     {
       assign = dyn_cast<gassign *> (SSA_NAME_DEF_STMT (a));
@@ -1053,14 +1057,14 @@ expand_vector_condition (gimple_stmt_iterator *gsi,
bitmap dce_ssa_names)
          comp_inner_type = TREE_TYPE (TREE_TYPE (a1));
          comp_width = vector_element_bits_tree (TREE_TYPE (a1));
        }
-    }

-  if (expand_vec_cond_expr_p (type, TREE_TYPE (a1), code)
-      || (integer_all_onesp (b) && integer_zerop (c)
-         && expand_vec_cmp_expr_p (type, TREE_TYPE (a1), code)))
-    {
-      gcc_assert (TREE_CODE (a) == SSA_NAME || TREE_CODE (a) == VECTOR_CST);
-      return true;
+      if (expand_vec_cond_expr_p (type, TREE_TYPE (a1), code)
+         || (integer_all_onesp (b) && integer_zerop (c)
+             && expand_vec_cmp_expr_p (type, TREE_TYPE (a1), code)))
+       {
+         gcc_assert (TREE_CODE (a) == SSA_NAME || TREE_CODE (a) ==
VECTOR_CST);
+         return true;
+       }
     }

   /* Handle vector boolean types with bitmasks.  If there is a comparison