[Bug tree-optimization/109176] [13 Regression] internal compiler error: in to_constant, at poly-int.h:504

["jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109176

--- Comment #14 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
So, e.g. in the gcc.target/i386/sse4_1-pr88547-1.c testcase, we have before
veclower21:
v2di f13 (v2di x, v2di y)
{
  vector(2) <signed-boolean:64> _1;
  v2di _4;

  <bb 2> [local count: 1073741824]:
  _1 = x_2(D) <= y_3(D);
  _4 = VEC_COND_EXPR <_1, { -1, -1 }, { 0, 0 }>;
  return _4;
}
And the problem is quite obvious.  We do have vcond_mask_v2div2di optab,
so with the changed code expand_vector_condition just returns true.
But, expand_vector_comparison has:
  /* As seen in PR95830, we should not expand comparisons that are only
     feeding a VEC_COND_EXPR statement.  */
  auto_vec<gimple *> uses;
  FOR_EACH_IMM_USE_FAST (use_p, iterator, lhs)
    {
      gimple *use = USE_STMT (use_p);
      if (is_gimple_debug (use))
        continue;
      if (is_gimple_assign (use)
          && gimple_assign_rhs_code (use) == VEC_COND_EXPR
          && gimple_assign_rhs1 (use) == lhs
          && gimple_assign_rhs2 (use) != lhs
          && gimple_assign_rhs3 (use) != lhs)
        uses.safe_push (use);
      else
        vec_cond_expr_only = false;
    }

  if (vec_cond_expr_only)
    for (gimple *use : uses)
      {
        gimple_stmt_iterator it = gsi_for_stmt (use);
        if (!expand_vector_condition (&it, dce_ssa_names))
          {
            vec_cond_expr_only = false;
            break;
          }
      }

  if (!uses.is_empty () && vec_cond_expr_only)
    return NULL_TREE;
So, it effectively relies on expand_vector_condition doing its previous job
when the def stmt of a is a comparison, as nothing else checks it in that case,
and here x86 doesn't have V2DImode comparison instruction for this case.

Thus, I think we should instead of the tree-vect-generic.cc changes do:
--- gcc/tree-vect-generic.cc.jj 2023-03-21 13:28:21.354671095 +0100
+++ gcc/tree-vect-generic.cc    2023-03-22 12:53:27.853986127 +0100
@@ -1063,6 +1063,15 @@ expand_vector_condition (gimple_stmt_ite
       return true;
     }

+  /* If a has vector boolean type and is a comparison, above
+     expand_vec_cond_expr_p might fail, even if both the comparison and
+     VEC_COND_EXPR could be supported individually.  See PR109176.  */
+  if (a_is_comparison
+      && VECTOR_BOOLEAN_TYPE_P (TREE_TYPE (a))
+      && expand_vec_cond_expr_p (type, TREE_TYPE (a), SSA_NAME)
+      && expand_vec_cmp_expr_p (TREE_TYPE (a1), TREE_TYPE (a), code))
+    return true;
+
   /* Handle vector boolean types with bitmasks.  If there is a comparison
      and we can expand the comparison into the vector boolean bitmask,
      or otherwise if it is compatible with type, we can transform

i.e. always look at the comparison and always try to handle it together, and if
that
fails, try to handle this special case by handling both stmts individually, and
only
if not do the piecewise etc. lowering.
At least, with the above patch, all FAILs I got in the x86_64-linux bootstrap
got fixed
(tested just those for now and nothing else).