[Bug tree-optimization/94727] [10 Regression] GCC produces incorrect code with -O3 since r10-5071-g02d895504cc59be0

["cvs-commit at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94727

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Sandiford <rsandifo@gcc.gnu.org>:

https://gcc.gnu.org/g:901f5289d9465d4c388ae288f850ad4f29e99a2c

commit r10-7915-g901f5289d9465d4c388ae288f850ad4f29e99a2c
Author: Richard Sandiford <richard.sandiford@arm.com>
Date:   Thu Apr 23 15:45:43 2020 +0100

    vect: Fix comparisons between invariant booleans [PR94727]

    This PR was caused by mismatched expectations between
    vectorizable_comparison and SLP.  We had a "<" comparison
    between two booleans that were leaves of the SLP tree, so
    vectorizable_comparison fell back on:

      /* Invariant comparison.  */
      if (!vectype)
        {
          vectype = get_vectype_for_scalar_type (vinfo, TREE_TYPE (rhs1),
                                                 slp_node);
          if (maybe_ne (TYPE_VECTOR_SUBPARTS (vectype), nunits))
            return false;
        }

    rhs1 and rhs2 were *unsigned* boolean types, so we got back a vector
    of unsigned integers.  This in itself was OK, and meant that "<"
    worked as expected without the need for the boolean fix-ups:

      /* Boolean values may have another representation in vectors
         and therefore we prefer bit operations over comparison for
         them (which also works for scalar masks).  We store opcodes
         to use in bitop1 and bitop2.  Statement is vectorized as
           BITOP2 (rhs1 BITOP1 rhs2) or
           rhs1 BITOP2 (BITOP1 rhs2)
         depending on bitop1 and bitop2 arity.  */
      bool swap_p = false;
      if (VECTOR_BOOLEAN_TYPE_P (vectype))
        {

    However, vectorizable_comparison then used vect_get_slp_defs to get
    the actual operands.  The request went to vect_get_constant_vectors,
    which also has logic to calculate the vector type.  The problem was
    that this type was different from the one chosen above:

      if (VECT_SCALAR_BOOLEAN_TYPE_P (TREE_TYPE (op))
          && vect_mask_constant_operand_p (stmt_vinfo))
        vector_type = truth_type_for (stmt_vectype);
      else
        vector_type = get_vectype_for_scalar_type (vinfo, TREE_TYPE (op),
op_node);

    So the function gave back a vector of mask types, which here are vectors
    of *signed* booleans.  This meant that "<" gave:

      true (-1) < false (0)

    and so the boolean fixup above was needed after all.

    Fixed by making vectorizable_comparison also pick a mask type in
    this case.

    2020-04-23  Richard Sandiford  <richard.sandiford@arm.com>

    gcc/
            PR tree-optimization/94727
            * tree-vect-stmts.c (vectorizable_comparison): Use mask_type when
            comparing invariant scalar booleans.

    gcc/testsuite/
            PR tree-optimization/94727
            * gcc.dg/vect/pr94727.c: New test.