[Bug tree-optimization/96974] [10/11 Regression] ICE in vect_get_vector_types_for_stmt compiling for SVE

["stammark at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96974

--- Comment #8 from Stam Markianos-Wright <stammark at gcc dot gnu.org> ---
I have a liiitle bit more progress here, but I have a question about
vect_get_smallest_scalar_type.

If we look at the comment before the function:

>/* Return the smallest scalar part of STMT_INFO.
>   This is used to determine the vectype of the stmt.  We generally set the
>   vectype according to the type of the result (lhs).  For stmts whose
>   result-type is different than the type of the arguments (e.g., demotion,
>   promotion), vectype will be reset appropriately (later).  Note that we have
>   to visit the smallest datatype in this function, because that determines the
>   VF.  If the smallest datatype in the loop is present only as the rhs of a
>   promotion operation - we'd miss it.

Would this be "smallest datatype in all cases", or is this more like "the
smallest
datatype within the same promotion/demotion chain"?

i.e. how should we react if we detect a smallest datatype on the rhs of "float"
when everything else in the stmt has been in the integer chain (int or,
like in this case, long int)?

>   Such a case, where a variable of this datatype does not appear in the lhs
>   anywhere in the loop, can only occur if it's an invariant: e.g.:
>   'int_x = (int) short_inv', which we'd expect to have been optimized away by
>   invariant motion.  However, we cannot rely on invariant motion to always
>   take invariants out of the loop, and so in the case of promotion we also
>   have to check the rhs.
>   LHS_SIZE_UNIT and RHS_SIZE_UNIT contain the sizes of the corresponding
>   types.  */

I have found that this is why we end up with a smaller number in:
TYPE_VECTOR_SUBPARTS (nunits_vectype) == 4
than in:
TYPE_VECTOR_SUBPARTS (*stmt_vectype_out) == 8

So I'm thinking that either A) We shouldn't allow this, and add in some check
maybe for "GET_MODE_CLASS (x) == GET_MODE_CLASS (y)"
or B) Some of the logic that generates stmt_vectype_out is deficient and it
should also be detecting the existence of a "float" type to get the VF.