[Bug tree-optimization/105591] [13 Regression] ICE: in tree_to_poly_uint64, at tree.cc:3250 with -O -mavx512f -mno-avx2 since r13-379

["jakub at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105591

--- Comment #11 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Because VEC_PERM_EXPR doesn't require the mask argument to be constant (and
neither does __builtin_shuffle, unlike e.g. __builtin_shufflevector).
If the mask argument remains non-constant until end of compilation, the modulo
N or 2*N needs to be done at runtime (of course unless used hw instruction
performs something like that itself).
And that is the reason why it is defined this way.  During compilation there
are way too many spots where a constant could be propagated into a formerly
non-constant operand of the VEC_PERM_EXPR, and no guarantee that all such
propagations (it isn't in a single spot in a single pass, it is really many)
will do some extra code to canonicalize it.  It can be, but we can't guarantee
it.
For __builtin_shufflevector, we supposedly want to introduce some VEC_PERM_EXPR
variant which would only allow constant mask argument and which would have
different behavior, -1 standing for I don't care rather than -1 % N or -1 %
(2*N).  If we introduce something like that, we could certainly require that
the new expr's operand is only -1..N-1 or -1..2*N-1 and could have then say
match.pd that turns a VEC_PERM_EXPR with a constant argument into the new expr
with canonical argument.