[Bug tree-optimization/61634] [4.8/4.9/5 Regression] ICE in in vect_get_vec_def_for_operand, at tree-vect-stmts.c:1423

["rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61634

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
The issue seems that we fail to detect hybrid SLP here because
vect_detect_hybrid_slp_stmts walks immediate uses of SLP defs but those
do not reach pattern stmts but only original ones.  In this case we have

  _11 = *prephitmp_46;
  _12 = _11 >> 2;
  f_13 = (int) _12;
  *prephitmp_46 = _12;
  _17 = f_13 * f_13;

where the SLP def _12 is used in f_13 = (int) _12; but that stmt is marked
as vect_unused_in_scope because it feeds a stmt that was pattern detected
as widening multiplication.  But we never visit that stmt, as its
immediate uses are not accounted for (pattern stmts never get updated).

That makes the vect_detect_hybrid_slp_stmts functions non-functional
for non-trivial patterns.  We can "fix" this by marking the stmts
in some way and make sure to account for them in the loop.  For the
testcase forcing hybrid SLP by making !STMT_VINFO_RELEVANT stmts
force it fixes it.

I think that the best thing to do is to re-write hybrid SLP detection
in terms of walking all stmts in the loop, similar to
vect_mark_stmts_to_be_vectorized and mark pure SLP stmts as hybrid.
The flag then still has to be propagated to the SLP tree children of
such node.

I have a prototype but that still needs quite some TLC.