[Bug tree-optimization/106322] [12/13 Regression] tree-vectorize: Wrong code at O2 level (-fno-tree-vectorize is working) since r12-2404-ga1d27560770818c5

["linkw at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106322

--- Comment #37 from Kewen Lin <linkw at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #36)
> You might need to do -O2 -fPIE -pie to reproduce the issue as debian is
> configured with --enable-default-pie

Thanks for the hint! I can reproduce this but it needs one more explicit cpu
type like -mcpu=power4/5/6. The problem comes from slp1, so
-fno-tree-slp-vectorize can make it pass.

It seems to expose one latent issue, for the code in vect_recog_mulhs_pattern:

  vect_pattern_detected ("vect_recog_mulhs_pattern", last_stmt);

  /* Check for target support.  */
  tree new_vectype = get_vectype_for_scalar_type (vinfo, new_type);
  if (!new_vectype
      || !direct_internal_fn_supported_p
            (ifn, new_vectype, OPTIMIZE_FOR_SPEED))
    return NULL;

At this time, the new_vectype is 

(gdb) pge new_vectype
vector(2) short unsigned int

the current target doesn't support umul_highpart optab for V2HImode at all, but
the check doesn't fail since in the function direct_optab_supported_p

static bool
direct_optab_supported_p (direct_optab optab, tree_pair types,
                          optimization_type opt_type)
{
  machine_mode mode = TYPE_MODE (types.first);
  gcc_checking_assert (mode == TYPE_MODE (types.second));
  return direct_optab_handler (optab, mode, opt_type) != CODE_FOR_nothing;
}

(gdb) pge types.first
vector(2) short unsigned int
(gdb) p mode
$12 = E_SImode

the current target does support umul_highpart optab for SImode, so it doesn't
fail. But we expected to query with vector mode for the given type, it's wrong
in functionality to use scalar insn for vector operation here, so this result
is unexpected.