[Bug tree-optimization/95487] New: ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512

[Bug tree-optimization/95487] New: ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512

[vsevolod.livinskij at frtk dot ru]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

            Bug ID: 95487
           Summary: ICE: verify_gimple failed (error: invalid vector types
                    in nop conversion) with -O3 -march=skylake-avx512
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: vsevolod.livinskij at frtk dot ru
  Target Milestone: ---

Error:
>$ g++ -O3 -march=skylake-avx512 -c func.cpp
func.cpp: In function ‘void h()’:
func.cpp:7:6: error: invalid vector types in nop conversion
    7 | void h() {
      |      ^
char
vector(32) unsigned char
stmp_117 = (char) stmp_116;
func.cpp:7:6: error: invalid vector types in nop conversion
char
vector(32) unsigned char
stmp_120 = (char) stmp_119;
during GIMPLE pass: vect
func.cpp:7:6: internal compiler error: verify_gimple failed
0x1141df1 verify_gimple_in_cfg(function*, bool)
        /gcc/tree-cfg.c:5461
0x10152af execute_function_todo
        /gcc/passes.c:1985
0x101608e execute_todo
        /gcc/passes.c:2039

Reproducer:
int a;
bool d;
char e;
extern short f[];
extern int g[];
short j;
void h() {
  for (short b = j; b < 0; b += 2) {
    f[b] = 0;
    if (d) {
      for (char c = 0; c < a; c += 3)
        e = 0;
      g[b] = 0;
    }
  }
}

GCC version:
11.0.0 (4b68cb38ddca37a14a6f2f43de3a6d396ee1bc79)

bug 92741 might be related

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|ICE: verify_gimple failed   |[10/11 Regression] ICE:
                   |(error: invalid vector      |verify_gimple failed
                   |types in nop conversion)    |(error: invalid vector
                   |with -O3                    |types in nop conversion)
                   |-march=skylake-avx512       |with -O3
                   |                            |-march=skylake-avx512 since
                   |                            |r10-1052-gc29c92c789d93848
                 CC|                            |fxue at gcc dot gnu.org,
                   |                            |marxin at gcc dot gnu.org
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2020-06-03
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Confirmed, started with r10-1052-gc29c92c789d93848.

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |x86_64-*-* i?86-*-*
   Target Milestone|---                         |10.2
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |rguenth at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
So the issue is in the scatter store the mask is vectorized with V32QImode
but we expect a vector boolean (it seems).  And we run into

             if (TYPE_MODE (masktype) == TYPE_MODE (optype))
                utype = masktype;
              else
                utype = lang_hooks.types.type_for_mode (TYPE_MODE (optype), 1);

with maskype QImode and optype V32QImode.  utype is V32QImode as well after
the above and then with masktype still QImode we do

              if (!useless_type_conversion_p (masktype, utype))
                {
                  gcc_assert (TYPE_PRECISION (utype)
                              <= TYPE_PRECISION (masktype));
                  var = vect_get_new_ssa_name (masktype, vect_scalar_var);
                  new_stmt = gimple_build_assign (var, NOP_EXPR, mask_arg);
                  vect_finish_stmt_generation (vinfo, stmt_info, new_stmt,
gsi);
                  mask_arg = var;

and emit the bogus NOP_EXPR trying to convert from V32QImode to QImode.  Oops?

Jakub, you added the scatter support - do you remember what the code tries to
do here?

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #2)
> Jakub, you added the scatter support - do you remember what the code tries to
> do here?

I've only added gathers myself (and only the AVX2-ish), scatters were added by
Kirill and others from Intel.  And the integer mask in vector boolean vs.
normal vectors is a big pain.

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
I will have a closer look.

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
We also fail to unswitch the outer loop on if (d) (probably simply because we
don't unswitch outer loops).  It's likely the invariantness of the condition
that makes the problem.  We're vectorizing

  <bb 32> [local count: 437450365]:
  # b_43 = PHI <b_24(33), b_15(53)>
  _46 = (int) b_43;
  _65 = &g[_46];
  .MASK_STORE (_65, 32B, d.0_2, 0);
  b.3_25 = (unsigned short) b_43;
  _33 = b.3_25 + 2;
  b_24 = (short int) _33;
  if (b_24 >= 0)
    goto <bb 30>; [11.00%]
  else
    goto <bb 33>; [89.00%]

  <bb 33> [local count: 389330825]:
  goto <bb 32>; [100.00%]

where we use a scatter because we fail to analyze the evolution for g[b].
We don't get a vector boolean type for the mask because the "magic" here
doesn't work this way.

1508          else if (VECT_SCALAR_BOOLEAN_TYPE_P (TREE_TYPE (op))
1509                   && VECTOR_BOOLEAN_TYPE_P (stmt_vectype))
1510            vector_type = truth_type_for (stmt_vectype);

which does not consider that the mask in scatters might need a
vector boolean type.  But it appearantly does.

Test coverage for this code seems to be non-existent though :/

I do have a patch that fixes the testcase though.

[Bug tree-optimization/95487] [10/11 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:887c45fb5b047171e82710baa51108d5c210eb42

commit r11-878-g887c45fb5b047171e82710baa51108d5c210eb42
Author: Richard Biener <rguenther@suse.de>
Date:   Wed Jun 3 15:51:29 2020 +0200

    tree-optimization/95487 - use a truth type for scatter masks

    This makes sure to get a truth type for scatter masks even when they
    are invariant.

    2020-06-03  Richard Biener  <rguenther@suse.de>

            PR tree-optimization/95487
            * tree-vect-stmts.c (vectorizable_store): Use a truth type
            for the scatter mask.

            * g++.dg/vect/pr95487.cc: New testcase.

[Bug tree-optimization/95487] [10 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |11.0
      Known to fail|                            |10.1.0
            Summary|[10/11 Regression] ICE:     |[10 Regression] ICE:
                   |verify_gimple failed        |verify_gimple failed
                   |(error: invalid vector      |(error: invalid vector
                   |types in nop conversion)    |types in nop conversion)
                   |with -O3                    |with -O3
                   |-march=skylake-avx512 since |-march=skylake-avx512 since
                   |r10-1052-gc29c92c789d93848  |r10-1052-gc29c92c789d93848

--- Comment #7 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed on trunk sofar.

[Bug tree-optimization/95487] [10 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-10 branch has been updated by Richard Biener
<rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:2eed94cbf2e8182d700aac3ac52b0967f00a2621

commit r10-8353-g2eed94cbf2e8182d700aac3ac52b0967f00a2621
Author: Richard Biener <rguenther@suse.de>
Date:   Wed Jun 3 15:51:29 2020 +0200

    tree-optimization/95487 - use a truth type for scatter masks

    This makes sure to get a truth type for scatter masks even when they
    are invariant.

    2020-06-03  Richard Biener  <rguenther@suse.de>

            PR tree-optimization/95487
            * tree-vect-stmts.c (vectorizable_store): Use a truth type
            for the scatter mask.

            * g++.dg/vect/pr95487.cc: New testcase.

[Bug tree-optimization/95487] [10 Regression] ICE: verify_gimple failed (error: invalid vector types in nop conversion) with -O3 -march=skylake-avx512 since r10-1052-gc29c92c789d93848

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95487

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to work|                            |10.1.1
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED
           Priority|P3                          |P2

--- Comment #9 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed.