[Bug target/114184] New: ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with -Og -mavx512f and __builtin_memmove() _BitInt(256)

[Bug target/114184] New: ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with -Og -mavx512f and __builtin_memmove() _BitInt(256)

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

            Bug ID: 114184
           Summary: ICE: in extract_insn, at recog.cc:2812 (unrecognizable
                    insn ) with -Og -mavx512f and __builtin_memmove()
                    _BitInt(256)
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
                CC: jakub at gcc dot gnu.org
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu
            Target: x86_64-pc-linux-gnu

Created attachment 57582
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57582&action=edit
reduced testcase

Compiler output:
$ x86_64-pc-linux-gnu-gcc -Og -mavx512f testcase.c 
testcase.c: In function 'foo':
testcase.c:7:1: error: unrecognizable insn:
    7 | }
      | ^
(insn 5 2 6 2 (set (reg/v:XF 98 [ d ])
        (subreg:XF (const_vector:V32QI [
                    (const_int -107 [0xffffffffffffff95])
                    (const_int -120 [0xffffffffffffff88])
                    (const_int 89 [0x59])
                    (const_int 42 [0x2a])
                    (const_int 38 [0x26])
                    (const_int -16 [0xfffffffffffffff0])
                    (const_int -60 [0xffffffffffffffc4])
                    (const_int -62 [0xffffffffffffffc2])
                    (const_int 0 [0]) repeated x24
                ]) 0)) "testcase.c":6:3 -1
     (nil))
during RTL pass: vregs
testcase.c:7:1: internal compiler error: in extract_insn, at recog.cc:2812
0x807075 _fatal_insn(char const*, rtx_def const*, char const*, int, char
const*)
        /repo/gcc-trunk/gcc/rtl-error.cc:108
0x8070f2 _fatal_insn_not_found(rtx_def const*, char const*, int, char const*)
        /repo/gcc-trunk/gcc/rtl-error.cc:116
0x7f5f61 extract_insn(rtx_insn*)
        /repo/gcc-trunk/gcc/recog.cc:2812
0x113331d instantiate_virtual_regs_in_insn
        /repo/gcc-trunk/gcc/function.cc:1611
0x113331d instantiate_virtual_regs
        /repo/gcc-trunk/gcc/function.cc:1994
0x113331d execute
        /repo/gcc-trunk/gcc/function.cc:2041
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r14-9247-20240229194557-gc6d4fb0062c-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/14.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=yes,rtl,df,extra
--disable-bootstrap --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --enable-libsanitizer
--disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r14-9247-20240229194557-gc6d4fb0062c-checking-yes-rtl-df-extra-nobootstrap-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.0.1 20240301 (experimental) (GCC)

[Bug target/114184] [12/13/14 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P2
            Summary|ICE: in extract_insn, at    |[12/13/14 Regression] ICE:
                   |recog.cc:2812               |in extract_insn, at
                   |(unrecognizable insn ) with |recog.cc:2812
                   |-Og -mavx512f and           |(unrecognizable insn ) with
                   |__builtin_memmove()         |_Complex long double and
                   |_BitInt(256)                |vector VCE
   Target Milestone|---                         |12.4

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Seems unrelated to _BitInt.
E.g. following testcase ICEs with -O2 -mavx2 since
r14-4537-g70b5c6981fcdff246f90e57e91f3e1667eab2eb3
typedef unsigned char V __attribute__((vector_size (32)));
_Complex long double
foo (void)
{
  _Complex long double d;
  *(V *)&d = (V) { 149, 136, 89, 42, 38, 240, 196, 194 };
  return d;
}
and the same with -Og -mavx2 since
r12-7240-g2801f23fb82a5ef51c8b460a500786797943e1e9
I don't see bugs in either of those commits.

What I see happing is that expand_assignment, because the destination is
complex, does
6211                          emit_move_insn (XEXP (to_rtx, 0),
6212                                          read_complex_part (from_rtx,
false));
6213                          emit_move_insn (XEXP (to_rtx, 1),
6214                                          read_complex_part (from_rtx,
true));
where from_rtx at that point is
(subreg:XC (const_vector:V32QI [
            (const_int -107 [0xffffffffffffff95])
            (const_int -120 [0xffffffffffffff88])
            (const_int 89 [0x59])
            (const_int 42 [0x2a])
            (const_int 38 [0x26])
            (const_int -16 [0xfffffffffffffff0])
            (const_int -60 [0xffffffffffffffc4])
            (const_int -62 [0xffffffffffffffc2])
            (const_int 0 [0]) repeated x24
        ]) 0)
which is supposedly a valid subreg, reinterpretation of a vector as complex
extended double, which is not foldable to constant because it isn't a valid
IEEE value which the compiler can express.  Or should this have been a concat?
Anyway, read_complex_part returns for that
(const_double:XF 0.0 [0x0.0p+0])
for the imag part and
(subreg:XF (const_vector:V32QI [
            (const_int -107 [0xffffffffffffff95])
            (const_int -120 [0xffffffffffffff88])
            (const_int 89 [0x59])
            (const_int 42 [0x2a])
            (const_int 38 [0x26])
            (const_int -16 [0xfffffffffffffff0])
            (const_int -60 [0xffffffffffffffc4])
            (const_int -62 [0xffffffffffffffc2])
            (const_int 0 [0]) repeated x24
        ]) 0)
for the real part.  The latter makes it through validate_subreg due to the
  /* ??? Similarly, e.g. with (subreg:DF (reg:TI)).  Though store_bit_field
     is the culprit here, and not the backends.  */
  else if (known_ge (osize, regsize) && known_ge (isize, osize))
    ;
- osize is 16, isize is 32 and regsize is 8.  If that wasn't for that rule,
there would be the
  /* Subregs involving floating point modes are not allowed to
     change size unless it's an insert into a complex mode.
     Therefore (subreg:DI (reg:DF) 0) and (subreg:CS (reg:SF) 0) are fine, but
     (subreg:SI (reg:DF) 0) isn't.  */
rule which would reject it.

[Bug target/114184] [12/13/14 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

--- Comment #2 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
BTW,
typedef unsigned char V __attribute__((vector_size (16)));
long double
foo (void)
{
  long double d;
  *(V *)&d = (V) { 149, 136, 89, 42, 38, 240, 196, 194 };
  return d;
}
ICEs at -O2 too since r14-4537-g70b5c6981fcdff246f90e57e91f3e1667eab2eb3
so I'm afraid even trying to optimize the
XFmode lowpart SUBREG of the CONST_VECTOR into same size SUBREG from half sized
CONST_VECTOR to XFmode wouldn't help, because that is exactly what we have in
this testcase,
(insn 5 2 6 2 (set (reg/v:XF 98 [ d ])
        (subreg:XF (const_vector:V16QI [
                    (const_int -107 [0xffffffffffffff95])
                    (const_int -120 [0xffffffffffffff88])
                    (const_int 89 [0x59])
                    (const_int 42 [0x2a])
                    (const_int 38 [0x26])
                    (const_int -16 [0xfffffffffffffff0])
                    (const_int -60 [0xffffffffffffffc4])
                    (const_int -62 [0xffffffffffffffc2])
                    (const_int 0 [0]) repeated x8
                ]) 0)) "pr114184-4.c":6:12 -1
     (nil))

[Bug target/114184] [12/13/14 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |uros at gcc dot gnu.org

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
So, one way to fix this or work around that would be to
--- gcc/config/i386/i386-expand.cc.jj   2024-02-26 07:29:27.695974161 +0100
+++ gcc/config/i386/i386-expand.cc      2024-03-01 12:48:59.678574710 +0100
@@ -451,6 +451,12 @@ ix86_expand_move (machine_mode mode, rtx
          && GET_MODE (SUBREG_REG (op1)) == DImode
          && SUBREG_BYTE (op1) == 0)
        op1 = gen_rtx_ZERO_EXTEND (TImode, SUBREG_REG (op1));
+      /* As not all values in XFmode are representable in real_value,
+        we might be called with non-general_operand SUBREGs.  */
+      if (mode == XFmode
+         && !general_operand (op1, XFmode)
+         && can_create_pseudo_p ())
+       op1 = force_reg (XFmode, op1);
       break;
     }

Though, e.g. md.texi suggests against using force_reg in the mov optab (though,
perhaps it is meant just that it shouldn't be used during reload).
Of course, one could argue it is middle-end's fault for not honoring the mov
optab predicate, and that emit_move_insn_1's
  code = optab_handler (mov_optab, mode);
  if (code != CODE_FOR_nothing)
    return emit_insn (GEN_FCN (code) (x, y));
should verify the predicate there.  Changing that feels very risky in stage4 or
on release branches, moves are really quite special.  Or check it in
emit_move_insn callers where there are risks the predicates aren't satisfied
(though, there are almost 500 callers of that just in middle-end code).

[Bug target/114184] [12/13/14 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

--- Comment #4 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:ea1c16f95b8fbaba4a7f3663ff9933ebedfb92a5

commit r14-9289-gea1c16f95b8fbaba4a7f3663ff9933ebedfb92a5
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Mon Mar 4 10:04:19 2024 +0100

    i386: Fix ICEs with SUBREGs from vector etc. constants to XFmode [PR114184]

    The Intel extended format has the various weird number categories,
    pseudo denormals, pseudo infinities, pseudo NaNs and unnormals.
    Those are not representable in the GCC real_value and so neither
    GIMPLE nor RTX VIEW_CONVERT_EXPR/SUBREG folding folds those into
    constants.

    As can be seen on the following testcase, because it isn't folded
    (since GCC 12, before that we were folding it) we can end up with
    a SUBREG of a CONST_VECTOR or similar constant, which isn't valid
    general_operand, so we ICE during vregs pass trying to recognize
    the move instruction.
    Initially I thought it is a middle-end bug, the movxf instruction
    has general_operand predicate, but the middle-end certainly never
    tests that predicate, seems moves are special optabs.
    And looking at other mov optabs, e.g. for vector modes the i386
    patterns use nonimmediate_operand predicate on the input, yet
    ix86_expand_vector_move deals with CONSTANT_P and SUBREG of CONSTANT_P
    arguments which if the predicate was checked couldn't ever make it through.

    The following patch handles this case similarly to the
    ix86_expand_vector_move's SUBREG of CONSTANT_P case, does it just for
XFmode
    because I believe that is the only mode that needs it from the scalar ones,
    others should just be folded.

    2024-03-04  Jakub Jelinek  <jakub@redhat.com>

            PR target/114184
            * config/i386/i386-expand.cc (ix86_expand_move): If XFmode op1
            is SUBREG of CONSTANT_P, force the SUBREG_REG into memory or
            register.

            * gcc.target/i386/pr114184.c: New test.

[Bug target/114184] [12/13 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
            Summary|[12/13/14 Regression] ICE:  |[12/13 Regression] ICE: in
                   |in extract_insn, at         |extract_insn, at
                   |recog.cc:2812               |recog.cc:2812
                   |(unrecognizable insn ) with |(unrecognizable insn ) with
                   |_Complex long double and    |_Complex long double and
                   |vector VCE                  |vector VCE
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2024-03-04
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

[Bug target/114184] [12/13 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

--- Comment #5 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Jakub Jelinek
<jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:128860abd58605d616c184a9a68886a25862b2dd

commit r13-8446-g128860abd58605d616c184a9a68886a25862b2dd
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Mon Mar 4 10:04:19 2024 +0100

    i386: Fix ICEs with SUBREGs from vector etc. constants to XFmode [PR114184]

    The Intel extended format has the various weird number categories,
    pseudo denormals, pseudo infinities, pseudo NaNs and unnormals.
    Those are not representable in the GCC real_value and so neither
    GIMPLE nor RTX VIEW_CONVERT_EXPR/SUBREG folding folds those into
    constants.

    As can be seen on the following testcase, because it isn't folded
    (since GCC 12, before that we were folding it) we can end up with
    a SUBREG of a CONST_VECTOR or similar constant, which isn't valid
    general_operand, so we ICE during vregs pass trying to recognize
    the move instruction.
    Initially I thought it is a middle-end bug, the movxf instruction
    has general_operand predicate, but the middle-end certainly never
    tests that predicate, seems moves are special optabs.
    And looking at other mov optabs, e.g. for vector modes the i386
    patterns use nonimmediate_operand predicate on the input, yet
    ix86_expand_vector_move deals with CONSTANT_P and SUBREG of CONSTANT_P
    arguments which if the predicate was checked couldn't ever make it through.

    The following patch handles this case similarly to the
    ix86_expand_vector_move's SUBREG of CONSTANT_P case, does it just for
XFmode
    because I believe that is the only mode that needs it from the scalar ones,
    others should just be folded.

    2024-03-04  Jakub Jelinek  <jakub@redhat.com>

            PR target/114184
            * config/i386/i386-expand.cc (ix86_expand_move): If XFmode op1
            is SUBREG of CONSTANT_P, force the SUBREG_REG into memory or
            register.

            * gcc.target/i386/pr114184.c: New test.

    (cherry picked from commit ea1c16f95b8fbaba4a7f3663ff9933ebedfb92a5)

[Bug target/114184] [12 Regression] ICE: in extract_insn, at recog.cc:2812 (unrecognizable insn ) with _Complex long double and vector VCE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114184

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[12/13 Regression] ICE: in  |[12 Regression] ICE: in
                   |extract_insn, at            |extract_insn, at
                   |recog.cc:2812               |recog.cc:2812
                   |(unrecognizable insn ) with |(unrecognizable insn ) with
                   |_Complex long double and    |_Complex long double and
                   |vector VCE                  |vector VCE

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed for 13.3+ too.