[Bug target/113656] New: [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[Bug target/113656] New: [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[mjires at suse dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

            Bug ID: 113656
           Summary: [x86] ICE in simplify_const_unary_operation, at
                    simplify-rtx.cc:1954 with new -mavx10.1
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: mjires at suse dot cz
                CC: haochen.jiang at intel dot com
  Target Milestone: ---
            Target: x86_64-pc-linux-gnu

Compiling reduced testcase gcc.target/i386/avx512fp16-broadcast-2.c results in
ICE. Bisected to initial AVX10.1 patch r14-5607-g2f8f7ee2db82a3.

$ cat avx512fp16-broadcast-2.c
_Float16 test_256_src[8];
void test_256() {
  for (int i = 0; i < 8; i++)
    test_256_src[i] = i - 8.4;
}


$ gcc avx512fp16-broadcast-2.c -frounding-math -O3 -mavx10.1
-funsafe-math-optimizations
during RTL pass: combine
avx512fp16-broadcast-2.c: In function ‘test_256’:
avx512fp16-broadcast-2.c:5:1: internal compiler error: in
simplify_const_unary_operation, at simplify-rtx.cc:1954
    5 | }
      | ^
0x169ccd4 simplify_const_unary_operation(rtx_code, machine_mode, rtx_def*,
machine_mode)
        /home/mjires/git/GCC/master/gcc/simplify-rtx.cc:1954
0x1698783 simplify_context::simplify_unary_operation(rtx_code, machine_mode,
rtx_def*, machine_mode)
        /home/mjires/git/GCC/master/gcc/simplify-rtx.cc:889
0x1696b93 simplify_context::simplify_gen_unary(rtx_code, machine_mode,
rtx_def*, machine_mode)
        /home/mjires/git/GCC/master/gcc/simplify-rtx.cc:360
0x169a0bb simplify_context::simplify_unary_operation_1(rtx_code, machine_mode,
rtx_def*)
        /home/mjires/git/GCC/master/gcc/simplify-rtx.cc:1304
0x16987aa simplify_context::simplify_unary_operation(rtx_code, machine_mode,
rtx_def*, machine_mode)
        /home/mjires/git/GCC/master/gcc/simplify-rtx.cc:893
0x101ade9 simplify_unary_operation(rtx_code, machine_mode, rtx_def*,
machine_mode)
        /home/mjires/git/GCC/master/gcc/rtl.h:3486
0x2e56f58 combine_simplify_rtx
        /home/mjires/git/GCC/master/gcc/combine.cc:5690
0x2e56c13 subst
        /home/mjires/git/GCC/master/gcc/combine.cc:5609
0x2e56946 subst
        /home/mjires/git/GCC/master/gcc/combine.cc:5536
0x2e56946 subst
        /home/mjires/git/GCC/master/gcc/combine.cc:5536
0x2e4f6f1 try_combine
        /home/mjires/git/GCC/master/gcc/combine.cc:3302
0x2e49be5 combine_instructions
        /home/mjires/git/GCC/master/gcc/combine.cc:1264
0x2e71cf2 rest_of_handle_combine
        /home/mjires/git/GCC/master/gcc/combine.cc:15091
0x2e71dae execute
        /home/mjires/git/GCC/master/gcc/combine.cc:15135
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.


$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/home/mjires/built/master/libexec/gcc/x86_64-pc-linux-gnu/14.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /home/mjires/git/GCC/master/configure
--prefix=/home/mjires/built/master --disable-bootstrap
--enable-languages=c,c++,fortran,lto --disable-multilib --disable-libsanitizer
--enable-checking : (reconfigured) /home/mjires/git/GCC/master/configure
--prefix=/home/mjires/built/master --disable-bootstrap
--enable-languages=c,c++,fortran,lto --disable-multilib --disable-libsanitizer
--enable-checking
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.0.1 20240129 (experimental) (GCC)

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[haochen.jiang at intel dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #1 from Haochen Jiang <haochen.jiang at intel dot com> ---
From the first glance, it seems that the op here is wrongly interpreted.
Investigating why.

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[haochen.jiang at intel dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #2 from Haochen Jiang <haochen.jiang at intel dot com> ---
Actually it is caused by option -funsafe-math-optimizations but not -mavx10.1.

Before my commit, while using option:

-frounding-math -O3 -mavx512fp16 -mavx512vl -funsafe-math-optimizations

It will also report ICE. In GCC13.2, it won't, which means it is introduced in
GCC14.

You got that bisect result since it is when avx10.1 first introduced. -mavx10.1
will enable -mavx512fp16 and -mavx512vl.

When we eliminate -funsafe-math-optimizations, it will be ok.

Also if we are only using -mavx512vl, everything is ok. Seems like something
got disabled under -mavx512fp16.

Need more bisect with option: -frounding-math -O3 -mavx512fp16 -mavx512vl
-funsafe-math-optimizations to find out why.

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[haochen.jiang at intel dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #3 from Haochen Jiang <haochen.jiang at intel dot com> ---
(In reply to Haochen Jiang from comment #2)
> Actually it is caused by option -funsafe-math-optimizations but not
> -mavx10.1.
> 
> Before my commit, while using option:
> 
> -frounding-math -O3 -mavx512fp16 -mavx512vl -funsafe-math-optimizations
> 
> It will also report ICE. In GCC13.2, it won't, which means it is introduced
> in GCC14.
> 
> You got that bisect result since it is when avx10.1 first introduced.
> -mavx10.1 will enable -mavx512fp16 and -mavx512vl.
> 
> When we eliminate -funsafe-math-optimizations, it will be ok.
> 
> Also if we are only using -mavx512vl, everything is ok. Seems like something
> got disabled under -mavx512fp16.

What I mean "disabled" here is actually not enabled while using
-funsafe-math-optimizations with -mavx512fp16.

> 
> Need more bisect with option: -frounding-math -O3 -mavx512fp16 -mavx512vl
> -funsafe-math-optimizations to find out why.

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[haochen.jiang at intel dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

Haochen Jiang <haochen.jiang at intel dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |crazylht at gmail dot com

--- Comment #4 from Haochen Jiang <haochen.jiang at intel dot com> ---
From my bisect, it seems that the guilty commit is gcc-14-1707-ge52be6034fa.

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[liuhongt at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

Hongtao Liu <liuhongt at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |liuhongt at gcc dot gnu.org,
                   |                            |rsandifo at gcc dot gnu.org

--- Comment #5 from Hongtao Liu <liuhongt at gcc dot gnu.org> ---
It hit gcc_assert in simplify_const_unary_operation when combine try to
simplify 

(float_truncate:V4HF
  (float_truncate:V4SF
    (mem/u/c:V4DF (symbol_ref/u:DI ("*.LC4") [flags 0x2]) [0  S32 A256])))

to

(float_truncate:V4HF
    (const_vector:V4DF [
            (const_double:DF
-8.4000000000000003552713678800500929355621337890625e+0
[-0x0.86666666666668p+4])
            (const_double:DF
-7.4000000000000003552713678800500929355621337890625e+0
[-0x0.ecccccccccccdp+3])
            (const_double:DF
-6.4000000000000003552713678800500929355621337890625e+0
[-0x0.ccccccccccccdp+3])
            (const_double:DF
-5.4000000000000003552713678800500929355621337890625e+0
[-0x0.acccccccccccdp+3])
        ]))


  if (VECTOR_MODE_P (mode)
      && GET_CODE (op) == CONST_VECTOR
      && known_eq (GET_MODE_NUNITS (mode), CONST_VECTOR_NUNITS (op)))
    {
      gcc_assert (GET_MODE (op) == op_mode); --- hit assert here.

      rtx_vector_builder builder;
      if (!builder.new_unary_operation (mode, op, false))
        return 0;

      unsigned int count = builder.encoded_nelts ();
      for (unsigned int i = 0; i < count; i++)
        {
          rtx x = simplify_unary_operation (code, GET_MODE_INNER (mode),
                                            CONST_VECTOR_ELT (op, i),
                                            GET_MODE_INNER (op_mode));
          if (!x || !valid_for_const_vector_p (mode, x))
            return 0;
          builder.quick_push (x);
        }
      return builder.build ();
    }

The gcc_assert is added by r10-2139-g4ce6ab68894469

Author: Richard Sandiford <richard.sandiford@arm.com>
Date:   Mon Jul 29 08:40:21 2019 +0000

    Implement more rtx vector folds on variable-length vectors

    This patch extends the tree-level folding of variable-length vectors
    so that it can also be used on rtxes.  The first step is to move
    the tree_vector_builder new_unary/binary_operator routines to the
    parent vector_builder class (which in turn means adding a new
    template parameter).  The second step is to make simplify-rtx.c
    use a direct rtx analogue of the VECTOR_CST handling in fold-const.c.

    2019-07-29  Richard Sandiford  <richard.sandiford@arm.com>

    gcc/
            * vector-builder.h (vector_builder): Add a shape template
parameter.
            (vector_builder::new_unary_operation): New function, generalizing
            the old tree_vector_builder function.
            (vector_builder::new_binary_operation): Likewise.
            (vector_builder::binary_encoded_nelts): Likewise.
            * int-vector-builder.h (int_vector_builder): Update template
            parameters to vector_builder.
            (int_vector_builder::shape_nelts): New function.
            * rtx-vector-builder.h (rtx_vector_builder): Update template
            parameters to vector_builder.
            (rtx_vector_builder::shape_nelts): New function.
            (rtx_vector_builder::nelts_of): Likewise.
            (rtx_vector_builder::npatterns_of): Likewise.
            (rtx_vector_builder::nelts_per_pattern_of): Likewise.
            * tree-vector-builder.h (tree_vector_builder): Update template
            parameters to vector_builder.
            (tree_vector_builder::shape_nelts): New function.
            (tree_vector_builder::nelts_of): Likewise.
            (tree_vector_builder::npatterns_of): Likewise.
            (tree_vector_builder::nelts_per_pattern_of): Likewise.
            * tree-vector-builder.c (tree_vector_builder::new_unary_operation)
            (tree_vector_builder::new_binary_operation): Delete.
            (tree_vector_builder::binary_encoded_nelts): Likewise.
            * simplify-rtx.c: Include rtx-vector-builder.h.
            (distributes_over_addition_p): New function.
            (simplify_const_unary_operation)
            (simplify_const_binary_operation): Generalize handling of vector
            constants to include variable-length vectors.
            (test_vector_ops_series): Add more tests.


before that commit, it only assert for GET_MODE_NUNITS


 /* Simplification and canonicalization of RTL.  */

@@ -1753,27 +1754,23 @@ simplify_const_unary_operation (enum rtx_code code,
machine_mode mode,

   if (VECTOR_MODE_P (mode) && GET_CODE (op) == CONST_VECTOR)
     {
-      unsigned int n_elts;
-      if (!CONST_VECTOR_NUNITS (op).is_constant (&n_elts))
-       return NULL_RTX;
-
-      machine_mode opmode = GET_MODE (op);
-      gcc_assert (known_eq (GET_MODE_NUNITS (mode), n_elts));
-      gcc_assert (known_eq (GET_MODE_NUNITS (opmode), n_elts));
+      gcc_assert (GET_MODE (op) == op_mode);

-      rtvec v = rtvec_alloc (n_elts);
-      unsigned int i;
+      rtx_vector_builder builder;
+      if (!builder.new_unary_operation (mode, op, false))
+       return 0;

-      for (i = 0; i < n_elts; i++)
+      unsigned int count = builder.encoded_nelts ();
+      for (unsigned int i = 0; i < count; i++)
        {
          rtx x = simplify_unary_operation (code, GET_MODE_INNER (mode),
                                            CONST_VECTOR_ELT (op, i),
-                                           GET_MODE_INNER (opmode));

[Bug target/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org
   Last reconfirmed|                            |2024-01-30
             Status|UNCONFIRMED                 |ASSIGNED
     Ever confirmed|0                           |1
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

[Bug rtl-optimization/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 57265
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57265&action=edit
gcc14-pr113656.patch

Untested fix.

[Bug rtl-optimization/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #7 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:b59775b642bb2b1ecd2e6d52c988b9c432117bc8

commit r14-8652-gb59775b642bb2b1ecd2e6d52c988b9c432117bc8
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Jan 31 10:56:56 2024 +0100

    simplify-rtx: Fix up last argument to simplify_gen_unary [PR113656]

    When simplifying e.g. (float_truncate:SF (float_truncate:DF (reg:XF))
    or (float_truncate:SF (float_extend:XF (reg:DF)) etc. into
    (float_truncate:SF (reg:XF)) or (float_truncate:SF (reg:DF)) we call
    simplify_gen_unary with incorrect op_mode argument, it should be
    the argument's mode, but we call it with the outer mode instead.
    As these are all floating point operations, the argument always
    has non-VOIDmode and so we can just use that mode (as done in similar
    simplifications a few lines later), but neither FLOAT_TRUNCATE nor
    FLOAT_EXTEND are operations that should have the same modes of operand
    and result.  This bug hasn't been a problem for years because normally
    op_mode is used only if the mode of op is VOIDmode, otherwise it is
    redundant, but r10-2139 added an assertion in some spots that op_mode
    is right even in such cases.

    2024-01-31  Jakub Jelinek  <jakub@redhat.com>

            PR rtl-optimization/113656
            * simplify-rtx.cc (simplify_context::simplify_unary_operation_1)
            <case FLOAT_TRUNCATE>: Fix up last argument to simplify_gen_unary.

            * gcc.target/i386/pr113656.c: New test.

[Bug rtl-optimization/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #8 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Fixed on the trunk so far.

[Bug rtl-optimization/113656] [x86] ICE in simplify_const_unary_operation, at simplify-rtx.cc:1954 with new -mavx10.1

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113656

--- Comment #9 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Jakub Jelinek
<jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:21d7bffdbe2046cc787fa31a5df5c71dc645b304

commit r13-8387-g21d7bffdbe2046cc787fa31a5df5c71dc645b304
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Jan 31 10:56:56 2024 +0100

    simplify-rtx: Fix up last argument to simplify_gen_unary [PR113656]

    When simplifying e.g. (float_truncate:SF (float_truncate:DF (reg:XF))
    or (float_truncate:SF (float_extend:XF (reg:DF)) etc. into
    (float_truncate:SF (reg:XF)) or (float_truncate:SF (reg:DF)) we call
    simplify_gen_unary with incorrect op_mode argument, it should be
    the argument's mode, but we call it with the outer mode instead.
    As these are all floating point operations, the argument always
    has non-VOIDmode and so we can just use that mode (as done in similar
    simplifications a few lines later), but neither FLOAT_TRUNCATE nor
    FLOAT_EXTEND are operations that should have the same modes of operand
    and result.  This bug hasn't been a problem for years because normally
    op_mode is used only if the mode of op is VOIDmode, otherwise it is
    redundant, but r10-2139 added an assertion in some spots that op_mode
    is right even in such cases.

    2024-01-31  Jakub Jelinek  <jakub@redhat.com>

            PR rtl-optimization/113656
            * simplify-rtx.cc (simplify_context::simplify_unary_operation_1)
            <case FLOAT_TRUNCATE>: Fix up last argument to simplify_gen_unary.

            * gcc.target/i386/pr113656.c: New test.

    (cherry picked from commit b59775b642bb2b1ecd2e6d52c988b9c432117bc8)