[Bug c++/115599] New: ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[Bug c++/115599] New: ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[iamanonymous.cs at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

            Bug ID: 115599
           Summary: ICE: qsort checking failed during GIMPLE pass: reassoc
                    (error: qsort comparator non-negative on sorted
                    output: 150142972)
           Product: gcc
           Version: 15.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: iamanonymous.cs at gmail dot com
  Target Milestone: ---
            Target: x86_64

*******************************************************************************
The compiler produces an internal error during during GIMPLE pass: reassoc
compiling the provided code with the specified options. 
The issue can also be reproduced on Compiler Explorer.

*******************************************************************************
OS and Platform:
# uname -a
Linux ubuntu 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023
x86_64 x86_64 x86_64 GNU/Linux
*******************************************************************************
# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/root/gdbtest/gcc/gcc-15/libexec/gcc/x86_64-pc-linux-gnu/15.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /root/gdbtest/gcc/obj/../gcc/configure
--prefix=/root/gdbtest/gcc/gcc-15 --enable-languages=c,c++,fortran,go
--disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 15.0.0 20240509 (experimental) (GCC) 
*******************************************************************************
Program: Please refer to the attachment. 

*******************************************************************************

Command Lines: g++ f.cpp -O1  -std=c++2a -Wall -Wextra -pedantic
-fsanitize=undefined -c -o f.o
<source>: In function 'void tf_4_foo()':
<source>:213:79: warning: division by zero [-Wdiv-by-zero]
  213 |                         if ((((0 ? 0 : ((0 ? 0 : ((0 ? 0 :
(tf_4_var_6))))))) / 0) / 0)         {
      |                             
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
<source>:213:84: warning: division by zero [-Wdiv-by-zero]
  213 |                         if ((((0 ? 0 : ((0 ? 0 : ((0 ? 0 :
(tf_4_var_6))))))) / 0) / 0)         {
      |                            
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
<source>:215:1798: warning: division by zero [-Wdiv-by-zero]
  215 |                           if ((((bool) (((0 ? 0 : ((0 ? 0 : ((((bool) 
......
......
......
......
......
((bool) (tf_4_var_246))))))) : ((unsigned int) ((((bool) (((*(tf_4_ptr_152)) ?
(tf_4_array_4 [1]) : (tf_4_var_2)))) ? (((tf_4_var_178) ? ((int) (tf_4_var_72))
: ((int) (tf_4_var_150)))) : ((int) (((bool) (tf_4_var_34)) && ((bool)
(tf_4_var_94))))))))) : ((unsigned int) ((int) (((bool) ((((bool) ((((bool)
(4095U)) ? ((int) (tf_4_var_72)) : (-1)))) ? ((((bool) (-31)) ? ((int)
(tf_4_var_218)) : ((int) (tf_4_array_2 [2])))) : ((int) (((bool) (-1)) &&
((bool) (4095U))))))) || (((bool) ((((bool) (tf_4_var_56)) ? ((unsigned int)
((int) (tf_4_var_148))) : (*(tf_4_ptr_8))))) || (((bool) (tf_4_var_148)) ||
((bool) (tf_4_array_4 [2]))))))))));                                         }
<source>:209:28: error: qsort comparator non-negative on sorted output:
2147418113
  209 |                       void tf_4_foo () {
      |                            ^~~~~~~~
during GIMPLE pass: reassoc
<source>:209:28: internal compiler error: qsort checking failed
0x26cfeec internal_error(char const*, ...)
        ???:0
0x2715408 gcc_qsort(void*, unsigned long, unsigned long, int (*)(void const*,
void const*))
        ???:0
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
Compiler returned: 1
*******************************************************************************

Also ICE on trunk, compiler explorer:https://godbolt.org/z/7xoGq3T4h

*******************************************************************************

[Bug c++/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2024-06-23
           Keywords|                            |ice-checking,
                   |                            |missed-optimization
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
             Status|UNCONFIRMED                 |ASSIGNED
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
It's compare_repeat_factors

(gdb) p (*repeat_factor_vec.m_vec)[0]
$6 = (repeat_factor &) @0x5d6a7b8: {factor = <ssa_name 0x7ffff3396a20 36814>, 
  rank = 2147549184, count = 1, repr = <tree 0x0>}
(gdb) p (*repeat_factor_vec.m_vec)[4]
$7 = (repeat_factor &) @0x5d6a838: {factor = <ssa_name 0x7ffff33db8b8 15299>, 
  rank = 1, count = 1, repr = <tree 0x0>}

where one obvious issue might be that rank is unsigned (and count is
int64).

I'm testing a fix.

Note this bug only results in less optimal code.

[Bug tree-optimization/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |needs-reduction
          Component|c++                         |tree-optimization

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Testcase too big for the testsuite.

[Bug tree-optimization/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[sjames at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

Sam James <sjames at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |82407
                 CC|                            |sjames at gcc dot gnu.org

--- Comment #3 from Sam James <sjames at gcc dot gnu.org> ---
I'll throw some CPU hours at it.


Referenced Bugs:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82407
[Bug 82407] [meta-bug] qsort_chk fallout tracking

[Bug tree-optimization/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[sjames at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

--- Comment #4 from Sam James <sjames at gcc dot gnu.org> ---
Created attachment 58497
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58497&action=edit
foo.cxx.xz

Attaching original.

[Bug tree-optimization/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

--- Comment #5 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:ae13af26060eb686418ea9c9d455cd665049402d

commit r15-1577-gae13af26060eb686418ea9c9d455cd665049402d
Author: Richard Biener <rguenther@suse.de>
Date:   Sun Jun 23 14:37:53 2024 +0200

    tree-optimization/115599 - reassoc qsort comparator issue

    The compare_repeat_factors comparator fails qsort checking eventually
    because it uses rf2->rank - rf1->rank to compare unsigned numbers
    which causes issues for ranks that interpret negative as signed.

    Fixed by re-writing the obvious way.  I've also fixed the count
    comparison which suffers from truncation as count is 64bit signed
    while the comparator result is 32bit int (that's a lot less likely
    to hit in practice though).

    The testcase from the PR is too large to include.

            PR tree-optimization/115599
            * tree-ssa-reassoc.cc (compare_repeat_factors): Use explicit
            compares to avoid truncations.

[Bug tree-optimization/115599] ICE: qsort checking failed during GIMPLE pass: reassoc (error: qsort comparator non-negative on sorted output: 150142972)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115599

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed.