[Bug middle-end/114312] New: ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1 (tree-chrec.cc:336) at -O3 with _BitInt()

[Bug middle-end/114312] New: ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1 (tree-chrec.cc:336) at -O3 with _BitInt()

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114312

            Bug ID: 114312
           Summary: ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1
                    (tree-chrec.cc:336) at -O3 with _BitInt()
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: middle-end
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
                CC: jakub at gcc dot gnu.org
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu
            Target: x86_64-pc-linux-gnu

Created attachment 57673
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=57673&action=edit
reduced testcase

Compiler output:
$ x86_64-pc-linux-gnu-gcc -O3 testcase.c -wrapper gdb,--args
GNU gdb (Gentoo 14.1 vanilla) 14.1
...
(gdb) r
Starting program:
/repo/gcc-trunk/binary-trunk-r14-9425-20240311112226-g119f5ae0455-checking-yes-rtl-df-extra-nobootstrap-amd64/libexec/gcc/x86_64-pc-linux-gnu/14.0.1/cc1
-quiet -iprefix
/repo/gcc-trunk/binary-trunk-r14-9425-20240311112226-g119f5ae0455-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../lib/gcc/x86_64-pc-linux-gnu/14.0.1/
testcase.c -quiet -dumpdir a- -dumpbase testcase.c -dumpbase-ext .c
-mtune=generic -march=x86-64 -O3 -o /tmp/ccMD8BgN.s
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
ggc_internal_alloc (size=size@entry=32, f=f@entry=0x0, s=s@entry=0,
n=n@entry=1) at /repo/gcc-trunk/gcc/ggc-page.cc:1401
1401      memset (result, 0xaf, object_size);
(gdb) bt
#0  ggc_internal_alloc (size=size@entry=32, f=f@entry=0x0, s=s@entry=0,
n=n@entry=1) at /repo/gcc-trunk/gcc/ggc-page.cc:1401
#1  0x0000000001820695 in ggc_internal_alloc (s=32) at
/repo/gcc-trunk/gcc/ggc.h:136
#2  ggc_alloc_tree_node_stat (s=32) at /repo/gcc-trunk/gcc/ggc.h:319
#3  build1 (code=code@entry=NOP_EXPR, type=type@entry=0x7ffff75b1888,
node=node@entry=0x7ffff75ea870) at /repo/gcc-trunk/gcc/tree.cc:5010
#4  0x000000000110e71d in build1_loc (arg1=0x7ffff75ea870, type=0x7ffff75b1888,
code=NOP_EXPR, loc=0) at /repo/gcc-trunk/gcc/tree.h:4740
#5  fold_build1_loc (loc=0, code=NOP_EXPR, type=0x7ffff75b1888,
op0=0x7ffff75ea870) at /repo/gcc-trunk/gcc/fold-const.cc:14144
#6  0x00000000027db3a1 in chrec_convert_1 (type=0x7ffff75b1888,
chrec=0x7ffff75ea870, at_stmt=0x0, use_overflow_semantics=<optimized out>,
from=0x0)
    at /repo/gcc-trunk/gcc/tree-chrec.cc:1601
#7  0x00000000027dd2b4 in chrec_convert (from=0x0, use_overflow_semantics=true,
at_stmt=0x0, chrec=0x7ffff75ea870, type=0x7ffff75b1888) at
/repo/gcc-trunk/gcc/tree-chrec.cc:1655
#8  chrec_fold_plus_1 (op1=<optimized out>, op0=0x7ffff75ea870,
type=0x7ffff7422000, code=MINUS_EXPR) at /repo/gcc-trunk/gcc/tree-chrec.cc:336
#9  chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff7422000,
op0=0x7ffff75ea870, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
#10 0x00000000027dcfd5 in chrec_fold_plus_1 (op1=<optimized out>,
op0=0x7ffff5f4e900, type=0x7ffff75b1888, code=MINUS_EXPR) at
/repo/gcc-trunk/gcc/tree-chrec.cc:336
#11 chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff75b1888,
op0=0x7ffff5f4e900, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
#12 0x00000000027dcfd5 in chrec_fold_plus_1 (op1=<optimized out>,
op0=0x7ffff75ea870, type=0x7ffff7422000, code=MINUS_EXPR) at
/repo/gcc-trunk/gcc/tree-chrec.cc:336
#13 chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff7422000,
op0=0x7ffff75ea870, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
#14 0x00000000027dcfd5 in chrec_fold_plus_1 (op1=<optimized out>,
op0=0x7ffff5f4e8c0, type=0x7ffff75b1888, code=MINUS_EXPR) at
/repo/gcc-trunk/gcc/tree-chrec.cc:336
#15 chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff75b1888,
op0=0x7ffff5f4e8c0, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
#16 0x00000000027dcfd5 in chrec_fold_plus_1 (op1=<optimized out>,
op0=0x7ffff75ea870, type=0x7ffff7422000, code=MINUS_EXPR) at
/repo/gcc-trunk/gcc/tree-chrec.cc:336
#17 chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff7422000,
op0=0x7ffff75ea870, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
#18 0x00000000027dcfd5 in chrec_fold_plus_1 (op1=<optimized out>,
op0=0x7ffff5f4e880, type=0x7ffff75b1888, code=MINUS_EXPR) at
/repo/gcc-trunk/gcc/tree-chrec.cc:336
#19 chrec_fold_plus_1 (code=MINUS_EXPR, type=0x7ffff75b1888,
op0=0x7ffff5f4e880, op1=<optimized out>) at
/repo/gcc-trunk/gcc/tree-chrec.cc:233
...

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r14-9425-20240311112226-g119f5ae0455-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/14.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=yes,rtl,df,extra
--disable-bootstrap --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --enable-libsanitizer
--disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r14-9425-20240311112226-g119f5ae0455-checking-yes-rtl-df-extra-nobootstrap-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 14.0.1 20240311 (experimental) (GCC)

[Bug middle-end/114312] ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1 (tree-chrec.cc:336) at -O3 with _BitInt()

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114312

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2024-03-11
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Confirmed.

I thought there might be a way to hit this without _BitInt but even changing
`_BitInt(64)` to `long` causes the ICE/infinite loop to go away.

[Bug middle-end/114312] ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1 (tree-chrec.cc:336) at -O3 with _BitInt()

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114312

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
I will have a look.

[Bug middle-end/114312] ICE: SIGSEGV: infinite recursion in chrec_fold_plus_1 (tree-chrec.cc:336) at -O3 with _BitInt()

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=114312

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
Meh, fixed by

commit 73dac51b32575f980289c073969c6d825963d076 (origin/master, origin/HEAD)
Author: Richard Biener <rguenther@suse.de>
Date:   Tue Mar 12 14:00:05 2024 +0100

    tree-optimization/114121 - chrec_fold_{plus,multiply} and recursion

    The following addresses endless recursion in the
    chrec_fold_{plus,multiply} functions when handling sign-conversions.
    We only need to apply tricks when we'd fail (there's a chrec in the
    converted operand) and we need to make sure to not turn the other
    operand into something worse (for the chrec-vs-chrec case).

            PR tree-optimization/114121
            * tree-chrec.cc (chrec_fold_plus_1): Guard recursion with
            converted operand properly.
            (chrec_fold_multiply): Likewise.  Handle missed recursion.

            * gcc.dg/torture/pr114312.c: New testcase.