[Bug c/104644] New: [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[Bug c/104644] New: [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

            Bug ID: 104644
           Summary: [12 Regression] ICE: SIGSEGV (infinite recursion in
                    fold_binary_loc / fold_build2_loc /
                    generic_simplify_NE_EXPR)
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu
            Target: x86_64-pc-linux-gnu

Created attachment 52493
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52493&action=edit
reduced testcase

Compiler output:
$ x86_64-pc-linux-gnu-gcc testcase.c -wrapper valgrind,-q
testcase.c: In function 'foo':
testcase.c:4:29: warning: overflow in conversion from 'float' to 'short
unsigned int' changes value from '1.31072e+5f' to '65535' [-Woverflow]
    4 |   return __builtin_bswap16 ((float) 0x20000) != (char) (float) 0x20000;
      |                             ^~~~~~~~~~~~~~~
==21462== Stack overflow in thread #1: can't grow stack to 0x1ffe001000
==21462== Can't extend stack to 0x1ffe000e48 during signal delivery for thread
1:
==21462==   no stack segment
==21462== 
==21462== Process terminating with default action of signal 11 (SIGSEGV)
==21462==  Access not within mapped region at address 0x1FFE000E48
==21462== Stack overflow in thread #1: can't grow stack to 0x1ffe001000
==21462==    at 0xE22224: ggc_internal_alloc(unsigned long, void (*)(void*),
unsigned long, unsigned long) (ggc-page.cc:1278)
==21462==  If you believe this happened as a result of a stack
==21462==  overflow in your program's main thread (unlikely but
==21462==  possible), you can try to increase the size of the
==21462==  main thread stack using the --main-stacksize= flag.
==21462==  The main thread stack size used in this run was 16777216.
x86_64-pc-linux-gnu-gcc: internal compiler error: Segmentation fault signal
terminated program valgrind
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
See <https://gcc.gnu.org/bugs/> for instructions.


(gdb) bt
#0  generic_simplify_113 (loc=2147483649, type=0x7ffff76c35e8,
captures=0x7ffffbfff090, cmp=NE_EXPR, bswap=CFN_BUILT_IN_BSWAP16, 
    _p1=<optimized out>, _p0=<optimized out>) at generic-match.cc:6558
#1  0x0000000001afd8db in generic_simplify_NE_EXPR (loc=2147483649,
type=0x7ffff76c35e8, _p0=0x7ffff70c7620, _p1=0x7ffff77f2510, 
    code=NE_EXPR) at generic-match.cc:61429
#2  0x0000000000fe500f in fold_binary_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff70c7620, op1=0x7ffff77f2510)
    at /repo/gcc-trunk/gcc/fold-const.cc:10862
#3  0x0000000000fedb6a in fold_build2_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff70c7620, op1=0x7ffff77f2510)
    at /repo/gcc-trunk/gcc/fold-const.cc:13814
#4  0x0000000000fedb6a in fold_build2_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff77f2510, op1=0x7ffff70c7620)
    at /repo/gcc-trunk/gcc/fold-const.cc:13814
#5  0x0000000001afd8db in generic_simplify_NE_EXPR (loc=2147483649,
type=0x7ffff76c35e8, _p0=0x7ffff70c75e8, _p1=0x7ffff77f24f8, 
    code=NE_EXPR) at generic-match.cc:61429
#6  0x0000000000fe500f in fold_binary_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff70c75e8, op1=0x7ffff77f24f8)
    at /repo/gcc-trunk/gcc/fold-const.cc:10862
#7  0x0000000000fedb6a in fold_build2_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff70c75e8, op1=0x7ffff77f24f8)
    at /repo/gcc-trunk/gcc/fold-const.cc:13814
#8  0x0000000000fedb6a in fold_build2_loc (loc=2147483649, code=NE_EXPR,
type=0x7ffff76c35e8, op0=0x7ffff77f24f8, op1=0x7ffff70c75e8)
    at /repo/gcc-trunk/gcc/fold-const.cc:13814
#9  0x0000000001afd8db in generic_simplify_NE_EXPR (loc=2147483649,
type=0x7ffff76c35e8, _p0=0x7ffff70c75b0, _p1=0x7ffff77f2510, 
    code=NE_EXPR) at generic-match.cc:61429
...

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r12-7349-20220222175310-g54f74502327-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=yes,rtl,df,extra
--disable-bootstrap --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r12-7349-20220222175310-g54f74502327-checking-yes-rtl-df-extra-nobootstrap-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.0.1 20220222 (experimental) (GCC)

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Target Milestone|---                         |12.0
           Priority|P3                          |P1
             Status|UNCONFIRMED                 |NEW
          Component|c                           |tree-optimization
   Last reconfirmed|                            |2022-02-22
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Started with r12-2516-gcf5f544227f16b63e224529190329eb0edca791c

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |jakub at gcc dot gnu.org

--- Comment #2 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 52494
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52494&action=edit
gcc12-pr104644.patch

Untested fix.

The match.pd optimization relies on (bswap @1) actually being simplified into
something other than bswap when @1 is INTEGER_CST, but that isn't guaranteed at
least in GENERIC - due to TREE_OVERFLOW in there.
The patch uses ! to enforce simplification of that and because ! isn't
supported in GENERIC, also requires GIMPLE for the optimization.

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jakub Jelinek from comment #2)
> Created attachment 52494 [details]
> gcc12-pr104644.patch
> 
> Untested fix.
> 
> The match.pd optimization relies on (bswap @1) actually being simplified
> into something other than bswap when @1 is INTEGER_CST, but that isn't
> guaranteed at least in GENERIC - due to TREE_OVERFLOW in there.
> The patch uses ! to enforce simplification of that and because ! isn't
> supported in GENERIC, also requires GIMPLE for the optimization.

Huh, it's odd that GENERIC would care for TREE_OVERFLOW when constant folding
bswap .. where does it do that?

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I think the FEs rely on such trees not being folded.
It is done in fold-const-call.cc,
1186    static tree
1187    fold_const_call_1 (combined_fn fn, tree type, tree arg)
1188    {
1189      machine_mode mode = TYPE_MODE (type);
1190      machine_mode arg_mode = TYPE_MODE (TREE_TYPE (arg));
1191    
1192      if (integer_cst_p (arg))
1193        {
and integer_cst_p does:
35      /* Functions that test for certain constant types, abstracting away the
36         decision about whether to check for overflow.  */
37      
38      static inline bool
39      integer_cst_p (tree t)
40      {
41        return TREE_CODE (t) == INTEGER_CST && !TREE_OVERFLOW (t);
42      }

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[rguenther at suse dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

--- Comment #5 from rguenther at suse dot de <rguenther at suse dot de> ---
On Wed, 23 Feb 2022, jakub at gcc dot gnu.org wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644
> 
> --- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
> I think the FEs rely on such trees not being folded.

Does it?  I think it simply relies on the overflow being "sticky"
and not lost so that for bswap (100000000 * 1000000000) you get
sth with TREE_OVERFLOW.  That's also what we do when folding
1(OVF) + 2, we return 3(OVF).

Sure, not folding is a possibility as well but isn't fold_const_call_1
eventually invoked from GIMPLE as well where TREE_OVERFLOW doens't
have any semantics?

I suppose we could use force_fit_type instead of wide_int_to_tree
for building the result here.

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:fdc46830f1b793dc791099acfadc3f0f8cc24c0e

commit r12-7361-gfdc46830f1b793dc791099acfadc3f0f8cc24c0e
Author: Richard Biener <rguenther@suse.de>
Date:   Wed Feb 23 13:47:01 2022 +0100

    middle-end/104644 - recursion with bswap match.pd pattern

    The following patch avoids infinite recursion during generic folding.
    The (cmp (bswap @0) INTEGER_CST@1) simplification relies on
    (bswap @1) actually being simplified, if it is not simplified, we just
    move the bswap from one operand to the other and if @0 is also INTEGER_CST,
    we apply the same rule next.

    The reason why bswap @1 isn't folded to INTEGER_CST is that the INTEGER_CST
    has TREE_OVERFLOW set on it and fold-const-call.cc predicate punts in
    such cases:
    static inline bool
    integer_cst_p (tree t)
    {
      return TREE_CODE (t) == INTEGER_CST && !TREE_OVERFLOW (t);
    }
    The patch uses ! modifier to ensure the bswap is simplified and
    extends support to GENERIC by means of requiring !EXPR_P which
    is not perfect but a conservative approximation.

    2022-02-22  Richard Biener  <rguenther@suse.de>

            PR tree-optimization/104644
            * doc/match-and-simplify.texi: Amend ! documentation.
            * genmatch.cc (expr::gen_transform): Code-generate ! support
            for GENERIC.
            (parser::parse_expr): Allow ! for GENERIC.
            * match.pd (cmp (bswap @0) INTEGER_CST@1): Use ! modifier on
            bswap.

            * gcc.dg/pr104644.c: New test.

    Co-Authored-by: Jakub Jelinek <jakub@redhat.com>

[Bug tree-optimization/104644] [12 Regression] ICE: SIGSEGV (infinite recursion in fold_binary_loc / fold_build2_loc / generic_simplify_NE_EXPR)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104644

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #7 from Richard Biener <rguenth at gcc dot gnu.org> ---
Fixed.