[Bug tree-optimization/105385] New: ICE: Aborted (in do_mpfr_arg2): GNU MP: Cannot allocate memory (size=3458764513820540832) with __builtin_jn{,f,l}

[Bug tree-optimization/105385] New: ICE: Aborted (in do_mpfr_arg2): GNU MP: Cannot allocate memory (size=3458764513820540832) with __builtin_jn{,f,l}

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105385

            Bug ID: 105385
           Summary: ICE: Aborted (in do_mpfr_arg2): GNU MP: Cannot
                    allocate memory (size=3458764513820540832) with
                    __builtin_jn{,f,l}
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu

Created attachment 52875
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52875&action=edit
reduced testcase

I don't know if this should be reported against gcc or libmpfr.

Compiler output:
$ x86_64-pc-linux-gnu-gcc testcase.c
GNU MP: Cannot allocate memory (size=3458764513820540832)
testcase.c: In function 'foo':
testcase.c:4:3: internal compiler error: Aborted
    4 |   __builtin_jn (4292622104, 0xff00ff00ff00ff00);
      |   ^~~~~~~~~~~~
0x13a3dff crash_signal
        /repo/gcc-trunk/gcc/toplev.cc:322
0x100fe49 do_mpfr_arg2
        /repo/gcc-trunk/gcc/fold-const-call.cc:232
0x1012295 do_mpfr_arg2
        /repo/gcc-trunk/gcc/fold-const-call.cc:222
0x1012295 fold_const_call_sss
        /repo/gcc-trunk/gcc/fold-const-call.cc:1462
0x1012295 fold_const_call_1
        /repo/gcc-trunk/gcc/fold-const-call.cc:1562
0xe4ab9a fold_builtin_2
        /repo/gcc-trunk/gcc/builtins.cc:9376
0xe4ab9a fold_builtin_n
        /repo/gcc-trunk/gcc/builtins.cc:9540
0xfdcf3a fold_build_call_array_loc(unsigned int, tree_node*, tree_node*, int,
tree_node**)
        /repo/gcc-trunk/gcc/fold-const.cc:13965
0xd52b3f build_function_call_vec(unsigned int, vec<unsigned int, va_heap,
vl_ptr>, tree_node*, vec<tree_node*, va_gc, vl_embed>*, vec<tree_node*, va_gc,
vl_embed>*, tree_node*)
        /repo/gcc-trunk/gcc/c/c-typeck.cc:3217
0xd74093 c_parser_postfix_expression_after_primary
        /repo/gcc-trunk/gcc/c/c-parser.cc:10561
0xd6b3e0 c_parser_postfix_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:10234
0xd6f8ea c_parser_unary_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:8261
0xd71081 c_parser_cast_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:8103
0xd712ee c_parser_binary_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:7906
0xd7277b c_parser_conditional_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:7606
0xd72fa0 c_parser_expr_no_commas
        /repo/gcc-trunk/gcc/c/c-parser.cc:7521
0xd73211 c_parser_expression
        /repo/gcc-trunk/gcc/c/c-parser.cc:10697
0xd73978 c_parser_expression_conv
        /repo/gcc-trunk/gcc/c/c-parser.cc:10736
0xd68957 c_parser_statement_after_labels
        /repo/gcc-trunk/gcc/c/c-parser.cc:6263
0xd6af3a c_parser_compound_statement_nostart
        /repo/gcc-trunk/gcc/c/c-parser.cc:5800
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.

[Bug tree-optimization/105385] ICE: Aborted (in do_mpfr_arg2): GNU MP: Cannot allocate memory (size=3458764513820540832) with __builtin_jn{, f, l}

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105385

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |marxin at gcc dot gnu.org
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2022-04-25
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Confirmed:

Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6,
no_tid=no_tid@entry=0) at pthread_kill.c:44
44            return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO
(ret) : 0;
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff79081e3 in __pthread_kill_internal (signo=6, threadid=<optimized
out>) at pthread_kill.c:78
#2  0x00007ffff78b8306 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3  0x00007ffff78a1813 in __GI_abort () at abort.c:79
#4  0x00007ffff7c3cd47 in __gmp_default_allocate (size=3458764513820540832) at
/usr/src/debug/gmp-6.2.1-4.5.x86_64/memory.c:57
#5  __gmp_default_allocate (size=3458764513820540832) at
/usr/src/debug/gmp-6.2.1-4.5.x86_64/memory.c:46
#6  0x00007ffff7ce98a1 in mpfr_allocate_func (alloc_size=3458764513820540832)
at /usr/src/debug/mpfr-4.1.0-2.18.x86_64/src/mpfr-gmp.c:315
#7  0x00007ffff7d26838 in mpfr_jn (res=0x7fffffffc460, n=-2345192,
z=0x7fffffffc460, r=MPFR_RNDN) at
/usr/src/debug/mpfr-4.1.0-2.18.x86_64/src/jn.c:245
#8  0x0000000000bd73ba in do_mpfr_arg2 (result=result@entry=0x7fffffffc530,
func=0x7ffff7d264c0 <mpfr_jn>, arg0=..., arg1=arg1@entry=0x7ffff77592a0,
format=format@entry=0x2785c40 <ieee_double_format>) at
/home/marxin/Programming/gcc/gcc/fold-const-call.cc:232
#9  0x0000000000bd9aab in do_mpfr_arg2 (format=0x2785c40 <ieee_double_format>,
arg1=0x7ffff77592a0, arg0=..., func=<optimized out>, result=0x7fffffffc530) at
/home/marxin/Programming/gcc/gcc/fold-const-call.cc:222
#10 fold_const_call_sss (format=0x2785c40 <ieee_double_format>,
arg1=0x7ffff77592a0, arg0=..., fn=CFN_BUILT_IN_JN, result=0x7fffffffc530) at
/home/marxin/Programming/gcc/gcc/fold-const-call.cc:1462
#11 fold_const_call_1 (fn=CFN_BUILT_IN_JN, type=0x7ffff762f348,
arg0=0x7ffff7756450, arg1=<optimized out>) at
/home/marxin/Programming/gcc/gcc/fold-const-call.cc:1562

with:

Name           : mpfr-devel
Version        : 4.1.0-2.18

[Bug tree-optimization/105385] ICE: Aborted (in do_mpfr_arg2): GNU MP: Cannot allocate memory (size=3458764513820540832) with __builtin_jn{, f, l}

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105385

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
IIRC gmp/mpfr have no good way to report errors like this up the chain and so
they abort ...