[Bug rtl-optimization/104522] New: ICE: in real_to_decimal_for_mode, at real.cc:1770 with -O -fcompare-debug

[Bug rtl-optimization/104522] New: ICE: in real_to_decimal_for_mode, at real.cc:1770 with -O -fcompare-debug

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

            Bug ID: 104522
           Summary: ICE: in real_to_decimal_for_mode, at real.cc:1770 with
                    -O -fcompare-debug
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: ice-on-valid-code
          Severity: normal
          Priority: P3
         Component: rtl-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu
            Target: x86_64-pc-linux-gnu

Created attachment 52433
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52433&action=edit
reduced testcase

Compiler output:
$ x86_64-pc-linux-gnu-gcc -O -fcompare-debug testcase.c
during RTL pass: *clean_state
testcase.c: In function 'foo':
testcase.c:10:1: internal compiler error: in real_to_decimal_for_mode, at
real.cc:1770
   10 | }
      | ^
0x75757a real_to_decimal_for_mode(char*, real_value const*, unsigned long,
unsigned long, int, machine_mode)
        /repo/gcc-trunk/gcc/real.cc:1770
0x12bd744 rtx_writer::print_rtx(rtx_def const*)
        /repo/gcc-trunk/gcc/print-rtl.cc:981
0x12be1a2 rtx_writer::print_rtx_operand_code_e(rtx_def const*, int)
        /repo/gcc-trunk/gcc/print-rtl.cc:359
0x12bd34f rtx_writer::print_rtx(rtx_def const*)
        /repo/gcc-trunk/gcc/print-rtl.cc:938
0x12be1a2 rtx_writer::print_rtx_operand_code_e(rtx_def const*, int)
        /repo/gcc-trunk/gcc/print-rtl.cc:359
0x12bd34f rtx_writer::print_rtx(rtx_def const*)
        /repo/gcc-trunk/gcc/print-rtl.cc:938
0x12bf340 rtx_writer::print_rtl_single_with_indent(rtx_def const*, int)
        /repo/gcc-trunk/gcc/print-rtl.cc:1245
0x12bf39f print_rtl_single(_IO_FILE*, rtx_def const*)
        /repo/gcc-trunk/gcc/print-rtl.cc:1228
0xfc2a36 rest_of_clean_state
        /repo/gcc-trunk/gcc/final.cc:4494
0xfc2a36 execute
        /repo/gcc-trunk/gcc/final.cc:4584
Please submit a full bug report, with preprocessed source (by using
-freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.


$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r12-7217-20220213164646-g57da3493970-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=yes,rtl,df,extra
--disable-bootstrap --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r12-7217-20220213164646-g57da3493970-checking-yes-rtl-df-extra-nobootstrap-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.0.1 20220213 (experimental) (GCC)

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode, at real.cc:1770 with -O -fcompare-debug

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2022-02-13
     Ever confirmed|0                           |1
          Component|rtl-optimization            |middle-end
             Status|UNCONFIRMED                 |NEW

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Here is a simplier testcase:
typedef short __attribute__((__vector_size__(16))) V;

_Float64x t1;

void
foo(void)
{
  V t={1024>>1, 0, 0, 0, 32768>>1};
  _Float64x tt = *(_Float64x*)&t;
  t1 /= tt;
}

This ICEs at -O1 -fdump-tree-all-details during forwprop1.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|ICE: in                     |ICE: in
                   |real_to_decimal_for_mode    |real_to_decimal_for_mode
                   |with -O                     |with -O
                   |-fdump-tree-all-details and |-fdump-tree-all-details and
                   |_Float64x                   |long double
      Known to fail|                            |4.4.7

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Here is a better testcase which shows the issue all the way back to at least
4.4.7:
typedef short __attribute__((__vector_size__(16))) V;

long double t1;

void
foo(void)
{
  V t={1024>>1, 0, 0, 0, 32768>>1};
  long double tt = *(long double*)&t;
  t1 /= tt;
}

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
I will have a look.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org,
                   |                            |jsm28 at gcc dot gnu.org

--- Comment #4 from Richard Biener <rguenth at gcc dot gnu.org> ---
So the issue is that native_interpret_real produces something that's not
printable.  For MODE_COMPOSITE_P types we have code that verifies
back-and-forth
conversion - when we enable that we get until RTL which happens to generate

(insn # 0 0 2 (set (reg:XF 8 st)
        (mem/u/c:XF (symbol_ref/u:DI ("*.LC1") [flags 0x2]) [  S16 A128]))
"t.c":9:6# {*movxf_internal}
     (expr_list:REG_EQUIV (const_double:XF

with "invalid" XFmode real again.  Here DSE1 does

 (note 3 0 2 2 [bb 2] NOTE_INSN_BASIC_BLOCK)
-(note 2 3 5 2 NOTE_INSN_FUNCTION_BEG)
-(insn 5 2 6 2 (set (reg:V8HI 51 xmm15)
-        (mem/u/c:V8HI (symbol_ref/u:DI ("*.LC0") [flags 0x2]) [0  S16 A128]))
"t.c":8:15 1694 {movv8hi_internal}
-     (expr_list:REG_EQUAL (const_vector:V8HI [
-                (const_int 512 [0x200])
-                (const_int 0 [0]) repeated x3
-                (const_int 16384 [0x4000])
-                (const_int 0 [0]) repeated x3
-            ])
-        (nil)))
-(insn 6 5 7 2 (set (mem/c:V8HI (plus:DI (reg/f:DI 19 frame)
-                (const_int -16 [0xfffffffffffffff0])) [2  S16 A128])
-        (reg:V8HI 51 xmm15)) "t.c":8:15 1694 {movv8hi_internal}
-     (expr_list:REG_DEAD (reg:V8HI 51 xmm15)
-        (nil)))
-(insn 7 6 8 2 (set (reg:XF 86 [ t1 ])
-        (mem/c:XF (symbol_ref:DI ("t1") [flags 0x2]  <var_decl 0x7ffff6526bd0
t1>) [1 t1+0 S16 A128])) "t.c":9:6 140 {*movxf_internal}
-     (nil))

to

+(note 2 3 13 2 NOTE_INSN_FUNCTION_BEG)
+(insn 13 2 7 2 (set (reg:XF 88)
+        (mem/u/c:XF (symbol_ref/u:DI ("*.LC1") [flags 0x2]) [0  S16 A128]))
"t.c":8:15 140 {*movxf_internal}
+     (expr_list:REG_EQUAL (const_double:XF

so the issue needs to be mitigated in multiple places.  Even if we sanitize
the FP value we'd need to fix the DSE side which invokes
native_decode_rtx which seems to also lack the COMPOSITE_MODE handling.

But I do wonder whether real_from_target needs fixing to handle invalid
input gracefully which is ultimatively decode_ieee_extended?

Any opinions here?

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[ubizjak at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #5 from Uroš Bizjak <ubizjak at gmail dot com> ---
(In reply to Richard Biener from comment #4)
> But I do wonder whether real_from_target needs fixing to handle invalid
> input gracefully which is ultimatively decode_ieee_extended?

long double foo (void)
{
  union { int i[4]; long double ld; } __tmp
    = { .i = { 0x1, 0, 0x4000, 0 } };
  return __tmp.ld;
}

This will fail compilation (-O -dP), without -dP (which avoids printing), the
constant in the memory is wrong:

.LC0:
        .long   1
        .long   0
        .long   0  <--- here should be 16384
        .long   0

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Uroš Bizjak from comment #5)
> (In reply to Richard Biener from comment #4)
> > But I do wonder whether real_from_target needs fixing to handle invalid
> > input gracefully which is ultimatively decode_ieee_extended?
> 
> long double foo (void)
> {
>   union { int i[4]; long double ld; } __tmp
>     = { .i = { 0x1, 0, 0x4000, 0 } };
>   return __tmp.ld;
> }
> 
> This will fail compilation (-O -dP), without -dP (which avoids printing),
> the constant in the memory is wrong:
> 
> .LC0:
>         .long   1
>         .long   0
>         .long   0  <--- here should be 16384
>         .long   0

That's because encode_ieee_extended detects this (.uexp == 2) as denormal
and does

    case rvc_normal:
      {
        int exp = REAL_EXP (r);

        /* Recall that IEEE numbers are interpreted as 1.F x 2**exp,
           whereas the intermediate representation is 0.F x 2**exp.
           Which means we're off by one.

           Except for Motorola, which consider exp=0 and explicit
           integer bit set to continue to be normalized.  In theory
           this discrepancy has been taken care of by the difference
           in fmt->emin in round_for_format.  */

        if (denormal)
          exp = 0;

we do call round_for_format in advance but that does nothing here.

One question would be if native_interpret_real needs to call round_for_format
as well if that's the entity that's supposed to fixup "invalid" FP
encodings.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rguenth at gcc dot gnu.org
             Status|ASSIGNED                    |NEW
           Assignee|rguenth at gcc dot gnu.org         |unassigned at gcc dot gnu.org

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
The MODE_COMPOSITE_P handling can be surely enabled for other modes too, I have
guarded it just because it slows compile time down a little bit and for usual
IEEE754 modes it should be always true, shouldn't it?
If i386/m68k XFmode is a problem too, we'd need some predicate how to find out
it is also problematic (the problem is the unnormals and other weird
representations in it, basically that not all bit patterns are valid).
native_decode_rtx has the same MODE_COMPOSITE_P handling:
  rtx ret = native_decode_rtx (outermode, buffer, 0);
  if (ret && MODE_COMPOSITE_P (outermode))
    {
      auto_vec<target_unit, 128> buffer2 (buffer_bytes);
      if (!native_encode_rtx (outermode, ret, buffer2, 0, buffer_bytes))
        return NULL_RTX;
      for (unsigned int i = 0; i < buffer_bytes; ++i)
        if (buffer[i] != buffer2[i])
          return NULL_RTX;
    }     
so again, it could be enabled for all scalar floating modes or composite and
the XF-ish modes.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> ---
So my question is whether something should normalize those numbers and why
decode_ieee_extended happily initializes an invalid number (its documentation
of course might suggest it only expects valid encoded numbers).

But yes, enabling this re-interpretation checking unconditionally sounds like
the way to go.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #9 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Created attachment 52437
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52437&action=edit
gcc12-pr104522.patch

Patch I'll test tonight.

[Bug middle-end/104522] ICE: in real_to_decimal_for_mode with -O -fdump-tree-all-details and long double

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104522

--- Comment #10 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:2801f23fb82a5ef51c8b460a500786797943e1e9

commit r12-7240-g2801f23fb82a5ef51c8b460a500786797943e1e9
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Tue Feb 15 12:11:31 2022 +0100

    fold, simplify-rtx: Punt on non-representable floating point constants
[PR104522]

    For IBM double double I've added in PR95450 and PR99648 verification that
    when we at the tree/GIMPLE or RTL level interpret target bytes as a
REAL_CST
    or CONST_DOUBLE constant, we try to encode it back to target bytes and
    verify it is the same.
    This is because our real.c support isn't able to represent all valid values
    of IBM double double which has variable precision.
    In PR104522, it has been noted that we have similar problem with the
    Intel/Motorola extended XFmode formats, our internal representation isn't
    able to record pseudo denormals, pseudo infinities, pseudo NaNs and
unnormal
    values.
    So, the following patch is an attempt to extend that verification to all
    floats.
    Unfortunately, it wasn't that straightforward, because the
    __builtin_clear_padding code exactly for the XFmode long doubles needs to
    discover what bits are padding and does that by interpreting memory of
    all 1s.  That is actually a valid supported value, a qNaN with negative
    sign with all mantissa bits set, but the verification includes also the
    padding bits (exactly what __builtin_clear_padding wants to figure out)
    and so fails the comparison check and so we ICE.
    The patch fixes that case by moving that verification from
    native_interpret_real to its caller, so that clear_padding_type can
    call native_interpret_real and avoid that extra check.

    With this, the only thing that regresses in the testsuite is
    +FAIL: gcc.target/i386/auto-init-4.c scan-assembler-times long\\t-16843010
5
    because it decides to use a pattern that has non-zero bits in the padding
    bits of the long double, so the simplify-rtx.cc change prevents folding
    a SUBREG into a constant.  We emit (the testcase is -O0 but we emit worse
    code at all opt levels) something like:
            movabsq $-72340172838076674, %rax
            movabsq $-72340172838076674, %rdx
            movq    %rax, -48(%rbp)
            movq    %rdx, -40(%rbp)
            fldt    -48(%rbp)
            fstpt   -32(%rbp)
    instead of
            fldt    .LC2(%rip)
            fstpt   -32(%rbp)
    ...
    .LC2:
            .long   -16843010
            .long   -16843010
            .long   65278
            .long   0
    Note, neither of those sequences actually stores the padding bits, fstpt
    simply doesn't touch them.
    For vars with clear_padding_real_needs_padding_p types that are allocated
    to memory at expansion time, I'd say much better would be to do the stores
    using integral modes rather than XFmode, so do that:
            movabsq $-72340172838076674, %rax
            movq    %rax, -32(%rbp)
            movq    %rax, -24(%rbp)
    directly.  That is the only way to ensure the padding bits are initialized
    (or expand __builtin_clear_padding, but then you initialize separately the
    value bits and padding bits).

    2022-02-15  Jakub Jelinek  <jakub@redhat.com>

            PR middle-end/104522
            * fold-const.h (native_interpret_real): Declare.
            * fold-const.cc (native_interpret_real): No longer static.  Don't
            perform MODE_COMPOSITE_P verification here.
            (native_interpret_expr) <case REAL_TYPE>: But perform it here
instead
            for all modes.
            * gimple-fold.cc (clear_padding_type): Call native_interpret_real
            instead of native_interpret_expr.
            * simplify-rtx.cc (simplify_immed_subreg): Perform the
native_encode_rtx
            and comparison verification for all FLOAT_MODE_P modes, not just
            MODE_COMPOSITE_P.

            * gcc.dg/pr104522.c: New test.