[Bug tree-optimization/106960] New: [12/13 Regression] Incorrect optimization of signed integer comparisons

[Bug tree-optimization/106960] New: [12/13 Regression] Incorrect optimization of signed integer comparisons

[danglin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106960

            Bug ID: 106960
           Summary: [12/13 Regression] Incorrect optimization of signed
                    integer comparisons
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: danglin at gcc dot gnu.org
                CC: deller at gmx dot de
  Target Milestone: ---
              Host: hppa*-*-linux*
            Target: hppa*-*-linux*
             Build: hppa*-*-linux*

This problem was noticed in the Debian flint 2.9.0-5 testsuite.  The
fmpq/cmp_si.c testcase fails with a segmentation fault becaused of an
incorrect optimization.

Here is a reduced testcase:

dave@atlas:~/gnu/gcc/objdir$ cat foo.c
int bar (long);
int _fmpq_cmp_si (long c)
{
  long d;

  d = -c;
  if (d != c) /* Check for LONG_MIN */
    {
      d = c < 0 ? -c : c;
      return bar (d);
    }
  return 0;
}

When compiled at -O1 and above, the comparison only checks for 0 and not
for LONG_MIN (see movb instruction):

dave@atlas:~/gnu/gcc/objdir$ cat foo.s
        .LEVEL 1.1
        .text
        .align 4
.globl _fmpq_cmp_si
        .type   _fmpq_cmp_si, @function
_fmpq_cmp_si:
        .PROC
        .CALLINFO FRAME=64,CALLS,SAVE_RP
        .ENTRY
        stw %r2,-20(%r30)
        ldo 64(%r30),%r30
        movb,<>,n %r26,%r28,.L3
        ldw -84(%r30),%r2
.L4:
        bv %r0(%r2)
        ldo -64(%r30),%r30
.L3:
        or,>= %r0,%r28,%r26
        subi 0,%r26,%r26
        bl bar,%r2
        nop
        b .L4
        ldw -84(%r30),%r2
        .EXIT
        .PROCEND
        .size   _fmpq_cmp_si, .-_fmpq_cmp_si
        .ident  "GCC: (Debian 12.2.0-2) 12.2.0"

The movb instruction copies %r26 (c) to %r28 (the return value) and
branches to .L3 if the register value is nonzero.

We have in foo.c.253r.expand:

;;
;; Full RTL generated for this function:
;;
(note 1 0 4 NOTE_INSN_DELETED)
;; basic block 2, loop depth 0, count 1073741824 (estimated locally), maybe hot
;;  prev block 0, next block 4, flags: (NEW, REACHABLE, RTL, VISITED)
;;  pred:       ENTRY [always]  count:1073741824 (estimated locally) (FALLTHRU)
(note 4 1 2 2 [bb 2] NOTE_INSN_BASIC_BLOCK)
(insn 2 4 3 2 (set (reg/v:SI 98 [ c ])
        (reg:SI 26 %r26 [ c ])) "foo.c":3:1 -1
     (expr_list:REG_EQUIV (mem/c:SI (plus:SI (reg/f:SI 90
virtual-incoming-args)
                (const_int -4 [0xfffffffffffffffc])) [1 c+0 S4 A32])
        (nil)))
(note 3 2 6 2 NOTE_INSN_FUNCTION_BEG)
(jump_insn 6 3 7 2 (set (pc)
        (if_then_else (eq (reg/v:SI 98 [ c ])
                (const_int 0 [0]))
            (label_ref 12)
            (pc))) "foo.c":7:6 -1
     (int_list:REG_BR_PROB 856416484 (nil))
 -> 12)
;;  succ:       4 [20.2% (guessed)]  count:217325344 (estimated locally)
(FALLTHRU)
;;              5 [79.8% (guessed)]  count:856416480 (estimated locally)

;; basic block 4, loop depth 0, count 217325344 (estimated locally), maybe hot
;;  prev block 2, next block 5, flags: (NEW, REACHABLE, RTL, VISITED)
;;  pred:       2 [20.2% (guessed)]  count:217325344 (estimated locally)
(FALLTHRU)
(note 7 6 8 4 [bb 4] NOTE_INSN_BASIC_BLOCK)
(insn 8 7 9 4 (set (reg:SI 99 [ d ])
        (abs:SI (reg/v:SI 98 [ c ]))) "foo.c":9:9 -1
     (nil))
(insn 9 8 10 4 (set (reg:SI 26 %r26)
        (reg:SI 99 [ d ])) "foo.c":10:14 -1
     (nil))
(call_insn 10 9 11 4 (parallel [
            (set (reg:SI 28 %r28)
                (call (mem:SI (symbol_ref/v:SI ("@bar") [flags 0x41] 
<function_decl 0xf7821b80 bar>) [0 bar S4 A32])
                    (const_int 16 [0x10])))
            (clobber (reg:SI 1 %r1))
            (clobber (reg:SI 2 %r2))
            (use (const_int 0 [0]))
        ]) "foo.c":10:14 -1
     (nil)
    (expr_list:SI (use (reg:SI 26 %r26))
        (nil)))
(insn 11 10 12 4 (set (reg/v:SI 98 [ c ])
        (reg:SI 28 %r28)) "foo.c":10:14 -1
     (nil))
;;  succ:       5 [always]  count:217325344 (estimated locally) (FALLTHRU)
foo.c:10:14

;; basic block 5, loop depth 0, count 1073741824 (estimated locally), maybe hot
;;  prev block 4, next block 1, flags: (NEW, REACHABLE, RTL, VISITED)
;;  pred:       4 [always]  count:217325344 (estimated locally) (FALLTHRU)
foo.c:10:14
;;              2 [79.8% (guessed)]  count:856416480 (estimated locally)
(code_label 12 11 13 5 2 (nil) [1 uses])
(note 13 12 14 5 [bb 5] NOTE_INSN_BASIC_BLOCK)
(insn 14 13 18 5 (set (reg:SI 97 [ <retval> ])
        (reg/v:SI 98 [ c ])) "foo.c":13:1 -1
     (nil))
(insn 18 14 19 5 (set (reg/i:SI 28 %r28)
        (reg:SI 97 [ <retval> ])) "foo.c":13:1 -1
     (nil))
(insn 19 18 0 5 (use (reg/i:SI 28 %r28)) "foo.c":13:1 -1
     (nil))
;;  succ:       EXIT [always]  count:1073741824 (estimated locally) (FALLTHRU)

So, this appears to be a tree optimization bug.

This occurs with Debian 12.2.0-2 and trunk.  Debian 11.3.0-6 is okay.

[Bug tree-optimization/106960] [12/13 Regression] Incorrect optimization of signed integer comparisons

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106960

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Signed integer overflow is undefined behavior.
That is -INT_MIN is undefined behavior.
You need to either use -fwrapv or change the code to use unsigned types to get
defined behavior.