[Bug rtl-optimization/107772] New: [missed optimization] function prologue generated even though it's only needed in an unlikely path

["avi at scylladb dot com" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107772

            Bug ID: 107772
           Summary: [missed optimization] function prologue generated even
                    though it's only needed in an unlikely path
           Product: gcc
           Version: 13.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: rtl-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: avi at scylladb dot com
  Target Milestone: ---

Consider


int g(int);

void f(int* b, int* e) {
    while (b != e) {
        if (__builtin_expect(*b != 0, false)) [[unlikely]] {
            *b = g(*b);
        }
        ++b;
    }
}

If we believe the __builtin_expect and/or unlikely annotations (had both for
extra safety), the loop usually does nothing. So we would expect any register
saving and restoring to be pushed to the unlikely section. Yet (-O3):

f(int*, int*):
        cmp     rdi, rsi
        je      .L10
        push    rbp
        mov     rbp, rsi
        push    rbx
        mov     rbx, rdi
        sub     rsp, 8
.L4:
        mov     edi, DWORD PTR [rbx]
        test    edi, edi
        jne     .L14
.L3:
        add     rbx, 4
        cmp     rbp, rbx
        jne     .L4
        add     rsp, 8
        pop     rbx
        pop     rbp
        ret
.L14:
        call    g(int)
        mov     DWORD PTR [rbx], eax
        jmp     .L3
.L10:
        ret


I count 8 instructions that could/should have been pushed to .L14.