[Bug tree-optimization/103858] New: [12 Regression] strlen() implementation is optimized into a call to strlen() at -O2, causing infinite recursion

[Bug tree-optimization/103858] New: [12 Regression] strlen() implementation is optimized into a call to strlen() at -O2, causing infinite recursion

[dani at danielbertalan dot dev]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103858

            Bug ID: 103858
           Summary: [12 Regression] strlen() implementation is optimized
                    into a call to strlen() at -O2, causing infinite
                    recursion
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: dani at danielbertalan dot dev
  Target Milestone: ---

Compiler Explorer link: https://godbolt.org/z/h419G55P4

The following function is folded into a call to strlen() by gcc 12 (commit
61e53698a08dc1d9a54d785218af687a6751c1b3) at -O2, unless -fno-builtin-strlen is
specified. This function is taken from a (naive) libc implementation, where
this is the only definition of strlen(), so it ends up calling itself
recursively, overflowing the stack.

#include <stddef.h>

size_t strlen(const char* str)
{
    size_t len = 0;
    while (*(str++))
        ++len;
    return len;
}

x86_64 assembly output:
strlen:
        cmp     BYTE PTR [rdi], 0
        je      .L3
        lea     rax, [rdi+1]
        sub     rsp, 8
        mov     rdi, rax
        call    strlen
        add     rsp, 8
        add     rax, 1
        ret
.L3:
        xor     eax, eax
        ret

My workaround is to compile our libc with -ffreestanding/-fno-builtin, but I'd
like to avoid that, as it might prevent other (valid) optimizations from being
performed on our code.

[Bug tree-optimization/103858] [12 Regression] strlen() implementation is optimized into a call to strlen() at -O2, causing infinite recursion

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103858

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |marxin at gcc dot gnu.org
         Resolution|---                         |DUPLICATE
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Dup.

*** This bug has been marked as a duplicate of bug 102725 ***

[Bug tree-optimization/103858] [12 Regression] strlen() implementation is optimized into a call to strlen() at -O2, causing infinite recursion

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103858

Martin Sebor <msebor at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |msebor at gcc dot gnu.org

--- Comment #2 from Martin Sebor <msebor at gcc dot gnu.org> ---
As an aside, the infinite recursion could (and in the future, should) be
detected by -Winfinite-recursion.  It was mentioned as a possible future
enhancement for the warning at the time it was submitted for review
(https://gcc.gnu.org/pipermail/gcc-patches/2021-November/584205.html).  Here's
a simpler test case where it already is detected.  Detecting the problem in the
loopy case in comment #0 requires running that subset of the detection later
(e.g., just before expansion).

$ cat pr103858.c && gcc -S -Wall -fdump-tree-einline=/dev/stdout pr103858.c
#include <stddef.h>

size_t strlen(const char* str)
{
  size_t len = __builtin_strlen (str);
  return len;
}

;; Function strlen (strlen, funcdef_no=0, decl_uid=955, cgraph_uid=1,
symbol_order=0)

__attribute__((nothrow, leaf, pure))
__attribute__((nonnull))
size_t strlen (const char * str)
{
  size_t len;
  size_t D.1990;
  size_t _4;

  <bb 2> :
  len_3 = __builtin_strlen (str_2(D));
  _4 = len_3;

  <bb 3> :
<L0>:
  return _4;

}


pr103858.c: In function ‘strlen’:
pr103858.c:3:8: warning: infinite recursion detected [-Winfinite-recursion]
    3 | size_t strlen(const char* str)
      |        ^~~~~~
pr103858.c:5:16: note: recursive call
    5 |   size_t len = __builtin_strlen (str);
      |                ^~~~~~~~~~~~~~~~~~~~~~