[Bug lto/99828] New: inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[Bug lto/99828] New: inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

            Bug ID: 99828
           Summary: inlining failed in call to ‘always_inline’ ‘memcpy’:
                    --param max-inline-insns-auto limit reached
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
          Assignee: unassigned at gcc dot gnu.org
          Reporter: marxin at gcc dot gnu.org
                CC: hubicka at gcc dot gnu.org, marxin at gcc dot gnu.org
  Target Milestone: ---

Noticed by Andi Kleen in kernel, reduced to:

$ cat 1.i
__attribute__((__always_inline__)) void *memcpy();
void *foo = memcpy;

$ touch 2.s
$ cat 3.i
int kasan_check_range();
int __memcpy();

enum { false, true }
memcpy(void *dest, void *src, long len) {
  if (kasan_check_range(len, false, 0) || kasan_check_range(len, true, 0))
    return __memcpy(dest, src, len);
}

int *LZ4_decompress_generic_outputSize_op;
int mlen;

LZ4_decompress_generic_outputSize_match() {
  {
    __builtin_memcpy(LZ4_decompress_generic_outputSize_op,
                     LZ4_decompress_generic_outputSize_match, mlen);
  }
}

$ gcc 1.i 2.s 3.i -c -flto -Os -shared && gcc [123].o -shared
1.i:1:42: warning: ‘always_inline’ function might not be inlinable
[-Wattributes]
    1 | __attribute__((__always_inline__)) void *memcpy();
      |                                          ^~~~~~
3.i:5:1: warning: conflicting types for built-in function ‘memcpy’; expected
‘void *(void *, const void *, long unsigned int)’
[-Wbuiltin-declaration-mismatch]
    5 | memcpy(void *dest, void *src, long len) {
      | ^~~~~~
3.i:1:1: note: ‘memcpy’ is declared in header ‘<string.h>’
  +++ |+#include <string.h>
    1 | int kasan_check_range();
3.i:13:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
   13 | LZ4_decompress_generic_outputSize_match() {
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1.i:1:42: warning: type of ‘memcpy’ does not match original declaration
[-Wlto-type-mismatch]
    1 | __attribute__((__always_inline__)) void *memcpy();
      |                                          ^
3.i:5:1: note: return value type mismatch
    5 | memcpy(void *dest, void *src, long len) {
      | ^
<built-in>: warning: type of ‘__builtin_memcpy’ does not match original
declaration [-Wlto-type-mismatch]
3.i:5:1: note: return value type mismatch
3.i:5:1: note: ‘memcpy’ was previously declared here
3.i:5:1: note: code may be misoptimized unless ‘-fno-strict-aliasing’ is used
3.i: In function ‘LZ4_decompress_generic_outputSize_match’:
1.i:1:42: error: inlining failed in call to ‘always_inline’ ‘memcpy’: --param
max-inline-insns-auto limit reached
    1 | __attribute__((__always_inline__)) void *memcpy();
      |                                          ^
3.i:15:5: note: called from here
   15 |     __builtin_memcpy(LZ4_decompress_generic_outputSize_op,
      |     ^
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |diagnostic,
                   |                            |missed-optimization

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
The issue is that we do not "merge" the attributes and the memcpy
definition lacks the always_inline attribute, somehow
expand_call_inline then picks up the decl with the attribute set.

I'd argue the 1.i TU is clearly invalid in taking the address of memcpy,
but we don't diagnose this.

How we manage to pick up the attribute in 3.i during WPA / LTRANS remains to be
analyzed.  But the "fix" is in the kernel.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2021-03-30
             Status|UNCONFIRMED                 |NEW
                 CC|                            |rguenth at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Btw, one solution would be to drop __always_inline__ after always-inline
inlining
and thus make it reliably not present for IPA inlining.

__always_inline__ is a decl attribute and thus per call-stmt processing via
gimple_call_fntype doesn't work but instead this is subject to WPA cgraph
merging (I guess IPA inline still sees the unmerged cgraph).

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[andi-gcc at firstfloor dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #3 from Andi Kleen <andi-gcc at firstfloor dot org> ---
So what do you want to fix in the kernel? 

Use a wrapper for taking the address of the memcpy?
(I hope nothing in gcc would remove such a wrapper)

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenther at suse dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #4 from rguenther at suse dot de <rguenther at suse dot de> ---
On March 30, 2021 7:44:56 PM GMT+02:00, andi-gcc at firstfloor dot org
<gcc-bugzilla@gcc.gnu.org> wrote:
>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828
>
>--- Comment #3 from Andi Kleen <andi-gcc at firstfloor dot org> ---
>So what do you want to fix in the kernel? 
>
>Use a wrapper for taking the address of the memcpy?
>(I hope nothing in gcc would remove such a wrapper)

Remove the always_inline or also place it on the definition.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #5 from Jan Hubicka <hubicka at ucw dot cz> ---
> Btw, one solution would be to drop __always_inline__ after always-inline
> inlining
> and thus make it reliably not present for IPA inlining.
Removing it would make you to lose those errors, but we can ignore it
for late inlining if we decide we do not really care about always
inlining indirect calls (that are not reliably inlined by early
inliner).

But I tried that at some point and broke kernel.

Note that we could also use syntactic aliase and consider two decls
unmergeable if they differ by always_inline attribute.  That should make
it to behave consistently...

Honza

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #6 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Jan Hubicka from comment #5)
> > Btw, one solution would be to drop __always_inline__ after always-inline
> > inlining
> > and thus make it reliably not present for IPA inlining.
> Removing it would make you to lose those errors, but we can ignore it
> for late inlining if we decide we do not really care about always
> inlining indirect calls (that are not reliably inlined by early
> inliner).
> 
> But I tried that at some point and broke kernel.
> 
> Note that we could also use syntactic aliase and consider two decls
> unmergeable if they differ by always_inline attribute.  That should make
> it to behave consistently...

Yeah, and then maybe diagnose this "ODR violation".  Still

__attribute__((__always_inline__)) void *memcpy();
void *foo = memcpy;

should be ill-formed (but yeah, maybe this ship has sailed...).

Now, I do wonder why during cgraph merging we prefer the non-definition
declaration ... the code "works fine" if it's not memcpy but memcpyx
(and thus not __builtin_memcpy but also memcpyx).

I also wonder if we can get a better reduction of the kernel problem
due to all the diagnostics I get:

1.i:1:42: warning: 'always_inline' function might not be inlinable
[-Wattribute]
    1 | __attribute__((__always_inline__)) void *memcpy();
      |                                          ^~~~~~
3.i:5:1: warning: conflicting types for built-in function 'memcpy'; expected
'void *(void *, const void *, long unsigned int)'
[-Wbuiltin-declaration-mismatch]
    5 | memcpy(void *dest, void *src, long len) {
      | ^~~~~~
1.i:1:42: warning: type of 'memcpy' does not match original declaration
[-Wlto-type-mismatch]
    1 | __attribute__((__always_inline__)) void *memcpy();
      |                                          ^

...

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[hubicka at ucw dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #7 from Jan Hubicka <hubicka at ucw dot cz> ---
> Yeah, and then maybe diagnose this "ODR violation".  Still

I think we do have this kinds of divergence (like glibcs
fortification), so I am not sure we want to warn by default.
> 
> __attribute__((__always_inline__)) void *memcpy();
> void *foo = memcpy;
> 
> should be ill-formed (but yeah, maybe this ship has sailed...).

Yep. We took wrong move calling alwys_inline always_inline at the time
it really meant disregard_inline_limits (that is correclty named
internally) and then giving it the semantics of really always inlining.
> 
> Now, I do wonder why during cgraph merging we prefer the non-definition
> declaration ... the code "works fine" if it's not memcpy but memcpyx
> (and thus not __builtin_memcpy but also memcpyx).

Hmm, we follow resolution file here and linker doesn't know about the
attributes.  I guess disabling merging here is something to do
(it was alwyas on my TODO list but not very high).  We eventualy ought
to also support keeping multiple inline bodies for different decls we
decide to not merge that is bit harder to get right.
> 
> I also wonder if we can get a better reduction of the kernel problem
> due to all the diagnostics I get:
> 
> 1.i:1:42: warning: 'always_inline' function might not be inlinable
> [-Wattribute]
>     1 | __attribute__((__always_inline__)) void *memcpy();
>       |                                          ^~~~~~
> 3.i:5:1: warning: conflicting types for built-in function 'memcpy'; expected
> 'void *(void *, const void *, long unsigned int)'
> [-Wbuiltin-declaration-mismatch]
>     5 | memcpy(void *dest, void *src, long len) {
>       | ^~~~~~
> 1.i:1:42: warning: type of 'memcpy' does not match original declaration
> [-Wlto-type-mismatch]
>     1 | __attribute__((__always_inline__)) void *memcpy();
>       |                                          ^

Hmm these warnings come from different places but indeed it is bit too
much :)

Honza

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #8 from Martin Liška <marxin at gcc dot gnu.org> ---
All right, I vanished the test-case:

$ cat 1.i
inline __attribute__((__always_inline__)) __attribute__((gnu_inline)) void *
memcpy();
void *apply_relocate_add_write = memcpy;

$ touch 2.s
$ cat 3.i
enum { false, true } * __memcpy();
_Bool kasan_check_range();
void *memcpy(void *dest, void *src, long len) {
  if (kasan_check_range(len, false, 0) || kasan_check_range(len, true, 0))
    return __memcpy(dest, src, len);
}

long LZ4_decompress_generic_dst_restSize;
char LZ4_decompress_generic_dst_lowPrefix;
void LZ4_decompress_generic_dst() {
  __builtin_memcpy(LZ4_decompress_generic_dst,
                   &LZ4_decompress_generic_dst_lowPrefix,
                   LZ4_decompress_generic_dst_restSize);
}

$ gcc 1.i -c -flto && gcc 3.i -c -flto -Os -fno-builtin && gcc -r [13].o 2.s
/usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld:
warning: incremental linking of LTO and non-LTO objects; using
-flinker-output=nolto-rel which will bypass whole program optimization
3.i: In function ‘LZ4_decompress_generic_dst’:
1.i:2:1: error: inlining failed in call to ‘always_inline’ ‘memcpy’: --param
max-inline-insns-auto limit reached
    2 | memcpy();
      | ^
3.i:11:3: note: called from here
   11 |   __builtin_memcpy(LZ4_decompress_generic_dst,
      |   ^
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld:
error: lto-wrapper failed
collect2: error: ld returned 1 exit status

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenther at suse dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #9 from rguenther at suse dot de <rguenther at suse dot de> ---
On Wed, 31 Mar 2021, marxin at gcc dot gnu.org wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828
> 
> --- Comment #8 from Martin Liška <marxin at gcc dot gnu.org> ---
> All right, I vanished the test-case:
> 
> $ cat 1.i
> inline __attribute__((__always_inline__)) __attribute__((gnu_inline)) void *
> memcpy();
> void *apply_relocate_add_write = memcpy;
> 
> $ touch 2.s
> $ cat 3.i
> enum { false, true } * __memcpy();

??  obviously bad reduction.

> _Bool kasan_check_range();
> void *memcpy(void *dest, void *src, long len) {
>   if (kasan_check_range(len, false, 0) || kasan_check_range(len, true, 0))
>     return __memcpy(dest, src, len);
> }
> 
> long LZ4_decompress_generic_dst_restSize;
> char LZ4_decompress_generic_dst_lowPrefix;
> void LZ4_decompress_generic_dst() {
>   __builtin_memcpy(LZ4_decompress_generic_dst,
>                    &LZ4_decompress_generic_dst_lowPrefix,
>                    LZ4_decompress_generic_dst_restSize);

I wonder if this use of __builtin_memcpy intends to not use the
kernels always-inline memcpy but GCCs own inline expansion?
This obviously doesn't work, not with LTO at least.

It looks like with kasan enabled (and memcpy "wrapped") the
memcpy declaration should _not_ have the always-inline (since
the implementation is no longer always-inline).  That would be
a fix on the kernel side, but I'd also diagnose any such
always-inline "mismatch" we get to at WPA.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #10 from Martin Liška <marxin at gcc dot gnu.org> ---
> ??  obviously bad reduction.

Fixed with:

$ cat 3.i
enum { false, true } * __memcpy();
_Bool kasan_check_range();
void *memcpy(void *dest, void *src, long len) {
  if (kasan_check_range(len, false, 0) || kasan_check_range(len, true, 0))
    return __memcpy(dest, src, len);
}

char LZ4_decompress_safe_forceExtDict_lowPrefix,
    LZ4_decompress_safe_forceExtDict_op;
long LZ4_decompress_safe_forceExtDict_restSize;
void LZ4_decompress_safe_forceExtDict() {
  __builtin_memcpy(&LZ4_decompress_safe_forceExtDict_op,
                   &LZ4_decompress_safe_forceExtDict_lowPrefix,
                   LZ4_decompress_safe_forceExtDict_restSize);
}

$ gcc -r [123].o
/usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld:
warning: incremental linking of LTO and non-LTO objects; using
-flinker-output=nolto-rel which will bypass whole program optimization
3.i: In function ‘LZ4_decompress_safe_forceExtDict’:
1.i:2:1: error: inlining failed in call to ‘always_inline’ ‘memcpy’: --param
max-inline-insns-auto limit reached
    2 | memcpy();
      | ^
3.i:12:3: note: called from here
   12 |   __builtin_memcpy(&LZ4_decompress_safe_forceExtDict_op,
      |   ^
lto-wrapper: fatal error: gcc returned 1 exit status
compilation terminated.
/usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld:
error: lto-wrapper failed

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[jirislaby at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

Jiri Slaby <jirislaby at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jirislaby at gmail dot com

--- Comment #11 from Jiri Slaby <jirislaby at gmail dot com> ---
Looking at the kernel patch, it says:
    This is fixed now in gcc, but work around it on older compilers
    by using a wrapper.

If so, how was this fixed an when (in what version)?

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[jirislaby at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #12 from Jiri Slaby <jirislaby at gmail dot com> ---
(In reply to Jiri Slaby from comment #11)
> Looking at the kernel patch, it says:
>     This is fixed now in gcc, but work around it on older compilers
>     by using a wrapper.
> 
> If so, how was this fixed an when (in what version)?

FWIW it builds with the patch reverted using gcc 12.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
      Known to fail|                            |12.2.0
   Last reconfirmed|2021-03-30 00:00:00         |2022-9-22

--- Comment #13 from Richard Biener <rguenth at gcc dot gnu.org> ---
The testcase still does not work on master or with 12.2, thus reconfirmed.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[jirislaby at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #14 from Jiri Slaby <jirislaby at gmail dot com> ---
(In reply to Richard Biener from comment #13)
> The testcase still does not work on master or with 12.2, thus reconfirmed.

Hmm:

(In reply to Martin Liška from comment #0)
> Noticed by Andi Kleen in kernel, reduced to:
> 
> $ cat 1.i
> __attribute__((__always_inline__)) void *memcpy();
> void *foo = memcpy;

Provided I cannot reproduce on the current kernel, where exactly does this come
from?

I see:

arch/x86/include/asm/string_64.h:
#define __HAVE_ARCH_MEMCPY 1
extern void *memcpy(void *to, const void *from, size_t len);
extern void *__memcpy(void *to, const void *from, size_t len);

===

For KASAN also:
#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)

#undef memcpy
#define memcpy(dst, src, len) __memcpy(dst, src, len)

===

arch/x86/lib/memcpy_64.S:
SYM_FUNC_START(__memcpy)
...
SYM_FUNC_END(__memcpy)
...
SYM_FUNC_ALIAS_WEAK(memcpy, __memcpy)

===

$ nm ../lto/vmlinux|grep -wE '__memcpy|memcpy'
ffffffff81dcf0b0 T __memcpy
ffffffff81dcf0b0 W memcpy

===

arch/x86/lib/memcpy_32.c:
__used __visible void *memcpy(void *to, const void *from, size_t n)
{
        return __memcpy(to, from, n);
}

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[andi at firstfloor dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #15 from andi at firstfloor dot org ---
> Provided I cannot reproduce on the current kernel, where exactly does this come
> from?

Usually I had to do a longer loop of randconfig builds to find it. It only
happens in some specific configs.

[Bug lto/99828] inlining failed in call to ‘always_inline’ ‘memcpy’: --param max-inline-insns-auto limit reached

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99828

--- Comment #16 from Martin Liška <marxin at gcc dot gnu.org> ---
(In reply to andi from comment #15)
> > Provided I cannot reproduce on the current kernel, where exactly does this come
> > from?
> 
> Usually I had to do a longer loop of randconfig builds to find it. It only
> happens in some specific configs.

I've just built 300 randconfigs w/ LTO and can't reproduce this issue.