[Bug c/112486] New: GCC: 14: hangs with always_inline

[Bug c/112486] New: GCC: 14: hangs with always_inline

[141242068 at smail dot nju.edu.cn]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112486

            Bug ID: 112486
           Summary: GCC: 14: hangs with always_inline
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: 141242068 at smail dot nju.edu.cn
  Target Milestone: ---

Compiler Explorer: https://gcc.godbolt.org/z/3n34j7fPT

When try to compile this program with `gcc-14 -O2`:
```
unsigned long flg = 0;

int n_1 = {0};
__attribute__((always_inline))
long sub(unsigned long n) {
  int a, b;
  if (n_1 >= 2) {
    if (n_1 % 2 == 0) {
      a = sub(n_1 / 2);
      { return (a & 2 >> sub(n_1 / 2 - 1)) * a; };
    } else {
      a = sub(n_1 / 2 + 1);
      b = sub(n_1 / 2);
      return (sub(n_1 / 2)) * a + b * b;
    }
  } else
    return (long)n_1;
}

int main(void) {
  if (sub(30) != 832040L) flg |= 0x100L;
  if (flg) abort();
  exit(0);
}
```

GCC 14 hangs with a lot of output, according to the output, it seems GCC tries
to recursively expand all invocation to `sub`.

The partial output pasted from compiler explorer:
```
In function 'sub',
    inlined from 'sub' at <source>:13:11,
    inlined from 'sub' at <source>:12:11,
    inlined from 'sub' at <source>:13:11,
    inlined from 'sub' at <source>:10:26,
    inlined from 'sub' at <source>:10:26,
    inlined from 'sub' at <source>:10:26,
    inlined from 'sub' at <source>:10:26,
    inlined from 'sub' at <source>:9:11,
    inlined from 'sub' at <source>:9:11,
    inlined from 'sub' at <source>:12:11,
    inlined from 'sub' at <source>:12:11,
    inlined from 'main' at <source>:21:7:
<source>:5:6: error: inlining failed in call to 'always_inline' 'sub':
recursive inlining
    5 | long sub(unsigned long n) {
      |      ^~~
<source>:10:26: note: called from here
   10 |       { return (a & 2 >> sub(n_1 / 2 - 1)) * a; };
      |                          ^~~~~~~~~~~~~~~~
<source>:5:6: error: inlining failed in call to 'always_inline' 'sub':
recursive inlining
    5 | long sub(unsigned long n) {
      |      ^~~
<source>:9:11: note: called from here
    9 |       a = sub(n_1 / 2);
      |           ^~~~~~~~~~~~
<source>:5:6: error: inlining failed in call to 'always_inline' 'sub':
recursive inlining
    5 | long sub(unsigned long n) {
      |      ^~~
<source>:13:11: note: called from here
   13 |       b = sub(n_1 / 2);
      |           ^~~~~~~~~~~~
<source>:5:6: error: inlining failed in call to 'always_inline' 'sub':
recursive inlining
    5 | long sub(unsigned long n) {
      |      ^~~
<source>:12:11: note: called from here
   12 |       a = sub(n_1 / 2 + 1);
      |           ^~~~~~~~~~~~~~~~
In function 'sub',
    inlined from 'sub' at <source>:13:11,
    inlined from 'sub' at <source>:13:11,
    inlined from 'sub' at <source>:10:26,
    inlined from 'sub'
[Truncated]
Compiler returned: 143
```

[Bug ipa/112486] GCC: 14: hangs with always_inline

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112486

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|c                           |ipa
                 CC|                            |marxin at gcc dot gnu.org

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I don't think this is exactly a bug. Gcc recursively inlines sub as much as it
can and then it starts to bails out and since there are many calls of sub that
it is failing at it takes for every to outputs error messages.

[Bug ipa/112486] GCC: 14: hangs with always_inline

[141242068 at smail dot nju.edu.cn]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112486

wierton <141242068 at smail dot nju.edu.cn> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |INVALID

--- Comment #2 from wierton <141242068 at smail dot nju.edu.cn> ---
Thank you for your reply! I will close this issue.

[Bug ipa/112486] GCC: 14: hangs with always_inline

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112486

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #3 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
FWIW you can use "-fmax-error=256" to limit the amount of error messages.

But generally if you want to compile some potentially malicious source code,
you still need to run GCC in a sandbox with an output limit (along with other
resource limits) as GCC security policy explicitly notes.