[Bug inline-asm/101438] New: Compiler hang on inline asm with local register and VLA operands

[Bug inline-asm/101438] New: Compiler hang on inline asm with local register and VLA operands

[andrey.vihrov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

            Bug ID: 101438
           Summary: Compiler hang on inline asm with local register and
                    VLA operands
           Product: gcc
           Version: 11.1.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: inline-asm
          Assignee: unassigned at gcc dot gnu.org
          Reporter: andrey.vihrov at gmail dot com
  Target Milestone: ---
            Target: x86_64-pc-linux-gnu

Compiling this C source:

  int main(void)
  {
    register long long rax __asm__("rax") = 0;
    register long long rbx __asm__("rbx") = 0;
    register long long rcx __asm__("rcx") = 0;
    register long long rdx __asm__("rdx") = 0;
    register long long rsi __asm__("rsi") = 0;
    register long long rdi __asm__("rdi") = 0;
    register long long r8  __asm__("r8")  = 0;
    register long long r9  __asm__("r9")  = 0;
    register long long r10 __asm__("r10") = 0;
    register long long r11 __asm__("r11") = 0;
    register long long r12 __asm__("r12") = 0;
    register long long r13 __asm__("r13") = 0;
    register long long r14 __asm__("r14") = 0;
    register long long r15 __asm__("r15") = 0;

    volatile int x = 1;
    char arr[x];

    __asm__ (""
      :
      : "r"(rax),
        "r"(rbx),
        "r"(rcx),
        "r"(rdx),
        "r"(rsi),
        "r"(rdi),
        "r"(r8),
        "r"(r9),
        "r"(r10),
        "r"(r11),
        "r"(r12),
        "r"(r13),
        "r"(r14),
        "r"(r15),
        "m"(arr)
    );
  }

with "gcc -O1 test.c" causes the compiler to hang in an infinite loop.

Note: This code could possibly be formally incorrect, as [1] warns about
potential clobbering after initialization of local register variables. 

[1] https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html

[Bug inline-asm/101438] Compiler hang on inline asm with local register and VLA operands

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |accepts-invalid,
                   |                            |compile-time-hog, ra
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2021-07-14
     Ever confirmed|0                           |1

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
At -O0 we get

t3.c: In function 'main':
t3.c:21:5: error: 'asm' operand has impossible constraints
   21 |     __asm__ (""
      |     ^~~~~~~

likely because there's not enough registers to pass in 'arr' (and we need a FP
to allocate arr).  At -O1 it looks like we think we can manage but do not make
progress reloading.

So confirmed, a diagnostic would be nice.

[Bug inline-asm/101438] Compiler hang on inline asm with local register and VLA operands

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
This code is almost definitely invalid:
"m"(arr)

You are asigning the array/pointer that the VLA is.
This is very much related to PR 71572 for similar reasons.

[Bug inline-asm/101438] Compiler hang on inline asm with local register and VLA operands

[andrey.vihrov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

--- Comment #3 from Andrey Vihrov <andrey.vihrov at gmail dot com> ---
Created attachment 51152
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51152&action=edit
Alternative testcase using __builtin_alloca()

Thanks.

This code is the result of minimization. It seems that it is sufficient to make
inline asm depend on arr in any form.

"m"(arr[0]) (and initializing the array element) reproduces the issue as well.

I've also attached a similar testcase that compiles and runs at -O0, but hangs
the compiler at -O1 (and which likewise references the pointer itself with
"m").

[Bug target/101438] Compiler hang on inline asm with local register and alloca

[andrey.vihrov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

--- Comment #4 from Andrey Vihrov <andrey.vihrov at gmail dot com> ---
Looks like this is a duplicate of bug 105032 (or vice versa).

Both samples from this bug succeed for me with GCC 11.3 and GCC 12.

[Bug target/101438] Compiler hang on inline asm with local register and alloca

[andrey.vihrov at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101438

Andrey Vihrov <andrey.vihrov at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #5 from Andrey Vihrov <andrey.vihrov at gmail dot com> ---
Closing as a duplicate.

*** This bug has been marked as a duplicate of bug 105032 ***