[Bug target/115054] New: __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[Bug target/115054] New: __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[tmgross at umich dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

            Bug ID: 115054
           Summary: __float128 and _Float16 use incorrect ABI on x86-64
                    MinGW
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: tmgross at umich dot edu
  Target Milestone: ---

GCC appears to try to pass half and quad-precision floating point numbers
(_Float16 and __float128) using the integer calling convention on x86-64 MinGW.
They should get passed with the floating point CC in SSE registers, like other
floats, as is done by GCC x86-64 linux, Clang x86-64 linux, and Clang x86-64
MinGW.

Sample:

    void f16_ext(_Float16);
    void f16_entry(float _, _Float16 a) {
        asm("nop # marker");
        f16_ext(a);
    }

    void f32_ext(float);
    void f32_entry(float _, float a) {
        asm("nop # marker");
        f32_ext(a);
    }

    void f64_ext(double);
    void f64_entry(float _, double a) {
        asm("nop # marker");
        f64_ext(a);
    }

    void f128_ext(__float128);
    void f128_entry(float _, __float128 a) {
        asm("nop # marker");
        f128_ext(a);
    }

Incorrect output from GCC on x64 MinGW (O2):

    f16_entry:
        mov     ecx, edx
        nop # marker
        jmp     f16_ext
    f32_entry:
        movaps  xmm0, xmm1
        nop # marker
        jmp     f32_ext
    f64_entry:
        movapd  xmm0, xmm1
        nop # marker
        jmp     f64_ext
    f128_entry:
        sub     rsp, 56
        movdqa  xmm0, XMMWORD PTR [rdx]
        nop # marker
        lea     rcx, 32[rsp]
        movaps  XMMWORD PTR 32[rsp], xmm0
        call    f128_ext
        nop
        add     rsp, 56
        ret

Correct output from GCC on x64 Linux (O2):

    f16_entry:
        movaps  xmm0, xmm1
        nop # marker
        jmp     f16_ext
    f32_entry:
        movaps  xmm0, xmm1
        nop # marker
        jmp     f32_ext
    f64_entry:
        movapd  xmm0, xmm1
        nop # marker
        jmp     f64_ext
    f128_entry:
        movdqa  xmm0, xmm1
        nop # marker
        jmp     f128_ext

Correct output from Clang on both x64 Linux and x64 MinGW:

    f16_entry:                              # @f16_entry
        movaps  xmm0, xmm1
        nop     # marker
        jmp     f16_ext                         # TAILCALL
    f32_entry:                              # @f32_entry
        movaps  xmm0, xmm1
        nop     # marker
        jmp     f32_ext                         # TAILCALL
    f64_entry:                              # @f64_entry
        movaps  xmm0, xmm1
        nop     # marker
        jmp     f64_ext                         # TAILCALL
    f128_entry:                             # @f128_entry
        movaps  xmm0, xmm1
        nop     # marker
        jmp     f128_ext                        # TAILCALL


Tested with GCC 13.1. Link: https://gcc.godbolt.org/z/hdojahes5

[Bug target/115054] __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
What does msvc do?

[Bug target/115054] __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Does Microsoft's abi documents this case?

If not then gcc is as correct here as clang is.

[Bug target/115054] __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-170#parameter-passing

So it uses floating point as the type. But then it is vague on those kind of
type. Gcc treats _Float16 similar to how __m64 and __m128 are passed that is
via integer registers.

[Bug target/115054] __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[tmgross at umich dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

--- Comment #4 from Trevor Gross <tmgross at umich dot edu> ---
To my knowledge, MSVC does not support or specify an ABI for 16- or 128-bit
IEEE floating point types, so I do suppose that either GCC or Clang could be
considered correct here.

SysV doesn't say anything about f16 but does clarify that f128 should be SSE:

> Arguments of types __float128, _Decimal128 and __m128 are split
into two halves. The least significant ones belong to class SSE, the most
significant one to class SSEUP

Falling back to what SysV says seems reasonable to me since MSVC doesn't
provide any guidance, and passing via xmm is better register use anyway. Is
there any reason not to match SysV and Clang here? One side needs to change,
the mismatch is causing problems with rt math symbols.

[Bug target/115054] __float128 and _Float16 use incorrect ABI on x86-64 MinGW

[tmgross at umich dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115054

--- Comment #5 from Trevor Gross <tmgross at umich dot edu> ---
Looks like bugz cut off part of the sysv quote, here for reference:

> Arguments of types __float128, _Decimal128 and __m128 are split
> into two halves. The least significant ones belong to class SSE, the most
> significant one to class SSEUP.