[Bug gdb/31591] New: AMD64 calling convention of attributed vector type inconsistent between GCC and GDB

[Bug gdb/31591] New: AMD64 calling convention of attributed vector type inconsistent between GCC and GDB

[tsqurt at outlook dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=31591

            Bug ID: 31591
           Summary: AMD64 calling convention of attributed vector type
                    inconsistent between GCC and GDB
           Product: gdb
           Version: HEAD
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: gdb
          Assignee: unassigned at sourceware dot org
          Reporter: tsqurt at outlook dot com
  Target Milestone: ---

Tested Platform: AMD64, Ubuntu 20.04
Tested GCC: GCC 11.4.0, 12.3.0, 13.2.0,  14.0.1 20240331 (experimental)
Tested GDB: GNU GDB 13.2, 14.2 (without GCC 11.4.0), 15.0.50.20240331-git

Type with long __attribute__ ((vector_size(1))) should be passed by %xmm0
(%ymm0) (implemented by GCC), but GDB pass it by %rax. 

Here is a simple case of this issue.
This program should abort at line 26, because baz () is written unexpectedly
somehow. But by breaking at line 14 and returning the expected value, the
program should return 0 instead: 

typedef __INTPTR_TYPE__ V __attribute__((__vector_size__(sizeof
(__INTPTR_TYPE__))));

__attribute__((noinline, noclone)) V
foo ()
{
  V v = { (__INTPTR_TYPE__) foo };
  return v;
}

__attribute__((noinline, noclone)) V
baz ()
{
  V v = { (__INTPTR_TYPE__) baz };
  return v - 1; // (gdb) return v 
}

int
main ()
{
  V v;
  v = foo ();
  if (v[0] != (__INTPTR_TYPE__) foo)
    __builtin_abort ();
  v = baz ();
  if (v[0] != (__INTPTR_TYPE__) baz)
    __builtin_abort ();
  return 0;
}

Test code below shows that when GDB makes such functions return, GDB passes the
return value in %rax, rather than %xmm0.
gdb a.out
(gdb) b 14
(gdb) run
...
Breakpoint 1, baz () at a.c:15
15      }

(gdb) return v
Make baz return now? (y or n) y
#0  0x0000000000401191 in main () at a.c:24
24        v = baz ();

(gdb) info reg
rax            0x40113e <baz>           4198718
...

(gdb) info vec
...
ymm0           {... v4_int64 = {0x401126<foo> ...}

(gdb) continue
Program received signal SIGABRT, Aborted.
0x00007ffff7e0e00b in raise () from /lib/x86_64-linux-gnu/libc.so.6

I am wondering this problem could be caused by this function: gdb/amd64-tdep.c:
static void amd64_classify (struct type *type, enum amd64_reg_class
theclass[2]). As the following code show, float, double, decimal are realized,
and __m64 is recorded, but type attributed as vector is implemented by GCC but
not mentioned by GDB, or something lost when gdb command interpreter return
such command with expression.
/*  gdb/amd64-tdep.c */
...
static void amd64_classify (struct type *type, enum amd64_reg_class
theclass[2])
{
...
/* Arguments of types (signed and unsigned) _Bool, char, short, int,
   long, long long, and pointers are in the INTEGER class.  Similarly,
   range types, used by languages such as Ada, are also in the INTEGER
   class.  */
if ((code == TYPE_CODE_INT || code == TYPE_CODE_ENUM
     || code == TYPE_CODE_BOOL || code == TYPE_CODE_RANGE
     || code == TYPE_CODE_CHAR
     || code == TYPE_CODE_PTR || TYPE_IS_REFERENCE (type))
    && (len == 1 || len == 2 || len == 4 || len == 8))
  theclass[0] = AMD64_INTEGER;

/* Arguments of types _Float16, float, double, _Decimal32, _Decimal64 and
   __m64 are in class SSE.  */
else if ((code == TYPE_CODE_FLT || code == TYPE_CODE_DECFLOAT)
   && (len == 2 || len == 4 || len == 8))
  /* FIXME: __m64 .  */
  theclass[0] = AMD64_SSE;
...
}

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/31591] AMD64 calling convention of attributed vector type inconsistent between GCC and GDB

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31591

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug tdep/31591] AMD64 calling convention of attributed vector type inconsistent between GCC and GDB

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31591

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|gdb                         |tdep
                 CC|                            |tromey at sourceware dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.