[gcc r14-9503] i386: Fix setup of incoming varargs for (...) functions which return large aggregates [PR114175]

[gcc r14-9503] i386: Fix setup of incoming varargs for (...) functions which return large aggregates [PR114175]

[Jakub Jelinek]

https://gcc.gnu.org/g:218d17496122abe1fd831bd003f129310b32ca83

commit r14-9503-g218d17496122abe1fd831bd003f129310b32ca83
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Sat Mar 16 15:16:33 2024 +0100

    i386: Fix setup of incoming varargs for (...) functions which return large aggregates [PR114175]
    
    The c23-stdarg-6.c testcase I've added recently apparently works fine with
    -O0 but aborts with -O1 and higher on x86_64-linux.
    The problem is in setup of incoming varargs.
    
    Like function.cc before r14-9249 even ix86_setup_incoming_varargs assumes
    that TYPE_NO_NAMED_ARGS_STDARG_P don't have any named arguments and there
    is nothing to advance, but that is not the case for (...) functions
    returning by hidden reference which have one such artificial argument.
    If the setup_incoming_varargs hook is called from the
      if (TYPE_NO_NAMED_ARGS_STDARG_P (TREE_TYPE (fndecl))
          && fnargs.is_empty ())
        {
          struct assign_parm_data_one data = {};
          assign_parms_setup_varargs (&all, &data, false);
        }
    spot, i.e. where there is no hidden return argument passed, arg.type
    is always NULL, while when it is called in the
          if (cfun->stdarg && !DECL_CHAIN (parm))
            assign_parms_setup_varargs (&all, &data, false);
    spot, even when it is TYPE_NO_NAMED_ARGS_STDARG_P arg.type will be non-NULL.
    The tree-stdarg.cc pass in f in c23-stdarg-6.cc at -O1 or higher determines
    that va_arg is used on integral types at most twice (loads 2 words),
    and because ix86_setup_incoming_varargs doesn't advance, the code saves
    just the %rdi and %rsi registers to the save area.  But that isn't correct,
    it should save %rsi and %rdx because %rdi is the hidden return argument.
    With -O0 tree-stdarg.cc doesn't attempt to optimize and we save all the
    registers, so it works fine in that case.
    
    Now, I think we'll need the same fix also on
    aarch64, alpha, arc, csky, ia64, loongarch, mips, mmix, nios2, riscv, visium
    which have pretty much the similarly looking snippet in their hooks
    changed by the r13-3549 commit.
    Then arm, epiphany, fr30, frv, ft32, m32r, mcore, nds32, rs6000, sh
    have different changes but most likely need something similar too.
    I don't have access to most of those, could test aarch64 and rs6000 I guess.
    
    2024-03-16  Jakub Jelinek  <jakub@redhat.com>
    
            PR target/114175
            * config/i386/i386.cc (ix86_setup_incoming_varargs): Only skip
            ix86_function_arg_advance for TYPE_NO_NAMED_ARGS_STDARG_P functions
            if arg.type is NULL.
    
            * gcc.dg/c23-stdarg-7.c: New test.
            * gcc.dg/c23-stdarg-8.c: New test.

Diff:
---
 gcc/config/i386/i386.cc             | 3 ++-
 gcc/testsuite/gcc.dg/c23-stdarg-7.c | 6 ++++++
 gcc/testsuite/gcc.dg/c23-stdarg-8.c | 6 ++++++
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/gcc/config/i386/i386.cc b/gcc/config/i386/i386.cc
index 4b6b665e599..2357ba3f4c6 100644
--- a/gcc/config/i386/i386.cc
+++ b/gcc/config/i386/i386.cc
@@ -4643,7 +4643,8 @@ ix86_setup_incoming_varargs (cumulative_args_t cum_v,
   /* For varargs, we do not want to skip the dummy va_dcl argument.
      For stdargs, we do want to skip the last named argument.  */
   next_cum = *cum;
-  if (!TYPE_NO_NAMED_ARGS_STDARG_P (TREE_TYPE (current_function_decl))
+  if ((!TYPE_NO_NAMED_ARGS_STDARG_P (TREE_TYPE (current_function_decl))
+       || arg.type != NULL_TREE)
       && stdarg_p (fntype))
     ix86_function_arg_advance (pack_cumulative_args (&next_cum), arg);
 
diff --git a/gcc/testsuite/gcc.dg/c23-stdarg-7.c b/gcc/testsuite/gcc.dg/c23-stdarg-7.c
new file mode 100644
index 00000000000..b05a9623cd7
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/c23-stdarg-7.c
@@ -0,0 +1,6 @@
+/* Test C23 variadic functions with no named parameters, or last named
+   parameter with a declaration not allowed in C17.  Execution tests.  */
+/* { dg-do run } */
+/* { dg-options "-O2 -std=c23 -pedantic-errors" } */
+
+#include "c23-stdarg-4.c"
diff --git a/gcc/testsuite/gcc.dg/c23-stdarg-8.c b/gcc/testsuite/gcc.dg/c23-stdarg-8.c
new file mode 100644
index 00000000000..81140312731
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/c23-stdarg-8.c
@@ -0,0 +1,6 @@
+/* Test C23 variadic functions with no named parameters, or last named
+   parameter with a declaration not allowed in C17.  Execution tests.  */
+/* { dg-do run } */
+/* { dg-options "-O2 -std=c23 -pedantic-errors" } */
+
+#include "c23-stdarg-6.c"