[gcc r12-3292] Objective-C, NeXT: Fix messenging non-aggregate return-in-memory.

[Iain D Sandoe <iains@gcc.gnu.org>]

https://gcc.gnu.org/g:1cef3039b880a21fbdf4153e6fc42026619fd4ad

commit r12-3292-g1cef3039b880a21fbdf4153e6fc42026619fd4ad
Author: Iain Sandoe <iain@sandoe.co.uk>
Date:   Mon Aug 16 21:22:13 2021 +0100

    Objective-C, NeXT: Fix messenging non-aggregate return-in-memory.
    
    When a method returns a type that the platform ABI says should be
    returned in memory, and that is done by a hidden 'sret' parameter,
    the message send calls must be adjusted to inform the runtime that
    the sret parameter is present.  As reported in the PR, this is not
    working for non-aggregate types that use this mechanism.  The fix
    here is to adjust the logic such that all return values that flag
    'in memory' are considered to use the mechanism *unless* they
    provide a struct_value_rtx *and* the return object is an aggregate.
    
    Signed-off-by: Iain Sandoe <iain@sandoe.co.uk>
    
    PR objc/101718 - Objective-C frontend emits wrong code to call methods returning scalar types returned in memory
    
            PR objc/101718
    
    gcc/objc/ChangeLog:
    
            * objc-next-runtime-abi-02.c (build_v2_build_objc_method_call):
            Revise for cases where scalar objects use an sret parameter.
            (next_runtime_abi_02_build_objc_method_call): Likwise.

Diff:
---
 gcc/objc/objc-next-runtime-abi-02.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/gcc/objc/objc-next-runtime-abi-02.c b/gcc/objc/objc-next-runtime-abi-02.c
index ce831fc34ae..9c35738a95c 100644
--- a/gcc/objc/objc-next-runtime-abi-02.c
+++ b/gcc/objc/objc-next-runtime-abi-02.c
@@ -1739,15 +1739,16 @@ build_v2_build_objc_method_call (int super, tree method_prototype,
   /* Param list + 2 slots for object and selector.  */
   vec_alloc (parms, nparm + 2);
 
-  /* If we are returning a struct in memory, and the address
-     of that memory location is passed as a hidden first
-     argument, then change which messenger entry point this
-     expr will call.  NB: Note that sender_cast remains
-     unchanged (it already has a struct return type).  */
-  if (!targetm.calls.struct_value_rtx (0, 0)
-      && (TREE_CODE (ret_type) == RECORD_TYPE
-	  || TREE_CODE (ret_type) == UNION_TYPE)
-      && targetm.calls.return_in_memory (ret_type, 0))
+  /* If we are returning an item that must be returned in memory, and the
+     target ABI does this by an invisible pointer provided as the first arg,
+     we need to adjust the message signature to include this.  The second
+     part of this excludes targets that provide some alternate scheme for
+     structure returns.  */
+  if (ret_type && !VOID_TYPE_P (ret_type)
+      && targetm.calls.return_in_memory (ret_type, 0)
+      && !(targetm.calls.struct_value_rtx (0, 0)
+	   && (TREE_CODE (ret_type) == RECORD_TYPE
+	       || TREE_CODE (ret_type) == UNION_TYPE)))
     {
       if (super)
 	sender = umsg_id_super2_stret_fixup_decl;
@@ -1849,10 +1850,12 @@ next_runtime_abi_02_build_objc_method_call (location_t loc,
 	     ? TREE_VALUE (TREE_TYPE (method_prototype))
 	     : objc_object_type;
 
-  if (!targetm.calls.struct_value_rtx (0, 0)
-      && (TREE_CODE (ret_type) == RECORD_TYPE
-	  || TREE_CODE (ret_type) == UNION_TYPE)
-      && targetm.calls.return_in_memory (ret_type, 0))
+  /* See comment for the fixup version above.  */
+  if (ret_type && !VOID_TYPE_P (ret_type)
+      && targetm.calls.return_in_memory (ret_type, 0)
+      && !(targetm.calls.struct_value_rtx (0, 0)
+	   && (TREE_CODE (ret_type) == RECORD_TYPE
+	       || TREE_CODE (ret_type) == UNION_TYPE)))
     {
       if (super)
 	message_func_decl = umsg_id_super2_stret_fixup_decl;