From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <gcc-bugs-return-379685-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Received: (qmail 7457 invoked by alias); 12 Jan 2012 10:23:21 -0000
Received: (qmail 7448 invoked by uid 22791); 12 Jan 2012 10:23:20 -0000
X-SWARE-Spam-Status: No, hits=-2.9 required=5.0	tests=ALL_TRUSTED,AWL,BAYES_00
X-Spam-Check-By: sourceware.org
Received: from localhost (HELO gcc.gnu.org) (127.0.0.1)    by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 12 Jan 2012 10:23:07 +0000
From: "rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c/8081] ICE with variably sized types returned from nested functions
Date: Thu, 12 Jan 2012 10:23:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c
X-Bugzilla-Keywords: accepts-invalid, ice-on-invalid-code
X-Bugzilla-Severity: normal
X-Bugzilla-Who: rguenth at gcc dot gnu.org
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields:
Message-ID: <bug-8081-4-GgJG9GGyZh@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-8081-4@http.gcc.gnu.org/bugzilla/>
References: <bug-8081-4@http.gcc.gnu.org/bugzilla/>
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
Content-Type: text/plain; charset="UTF-8"
MIME-Version: 1.0
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
X-SW-Source: 2012-01/txt/msg01360.txt.bz2

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8081

--- Comment #24 from Richard Guenther <rguenth at gcc dot gnu.org> 2012-01-12 10:23:05 UTC ---
Created attachment 26306
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=26306
A patch for using by-reference passing

(In reply to comment #23)
> as alternative to rejecting this case we can lower returning variable-size
> types during un-nesting to explicit passing of a return slot and using
> memcpy, making the nested function return nothing.

It's of course not that easy as we gimplify before un-nesting.  The frontend
would be responsible to arrange things that way, similar to how we pass
a return slot in the C++ frontend (DECL_BY_REFERENCE on the DECL_RESULT
variable).  Like for the attached patch.  Passes

extern void abort (void);
int
main (int argc, char **argv)
{
  int size = 10;
  typedef struct
    {
      char val[size];
    }
  block;
  block a, b;
  block __attribute__((noinline))
  retframe_block ()
    {
      return *(block *) &b;
    }
  b.val[0] = -1;
  b.val[1] = -2;
  a=retframe_block ();
  if (a.val[0] != -1
      || a.val[1] != -2)
    abort ();
  return 0;
}

I'm not sure if one can construct a testcase where using return-slot
optimization causes wrong-code generation.  Alternatively checking
DECL_BY_REFERENCE on the callees DECL_RESULT instead of applying it to
all VLA types could work (though not for indirect calls).