[Bug c/67265] New: [x86] 'asm' operand has impossible constraints with -fstack-check

[Bug c/67265] New: [x86] 'asm' operand has impossible constraints with -fstack-check

[gnu at JonnyJD dot net]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 4133 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

            Bug ID: 67265
           Summary: [x86] 'asm' operand has impossible constraints with
                    -fstack-check
           Product: gcc
           Version: 5.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: gnu at JonnyJD dot net
  Target Milestone: ---
              Host: i686-linux-gnu

Created attachment 36209
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=36209&action=edit
minimal working example (C file)

When compiling a 32 bit build of libx264 (on a 64 bit machine) it works in
general, even with PIC.
However the build breaks when I add -fstack-check to CFLAGS.

The failing code is:

int a, b, c, d, e;
int fn1() {
  __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1), "mm"(c));
}

(attached as me.c)

This works:
 gcc -O3 -m32 -fPIC -c me.c

This fails (and should work)
 gcc -O3 -m32 -fstack-check -fPIC -c me.c

This also works (as a workaround):
 gcc -O3 -m32 -fstack-check -c me.c


The error message is:
me.c: In function ‘fn1’:
me.c:3:3: error: ‘asm’ operand has impossible constraints
   __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1),
"mm"(c));
   ^


The related discussion for x264 is here:
https://mailman.videolan.org/pipermail/x264-devel/2015-August/011246.html

The flag -fstack-check is added by default on Arch Linux when building
packages, but also additionally using hardened wrapper, where I also mentioned
the problem:
https://github.com/thestinger/hardening-wrapper/issues/6

I am the maintainer of lib32-libx264 on Arch Linux where the problem came up
originally:
https://aur.archlinux.org/packages/lib32-libx264/
>From gcc-bugs-return-495148-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Aug 19 00:38:41 2015
Return-Path: <gcc-bugs-return-495148-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 55882 invoked by alias); 19 Aug 2015 00:38:41 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 55871 invoked by uid 48); 19 Aug 2015 00:38:37 -0000
From: "gnu at JonnyJD dot net" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c/67265] [x86] 'asm' operand has impossible constraints with -fstack-check
Date: Wed, 19 Aug 2015 00:38:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c
X-Bugzilla-Version: 5.2.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: gnu at JonnyJD dot net
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Resolution:
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-67265-4-qOdPaK9KrO@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-67265-4@http.gcc.gnu.org/bugzilla/>
References: <bug-67265-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-08/txt/msg01290.txt.bz2
Content-length: 451

https://gcc.gnu.org/bugzilla/show_bug.cgi?idg265

--- Comment #1 from Johannes Dewender <gnu at JonnyJD dot net> ---
Created attachment 36210
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id6210&actioníit
Makefile with failing, working and workaround flags

I found the bug using gcc 5.2.0 (Arch Linux), but I also tested gcc 5.1.0 and
gcc 4.9.2.
So this is not a brand new bug. Not sure if it did work as expected in versions
before that.

[Bug target/67265] [x86] 'asm' operand has impossible constraints with -fstack-check

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|c                           |target

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
X86 is very much under huge register pressure and that inline-asm uses 4
registers as input/output and two memory ones. -fastback-check forces one more
register not to be available really.

[Bug target/67265] [x86] 'asm' operand has impossible constraints with -fstack-check

[danielmicay at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

Daniel Micay <danielmicay at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |danielmicay at gmail dot com

--- Comment #3 from Daniel Micay <danielmicay at gmail dot com> ---
It only temporarily needs a register to perform a write. The implementation
might currently have to reserve it but it *shouldn't* have to. It should be
free other than guaranteeing a one byte write per page.

[Bug target/67265] [x86] 'asm' operand has impossible constraints with -fstack-check

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2015-10-28
                 CC|                            |ebotcazou at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #4 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
The implementation currently reserves the frame pointer to be able to unwind
the stack and recover from the stack overflow.

[Bug target/67265] [x86] 'asm' operand has impossible constraints with -fstack-check

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

--- Comment #5 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
For the minimal testcase we can do:

Index: ira.c
===================================================================
--- ira.c       (revision 229402)
+++ ira.c       (working copy)
@@ -2273,7 +2273,9 @@ ira_setup_eliminable_regset (void)
        || (cfun->calls_alloca && EXIT_IGNORE_STACK)
        /* We need the frame pointer to catch stack overflow exceptions
          if the stack pointer is moving.  */
-       || (flag_stack_check && STACK_CHECK_MOVING_SP)
+       || (flag_stack_check
+          && STACK_CHECK_MOVING_SP
+          && (!crtl->is_leaf || cfun->calls_alloca))
        || crtl->accesses_prior_frames
        || (SUPPORTS_STACK_ALIGNMENT && crtl->stack_realign_needed)
        /* We need a frame pointer for all Cilk Plus functions that use

i.e. do not reserve the frame pointer if the function is leaf and doesn't call
alloca, but this might not be sufficient for the original testcase.

[Bug target/67265] [x86] 'asm' operand has impossible constraints with -fstack-check

[ebotcazou at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

Eric Botcazou <ebotcazou at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
                 CC|ebotcazou at gcc dot gnu.org       |
           Assignee|unassigned at gcc dot gnu.org      |ebotcazou at gcc dot gnu.org

--- Comment #6 from Eric Botcazou <ebotcazou at gcc dot gnu.org> ---
Let's try to do something.