[Bug c/67265] New: [x86] 'asm' operand has impossible constraints with -fstack-check

["gnu at JonnyJD dot net" <gcc-bugzilla@gcc.gnu.org>]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 4133 bytes --]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67265

            Bug ID: 67265
           Summary: [x86] 'asm' operand has impossible constraints with
                    -fstack-check
           Product: gcc
           Version: 5.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: gnu at JonnyJD dot net
  Target Milestone: ---
              Host: i686-linux-gnu

Created attachment 36209
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=36209&action=edit
minimal working example (C file)

When compiling a 32 bit build of libx264 (on a 64 bit machine) it works in
general, even with PIC.
However the build breaks when I add -fstack-check to CFLAGS.

The failing code is:

int a, b, c, d, e;
int fn1() {
  __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1), "mm"(c));
}

(attached as me.c)

This works:
 gcc -O3 -m32 -fPIC -c me.c

This fails (and should work)
 gcc -O3 -m32 -fstack-check -fPIC -c me.c

This also works (as a workaround):
 gcc -O3 -m32 -fstack-check -c me.c


The error message is:
me.c: In function ‘fn1’:
me.c:3:3: error: ‘asm’ operand has impossible constraints
   __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1),
"mm"(c));
   ^


The related discussion for x264 is here:
https://mailman.videolan.org/pipermail/x264-devel/2015-August/011246.html

The flag -fstack-check is added by default on Arch Linux when building
packages, but also additionally using hardened wrapper, where I also mentioned
the problem:
https://github.com/thestinger/hardening-wrapper/issues/6

I am the maintainer of lib32-libx264 on Arch Linux where the problem came up
originally:
https://aur.archlinux.org/packages/lib32-libx264/
>From gcc-bugs-return-495148-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org Wed Aug 19 00:38:41 2015
Return-Path: <gcc-bugs-return-495148-listarch-gcc-bugs=gcc.gnu.org@gcc.gnu.org>
Delivered-To: listarch-gcc-bugs@gcc.gnu.org
Received: (qmail 55882 invoked by alias); 19 Aug 2015 00:38:41 -0000
Mailing-List: contact gcc-bugs-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc-bugs.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc-bugs/>
List-Post: <mailto:gcc-bugs@gcc.gnu.org>
List-Help: <mailto:gcc-bugs-help@gcc.gnu.org>
Sender: gcc-bugs-owner@gcc.gnu.org
Delivered-To: mailing list gcc-bugs@gcc.gnu.org
Received: (qmail 55871 invoked by uid 48); 19 Aug 2015 00:38:37 -0000
From: "gnu at JonnyJD dot net" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug c/67265] [x86] 'asm' operand has impossible constraints with -fstack-check
Date: Wed, 19 Aug 2015 00:38:00 -0000
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: gcc
X-Bugzilla-Component: c
X-Bugzilla-Version: 5.2.0
X-Bugzilla-Keywords:
X-Bugzilla-Severity: normal
X-Bugzilla-Who: gnu at JonnyJD dot net
X-Bugzilla-Status: UNCONFIRMED
X-Bugzilla-Resolution:
X-Bugzilla-Priority: P3
X-Bugzilla-Assigned-To: unassigned at gcc dot gnu.org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: attachments.created
Message-ID: <bug-67265-4-qOdPaK9KrO@http.gcc.gnu.org/bugzilla/>
In-Reply-To: <bug-67265-4@http.gcc.gnu.org/bugzilla/>
References: <bug-67265-4@http.gcc.gnu.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
X-Bugzilla-URL: http://gcc.gnu.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2015-08/txt/msg01290.txt.bz2
Content-length: 451

https://gcc.gnu.org/bugzilla/show_bug.cgi?idg265

--- Comment #1 from Johannes Dewender <gnu at JonnyJD dot net> ---
Created attachment 36210
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id6210&actioníit
Makefile with failing, working and workaround flags

I found the bug using gcc 5.2.0 (Arch Linux), but I also tested gcc 5.1.0 and
gcc 4.9.2.
So this is not a brand new bug. Not sure if it did work as expected in versions
before that.