[Bug bootstrap/56128] New: [4.8 Regression] No way to disable build of libsanitizer

[Bug bootstrap/56128] New: [4.8 Regression] No way to disable build of libsanitizer

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

             Bug #: 56128
           Summary: [4.8 Regression] No way to disable build of
                    libsanitizer
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: rguenth@gcc.gnu.org
            Target: x86_64-linux


There is no way to disable libsanitizer on "supported" platforms.  So with
(broken) kernel headers build fails with

In file included from
/gcc/spec/sb-vangelis-head-64/gcc/libsanitizer/sanitizer_common/sanitizer_linux.cc:35:0:
/usr/include/linux/futex.h:34:21: error: field '__user' has incomplete type
  struct robust_list __user *next;
                     ^
/usr/include/linux/futex.h:34:21: error: expected ';' at end of member
declaration
/usr/include/linux/futex.h:69:21: error: expected ';' at end of member
declaration
  struct robust_list __user *list_op_pending;
                     ^
/usr/include/linux/futex.h:96:26: error: expected ',' or '...' before '*' token
 long do_futex(u32 __user *uaddr, int op, u32 val, unsigned long timeout,
                          ^
/usr/include/linux/futex.h:100:31: error: expected ',' or '...' before '*'
token
 handle_futex_death(u32 __user *uaddr, struct task_struct *curr, int pi);
                               ^
make[3]: *** [sanitizer_linux.lo] Error 1

with no way to fix up (I don't need asan/tsan here).

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> 2013-01-28 12:21:14 UTC ---
13:17 < jakub> richi: libgomp defines FUTEX_{WAIT,WAKE} on its own rather then 
               including linux/futex.h, guess the googlers could consider such 
               a change on libasan side too

is one way of avoiding to include a kernel header (the failing header is
from 2.6.18, __user is not properly cleaned from it).

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2013-01-28
                 CC|                            |aoliva at gcc dot gnu.org,
                   |                            |bonzini at gnu dot org,
                   |                            |jakub at gcc dot gnu.org,
                   |                            |kcc at gcc dot gnu.org
     Ever Confirmed|0                           |1

--- Comment #2 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-01-28 13:06:40 UTC ---
I think we should both avoid linux/futex.h includes (both in libsanitizer and
testsuite/g++.dg/asan/, both inherited from upstream), either by doing a
configure test for it and only including it if it works properly, or just do
what libgomp or libitm do, don't include <linux/futex.h> and instead define
FUTEX_{WAIT,WAKE} directly, and also have some --disable-libsanitizer configury
option, CCing some build maintainers for that on where and how should that be
done.

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[bonzini at gnu dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #3 from Paolo Bonzini <bonzini at gnu dot org> 2013-01-28 13:36:36 UTC ---
--disable-target-libsanitizer should work:

# Handle --disable-<component> generically.
for dir in $configdirs $build_configdirs $target_configdirs ; do
  dirname=`echo $dir | sed -e s/target-//g -e s/build-//g -e s/-/_/g`
  varname=`echo $dirname | sed -e s/+/_/g`
  if eval test x\${enable_${varname}} "=" xno ; then
    noconfigdirs="$noconfigdirs $dir"
  fi
done

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |4.8.0

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #4 from Kostya Serebryany <kcc at gcc dot gnu.org> 2013-01-30 06:43:51 UTC ---
I'd prefer to simply disable libsanitizer build if something is not supported.
Since we are not testing it on older kernels, many things may be broken anyway. 
Dmitry?

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-01-30 08:13:29 UTC ---
The issue unfortunately isn't old vs. new kernels, just using <linux/*> and
<asm/*> headers, which as can be seen in this case sometimes aren't of a good
quality, kernel treats them as kernel headers and whether they are usable in
userland is far lower priority to them.  So, if <linux/*> or <asm/*> includes
can be avoided, it is always better to avoid them, and in this case it can be
very easily avoided.  And as for disabling whole sanitizer, how would you
expect it to work?  Just let users see a failed bootstrap and then find out
they need to add --disable-target-libsanitizer to configure next time?

OT, Richard, does --disable-target-libsanitizer work for you?

[Bug bootstrap/56128] [4.8 Regression] No way to disable build of libsanitizer

[rguenther at suse dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #6 from rguenther at suse dot de <rguenther at suse dot de> 2013-01-30 10:34:29 UTC ---
On Wed, 30 Jan 2013, jakub at gcc dot gnu.org wrote:

> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128
> 
> --- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-01-30 08:13:29 UTC ---
> The issue unfortunately isn't old vs. new kernels, just using <linux/*> and
> <asm/*> headers, which as can be seen in this case sometimes aren't of a good
> quality, kernel treats them as kernel headers and whether they are usable in
> userland is far lower priority to them.  So, if <linux/*> or <asm/*> includes
> can be avoided, it is always better to avoid them, and in this case it can be
> very easily avoided.  And as for disabling whole sanitizer, how would you
> expect it to work?  Just let users see a failed bootstrap and then find out
> they need to add --disable-target-libsanitizer to configure next time?
> 
> OT, Richard, does --disable-target-libsanitizer work for you?

Yes, it does.

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[dvyukov at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

Dmitry Vyukov <dvyukov at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dvyukov at google dot com

--- Comment #7 from Dmitry Vyukov <dvyukov at google dot com> 2013-01-30 15:32:02 UTC ---
Fixed upstream:
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/sanitizer_linux.cc?r1=173933&r2=173932&pathrev=173933

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |build
           Priority|P3                          |P1

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> 2013-02-08 15:16:59 UTC ---
Requires a merge as far as I can see.  Important enough build issue.

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #9 from Kostya Serebryany <kcc at gcc dot gnu.org> 2013-02-11 11:54:46 UTC ---
(In reply to comment #8)
> Requires a merge as far as I can see.  Important enough build issue.

I am reluctant to do another full merge so close to 4.8 -- we have too many
intrusive changes in upstream trunk. 

Jakub, WDYT? 
Maybe we just apply http://llvm.org/viewvc/llvm-project?rev=173933&view=rev
once 4.8 branches out? 
(Not sure if the gcc development process works this way...)

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #10 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-02-11 12:08:11 UTC ---
(In reply to comment #9)
> (In reply to comment #8)
> > Requires a merge as far as I can see.  Important enough build issue.
> 
> I am reluctant to do another full merge so close to 4.8 -- we have too many
> intrusive changes in upstream trunk. 
> 
> Jakub, WDYT? 
> Maybe we just apply http://llvm.org/viewvc/llvm-project?rev=173933&view=rev
> once 4.8 branches out? 
> (Not sure if the gcc development process works this way...)

We need more than that.  E.g. ppc64 asan is very much broken in the gcc tree,
that is fixed in upstream libasan already, right?

I'd think we should just do one more llvm -> libsanitizer merge soon (after the
scanf stuff is resolved?).  A slight complication is that asan_test.cc got
split into smaller tests, but even for GCC it will be worthwhile to have fewer
smaller tests I guess, we'll just need to add the wrappers.

BTW, any progress with the 0x7fff8000 shadow offset for x86_64?  If it looks
good benchmark-wise, I think it would be better if we switched the ASAN x86_64
ABI for GCC sooner (read, before 4.8) than have one asan ABI for 4.8 built
objects and another one for 4.9 built objects (which is still possible that it
will happen, but it wouldn't hurt to avoid that).

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #11 from Kostya Serebryany <kcc at gcc dot gnu.org> 2013-02-11 12:58:07 UTC ---
> We need more than that.  E.g. ppc64 asan is very much broken in the gcc tree,
> that is fixed in upstream libasan already, right?

Last time I tried, basic tests passed on ppc64 with 44 address space (did not
checke with 46 AS). 

> 
> I'd think we should just do one more llvm -> libsanitizer merge soon (after the
> scanf stuff is resolved?).  

Ok, I'll do the merge after the scanf stuff is in. 

> A slight complication is that asan_test.cc got
> split into smaller tests, but even for GCC it will be worthwhile to have fewer
> smaller tests I guess, we'll just need to add the wrappers.
> 
> BTW, any progress with the 0x7fff8000 shadow offset for x86_64?  

Not yet. Hopefully will try it this week. 

> If it looks
> good benchmark-wise, I think it would be better if we switched the ASAN x86_64
> ABI for GCC sooner (read, before 4.8) than have one asan ABI for 4.8 built
> objects and another one for 4.9 built objects (which is still possible that it
> will happen, but it wouldn't hurt to avoid that).

Agree. How much time do we have?

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #12 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-02-11 13:13:14 UTC ---
(In reply to comment #11)
> > We need more than that.  E.g. ppc64 asan is very much broken in the gcc tree,
> > that is fixed in upstream libasan already, right?
> 
> Last time I tried, basic tests passed on ppc64 with 44 address space (did not
> checke with 46 AS). 

http://ppc.koji.fedoraproject.org/packages/gcc/4.8.0/0.10.fc19/data/logs/ppc64/build.log
shows lots of failures.  Guess it is possible those boxes use 46-bit AS.

> Ok, I'll do the merge after the scanf stuff is in. 

Thanks.

> > BTW, any progress with the 0x7fff8000 shadow offset for x86_64?  
> 
> Not yet. Hopefully will try it this week. 

Looking forward to those results.
> 
> > If it looks
> > good benchmark-wise, I think it would be better if we switched the ASAN x86_64
> > ABI for GCC sooner (read, before 4.8) than have one asan ABI for 4.8 built
> > objects and another one for 4.9 built objects (which is still possible that it
> > will happen, but it wouldn't hurt to avoid that).
> 
> Agree. How much time do we have?

Not too much time, I guess if the real change on the gcc + libsanitizer side
will be tweaking a few constants, it can be done within next few weeks, but
preferably in February.  A full libsanitizer merge should be this or next week,
so that we have time to handle with any fallout of that (but at that point
already just selective backports).

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

--- Comment #13 from Jakub Jelinek <jakub at gcc dot gnu.org> 2013-02-11 15:54:17 UTC ---
Perhaps instead of global vars defined outside of libasan (which e.g. requires
GOT accesses to those vars in libasan), it might be better to have the scale
and offset as arguments of __asan_init?  Then you could easily test at runtime,
whether all compilation units agree on the same offset/scale, and complain if
they don't.  Then __asan_mapping_offset and __asan_mapping_scale or how are the
vars called could be hidden attribute, used with PC relative addressing and
avoid one extra indirection, and more importantly have better runtime checking
of mismatches.

[Bug bootstrap/56128] [4.8 Regression] libsanitizer doesn't build with broken kernel headers

[kcc at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56128

Kostya Serebryany <kcc at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #14 from Kostya Serebryany <kcc at gcc dot gnu.org> 2013-02-13 10:55:48 UTC ---
This particular problem (with futex headers) should be fixed by the last merge
(r196009)