[Bug bootstrap/55289] New: darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[Bug bootstrap/55289] New: darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

             Bug #: 55289
           Summary: darwin bootstrap fails due to missing
                    libsanitizer/interception/mach_override directory and
                    files
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: howarth@nitro.med.uc.edu


The merge of the asan branch breaks the bootstrap on darwin due to the missing
libsanitizer/interception/mach_override directory and associated files. These
were never ported from llvm compiler-rt into the asan gcc branch. Manually
adding the libsanitizer/interception/mach_override directory from the llvm
compiler-rt 3.2 branch restores the bootstrap on x86_64-apple-darwin12.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Jack Howarth <howarth at nitro dot med.uc.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Target|                            |*-*-darwin*
               Host|                            |*-*-darwin*
              Build|                            |*-*-darwin*
           Severity|normal                      |critical

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Dominique d'Humieres <dominiq at lps dot ens.fr> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2012-11-12
     Ever Confirmed|0                           |1

--- Comment #1 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-12 20:53:31 UTC ---
> Manually adding the libsanitizer/interception/mach_override directory from the llvm
> compiler-rt 3.2 branch restores the bootstrap on x86_64-apple-darwin12.

How do I get the libsanitizer/interception/mach_override directory on Xcode
3.2.6?

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #2 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 00:46:54 UTC ---
(In reply to comment #1)
> > Manually adding the libsanitizer/interception/mach_override directory from the llvm
> > compiler-rt 3.2 branch restores the bootstrap on x86_64-apple-darwin12.
> 
> How do I get the libsanitizer/interception/mach_override directory on Xcode
> 3.2.6?

This has nothing to do with Xcode. When the libsanitizer directories were
ported over from the llvm asan sources, an essential darwin-specific
interception/mach_override subdirectory was omitted. You can find these files
in..

http://llvm.org/svn/llvm-project/compiler-rt/branches/release_32/lib/interception/mach_override/

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #3 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-13 13:23:28 UTC ---
> You can find these files in..
>
> http://llvm.org/svn/llvm-project/compiler-rt/branches/release_32/lib/interception/mach_override/

With this files I have been able to do a clean bootstrap at revision 193472 on
x86_64-apple-darwin10. However, first, I have to supply the lib path for
libasan when using -faddress-sanitizer: -L/opt/gcc/gcc4.8w/lib -lasan; second,
the executable fails to run with:

dyld: Symbol not found: _CFStringCreateCopy
  Referenced from: /opt/gcc/gcc4.8w/lib/libasan.0.dylib
  Expected in: flat namespace
 in /opt/gcc/gcc4.8w/lib/libasan.0.dylib
Trace/BPT trap

Looking at the web, it seems to be a known issue with darwin, but I have been
unable to understand how it can be fixed.

Indeed I also see pr55291.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[iains at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #4 from Iain Sandoe <iains at gcc dot gnu.org> 2012-11-13 13:54:52 UTC ---
(In reply to comment #3)
> > You can find these files in..
> >
> > http://llvm.org/svn/llvm-project/compiler-rt/branches/release_32/lib/interception/mach_override/
> 
> With this files I have been able to do a clean bootstrap at revision 193472 on
> x86_64-apple-darwin10. However, first, I have to supply the lib path for
> libasan when using -faddress-sanitizer: -L/opt/gcc/gcc4.8w/lib -lasan; second,
> the executable fails to run with:
> 
> dyld: Symbol not found: _CFStringCreateCopy

this is found in the CoreFoundation framework (CFxxxxx is a good hint).

what you need is to have "-framework CoreFoundation" on the link line - and I
guess the configury  &c. needs to arrange this.


> Looking at the web, it seems to be a known issue with darwin, but I have been
> unable to understand how it can be fixed.
> 
> Indeed I also see pr55291.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #5 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 14:06:46 UTC ---
(In reply to comment #4)
> (In reply to comment #3)

> > dyld: Symbol not found: _CFStringCreateCopy
> 
> this is found in the CoreFoundation framework (CFxxxxx is a good hint).
> 
> what you need is to have "-framework CoreFoundation" on the link line - and I
> guess the configury  &c. needs to arrange this.
> 

The bootstrap completes without any other changes. What we need in
gcc/config/darwin.h is a LINK_SPEC entry for %faddress-sanitizer which passes
"-framework CoreFoundation -lasan" to the linker.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #6 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-13 14:10:36 UTC ---
> what you need is to have "-framework CoreFoundation" on the link line - and I
> guess the configury  &c. needs to arrange this.

When compiled with -faddress-sanitizer -framework CoreFoundation -lasan, the
executable fails to run with

dyld: lazy symbol binding failed: Symbol not found:
___asan_mach_override_ptr_custom
  Referenced from: /opt/gcc/gcc4.8w/lib/libasan.0.dylib
  Expected in: flat namespace

dyld: Symbol not found: ___asan_mach_override_ptr_custom
  Referenced from: /opt/gcc/gcc4.8w/lib/libasan.0.dylib
  Expected in: flat namespace

Trace/BPT trap

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #7 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 14:29:57 UTC ---
(In reply to comment #6)
> > what you need is to have "-framework CoreFoundation" on the link line - and I
> > guess the configury  &c. needs to arrange this.
> 
> When compiled with -faddress-sanitizer -framework CoreFoundation -lasan, the
> executable fails to run with
> 
> dyld: lazy symbol binding failed: Symbol not found:
> ___asan_mach_override_ptr_custom
>   Referenced from: /opt/gcc/gcc4.8w/lib/libasan.0.dylib
>   Expected in: flat namespace
> 
> dyld: Symbol not found: ___asan_mach_override_ptr_custom
>   Referenced from: /opt/gcc/gcc4.8w/lib/libasan.0.dylib
>   Expected in: flat namespace
> 
> Trace/BPT trap

This symbol is contained in mach_override.c which we currently don't build in
the Makefiles. Note that llvm's cmake files show mach_override.c is compiled
with -std=c99.

# Custom flags:
projects/compiler-rt/lib/interception/CMakeFiles/RTInterception.osx.dir/mach_override/mach_override.c.o_FLAGS
= -std=c99 

The interception/mach_override/mach_override.c.o object file will need to be
linked into both libasan.dylib and libasan.a.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #8 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 14:39:00 UTC ---
Iain,
    This brings up the sticky situation of having to modify the Makefile.am
file in libsanitizer/interception to add mach_override/mach_override.c to the
interception_files source list and the Makefile.am file in libsanitizer/asan to
link in the resulting interception/mach_override/mach_override.o object code to
libasan. Any ideas on how to do this in a target specific manner cleanly?

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #9 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 19:50:20 UTC ---
Created attachment 28676
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28676
hack to build asan support

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #10 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 19:52:35 UTC ---
The attached patch (with configure and Makefile.in regenerated) allows the asan
support to build on x86_64-apple-darwin12. It still isn't functional yet as
test cases compiled with...

g++-fsf-4.8 -faddress-sanitizer -framework CoreFoundation hello.cc -lasan

produce...

# ./a.out
mach_override: some instructions unknown! Need to update mach_override.c
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:308
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:321
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:327
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:340
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:345
Hello, world.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #11 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 20:13:18 UTC ---
For the simple test case...

int main()
{
int i;
i=5;
}

compiled with 'g++-fsf-4.8 -faddress-sanitizer -O0 -framework CoreFoundation
test.cc -lasan'
using a mach_override.c with...

#define DEBUG_DISASM 1
// #undef DEBUG_DISASM

the errors appear late in the debug output at...

Replacing function at 0x11183d630
First 16 bytes of the function: 48 8d 5 11 4f b 0 53 48 8d 5f e0 48 89 77 90 
To disassemble, save the following function as disas.c and run:
  gcc -c disas.c && gobjdump -d disas.o
The first 16 bytes of the original function will start after four nop
instructions.

void foo() {
  asm volatile("nop;nop;nop;nop;");
  asm volatile(".byte 0x48, 0x8d, 0x5, 0x11, 0x4f, 0xb, 0x0, 0x53;");
  asm volatile(".byte 0x48, 0x8d, 0x5f, 0xe0, 0x48, 0x89, 0x77, 0x90;");
}

Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
mach_override: some instructions unknown! Need to update mach_override.c
overridePossible = false @299
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:308
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:321
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:327
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:340
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:345
First 16 bytes of the function after slicing: 48 8d 5 11 4f b 0 53 48 8d 5f e0
48 89 77 90 
Replacing function at 0x7fff94c23364

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #12 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 20:21:35 UTC ---
The disasembled testcase that is choking on shows...


test.o:     file format mach-o-x86-64


Disassembly of section .text:

0000000000000000 <_main>:
   0:    55                       push   %rbp
   1:    48 89 e5                 mov    %rsp,%rbp
   4:    c7 45 fc 05 00 00 00     movl   $0x5,-0x4(%rbp)
   b:    b8 00 00 00 00           mov    $0x0,%eax
  10:    5d                       pop    %rbp
  11:    c3                       retq   

0000000000000012 <__GLOBAL__sub_I_00099_0_test.cc>:
  12:    55                       push   %rbp
  13:    48 89 e5                 mov    %rsp,%rbp
  16:    e8 00 00 00 00           callq  1b
<__GLOBAL__sub_I_00099_0_test.cc+0x9>
  1b:    5d                       pop    %rbp
  1c:    c3                       retq   

Disassembly of section __DATA.__mod_init_func:

0000000000000020 <__DATA.__mod_init_func>:
    ...

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #13 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 20:25:45 UTC ---
Also note the comment in mach_override.c...

#elif defined(__x86_64__)
// TODO(glider): disassembling the "0x48, 0x89" sequences is trickier than it's
done below.
// If it stops working, refer to
http://ref.x86asm.net/geek.html#modrm_byte_32_64 to do it
// more accurately.
// Note: 0x48 is in fact the REX.W prefix, but it might be wrong to treat it as
a separate
// instruction.

looks like we are bumping into this bug.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #14 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 20:50:37 UTC ---
Actually, this appears to be just noise on the output from a functional
libasan. If I use the testcase from
http://code.google.com/p/address-sanitizer/wiki/AddressSanitizer of...

% cat tests/use-after-free.c
#include <stdlib.h>
int main() {
  char *x = (char*)malloc(10 * sizeof(char*));
  free(x);
  return x[5];
}

I get...

howarth% gcc-fsf-4.8 -faddress-sanitizer -framework CoreFoundation -O1
-fno-omit-frame-pointer -g use-after-free.c -lasan
howarth% ./a.out
mach_override: some instructions unknown! Need to update mach_override.c
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:308
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:321
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:327
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:340
err = f8000001
../../../../gcc-4.8-20121113/libsanitizer/interception/mach_override/mach_override.c:345
=================================================================
==88551== ERROR: AddressSanitizer heap-use-after-free on address 0x000105cbaf45
at pc 0x103001f12 bp 0x7fff5cbfe8f0 sp 0x7fff5cbfe8e8
READ of size 1 at 0x000105cbaf45 thread T0
    #0 0x103001f11 (/Users/howarth/./a.out+0x100000f11)
    #1 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #2 0x0
0x000105cbaf45 is located 5 bytes inside of 80-byte region
[0x000105cbaf40,0x000105cbaf90)
freed by thread T0 here:
    #0 0x1030147a4 (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb7a4)
    #1 0x10301492a (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb92a)
    #2 0x103001ee5 (/Users/howarth/./a.out+0x100000ee5)
    #3 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #4 0x0
previously allocated by thread T0 here:
    #0 0x103014255 (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb255)
    #1 0x7fff94c3b152 (/usr/lib/system/libsystem_c.dylib+0x2d152)
    #2 0x7fff94c3bba6 (/usr/lib/system/libsystem_c.dylib+0x2dba6)
    #3 0x103001eda (/Users/howarth/./a.out+0x100000eda)
    #4 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
Shadow byte and word:
  0x100020b975e8: fd
  0x100020b975e8: fd fd fd fd fd fd fd fd
More shadow bytes:
  0x100020b975c8: fa fa fa fa fa fa fa fa
  0x100020b975d0: fa fa fa fa fa fa fa fa
  0x100020b975d8: fa fa fa fa fa fa fa fa
  0x100020b975e0: fa fa fa fa fa fa fa fa
=>0x100020b975e8: fd fd fd fd fd fd fd fd
  0x100020b975f0: fd fd fd fd fd fd fd fd
  0x100020b975f8: fa fa fa fa fa fa fa fa
  0x100020b97600: fa fa fa fa fa fa fa fa
  0x100020b97608: fa fa fa fa fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 8:2047; 
  mallocs by size class: 8:1; 
  frees   by size class: 8:1; 
  rfrees  by size class: 
Stats: malloc large: 0 small slow: 1
==88551== ABORTING

compared to...

howarth% /sw/opt/llvm-3.2/bin/clang -fsanitize=address -O1
-fno-omit-frame-pointer -g use-after-free.c

howarth% ./a.out
=================================================================
==88537== ERROR: AddressSanitizer: heap-use-after-free on address
0x00010a0a2f45 at pc 0x107dcae54 bp 0x7fff57e358f0 sp 0x7fff57e358e8
READ of size 1 at 0x00010a0a2f45 thread T0
    #0 0x107dcae53 (/Users/howarth/./a.out+0x100000e53)
    #1 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #2 0x0
0x00010a0a2f45 is located 5 bytes inside of 80-byte region
[0x00010a0a2f40,0x00010a0a2f90)
freed by thread T0 here:
    #0 0x107dd3878 (/Users/howarth/./a.out+0x100009878)
    #1 0x107dd2ef2 (/Users/howarth/./a.out+0x100008ef2)
    #2 0x107dcae1a (/Users/howarth/./a.out+0x100000e1a)
    #3 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #4 0x0
previously allocated by thread T0 here:
    #0 0x107dd3682 (/Users/howarth/./a.out+0x100009682)
    #1 0x7fff94c3b152 (/usr/lib/system/libsystem_c.dylib+0x2d152)
    #2 0x7fff94c3bba6 (/usr/lib/system/libsystem_c.dylib+0x2dba6)
    #3 0x107dcae0f (/Users/howarth/./a.out+0x100000e0f)
    #4 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
Shadow byte and word:
  0x1000214145e8: fd
  0x1000214145e8: fd fd fd fd fd fd fd fd
More shadow bytes:
  0x1000214145c8: fa fa fa fa fa fa fa fa
  0x1000214145d0: fa fa fa fa fa fa fa fa
  0x1000214145d8: fa fa fa fa fa fa fa fa
  0x1000214145e0: fa fa fa fa fa fa fa fa
=>0x1000214145e8: fd fd fd fd fd fd fd fd
  0x1000214145f0: fd fd fd fd fd fd fd fd
  0x1000214145f8: fa fa fa fa fa fa fa fa
  0x100021414600: fa fa fa fa fa fa fa fa
  0x100021414608: fa fa fa fa fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 8:2047; 
  mallocs by size class: 8:1; 
  frees   by size class: 8:1; 
  rfrees  by size class: 
Stats: malloc large: 0 small slow: 1
==88537== ABORTING

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[konstantin.s.serebryany at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Konstantin Serebryany <konstantin.s.serebryany at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |konstantin.s.serebryany at
                   |                            |gmail dot com

--- Comment #15 from Konstantin Serebryany <konstantin.s.serebryany at gmail dot com> 2012-11-13 21:27:08 UTC ---
Please not that upstream asan is in the process of getting rid of mach_override
in favor of Mac's function interposition.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Jack Howarth <howarth at nitro dot med.uc.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #28676|0                           |1
        is obsolete|                            |

--- Comment #16 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 21:55:22 UTC ---
Created attachment 28677
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28677
prototype patch for adding darwin asan support

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #17 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-13 22:14:10 UTC ---
(In reply to comment #16)
> Created attachment 28677 [details]
> prototype patch for adding darwin asan support

Note that you need to run...

autoconf -I. -I./config
cd libsanitizer
autoreconf -I. -I../config
automake-1.11

after applying the prototype patch.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #18 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-13 23:21:35 UTC ---
> prototype patch for adding darwin asan support

Already updated;-(


-  x86_64-*-linux-* | i?86-*-linux-*)
+  x86_64-*-linux-* | i?86-*-linux-* | *-*-darwin*)

should now go to libsanitizer/configure.tgt and not to configure.ac.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Jack Howarth <howarth at nitro dot med.uc.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #28677|0                           |1
        is obsolete|                            |

--- Comment #19 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 01:28:44 UTC ---
Created attachment 28680
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28680
revised prototype patch for adding darwin asan support

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #20 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 01:30:13 UTC ---
Revised patch builds at r193494 after...

cd libsanitizer
autoconf -I. -I../config
automake-1.11

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #21 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 14:24:43 UTC ---
Patch posted at http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01116.html which
assumes the libsanitizer maintainers import mach_override/mach_override.h and
mach_override/mach_override.c.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #22 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 14:45:48 UTC ---
Revised patch posted at
http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01119.html.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #23 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 15:51:44 UTC ---
Created attachment 28683
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28683
debug output from mach_override running use_after_free test case

Debugging output from mach_override/mach_override.c using test case built with
proposed patch using...

gcc-fsf-4.8 -faddress-sanitizer -framework CoreFoundation -O1
-fno-omit-frame-pointer -g -o use-after-free use-after-free.c -lasan

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #24 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 15:59:55 UTC ---
Created attachment 28684
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28684
disassembled use-after-free.o from clang 3.2

Disassembled use-after-free.o generated with...

/sw/opt/llvm-3.2/bin/clang -fsanitize=address -O1 -fno-omit-frame-pointer -g
use-after-free.c --save-temps 
objdump -d use-after-free.o > use-after-free.clang.disasm

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #25 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 16:02:03 UTC ---
Created attachment 28685
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28685
disassembled use-after-free.o from gcc trunk

Disassembled use-after-free.o generated with...

gcc-fsf-4.8 -faddress-sanitizer -framework CoreFoundation -O1
-fno-omit-frame-pointer -g -o use-after-free use-after-free.c -lasan
--save-temps
objdump -d use-after-free.o > use-after-free.gcc48.disasm

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #26 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 16:03:03 UTC ---
Created attachment 28686
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28686
diff between disassembled use-after-free.o generated by clang 3.2 and gcc trunk

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #27 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 16:06:10 UTC ---
The error in mach_override.c occurs for the use-after-free.c test case at...

Replacing function at 0x1119ef630
First 16 bytes of the function: 48 8d 5 11 4f b 0 53 48 8d 5f e0 48 89 77 90 
To disassemble, save the following function as disas.c and run:
  gcc -c disas.c && gobjdump -d disas.o
The first 16 bytes of the original function will start after four nop
instructions.

void foo() {
  asm volatile("nop;nop;nop;nop;");
  asm volatile(".byte 0x48, 0x8d, 0x5, 0x11, 0x4f, 0xb, 0x0, 0x53;");
  asm volatile(".byte 0x48, 0x8d, 0x5f, 0xe0, 0x48, 0x89, 0x77, 0x90;");
}

Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48  FAIL
Matching: 48 8d  FAIL
mach_override: some instructions unknown! Need to update mach_override.c
overridePossible = false @299
err = f8000001
../../../../gcc-4.8-20121114/libsanitizer/interception/mach_override/mach_override.c:308
err = f8000001
../../../../gcc-4.8-20121114/libsanitizer/interception/mach_override/mach_override.c:321
err = f8000001
../../../../gcc-4.8-20121114/libsanitizer/interception/mach_override/mach_override.c:327
err = f8000001
../../../../gcc-4.8-20121114/libsanitizer/interception/mach_override/mach_override.c:340
err = f8000001
../../../../gcc-4.8-20121114/libsanitizer/interception/mach_override/mach_override.c:345
First 16 bytes of the function after slicing: 48 8d 5 11 4f b 0 53 48 8d 5f e0
48 89 77 90

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[glider at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Alexander Potapenko <glider at google dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |glider at google dot com

--- Comment #28 from Alexander Potapenko <glider at google dot com> 2012-11-14 16:13:47 UTC ---
===========================================================
t.o:     file format mach-o-x86-64


Disassembly of section .text:

0000000000000000 <_foo>:
   0:    55                       push   %rbp
   1:    48 89 e5                 mov    %rsp,%rbp
   4:    90                       nop
   5:    90                       nop
   6:    90                       nop
   7:    90                       nop
   8:    48 8d 05 11 4f 0b 00     lea    0xb4f11(%rip),%rax        # b4f20
<_foo.eh+0xb4ee8>
   f:    53                       push   %rbx
  10:    48 8d 5f e0              lea    -0x20(%rdi),%rbx
  14:    48 89 77 90              mov    %rsi,-0x70(%rdi)
  18:    5d                       pop    %rbp
  19:    c3                       retq   
===========================================================

mach_override must be choking on the lea instruction.
The easiest way to handle it is to add "48 8d 05 00 00 00 00" to the list of
instructions and mark it as "lea $imm(%rip),%rax". You can also dive into the
opcodes a bit and mask the bits that select the destination register.
I'll take a look at mach_override.c code to suggest the actual patch.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[glider at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #29 from Alexander Potapenko <glider at google dot com> 2012-11-14 16:40:53 UTC ---
Index: mach_override.c
===================================================================
--- mach_override.c    (revision 167724)
+++ mach_override.c    (working copy)
@@ -725,6 +725,8 @@
         { 0x2, {0xFF, 0x00}, {0x89, 0x00} },                               //
mov r/m32,r32 or r/m16,r16
         { 0x3, {0xFF, 0xFF, 0xFF}, {0x49, 0x89, 0xF8} },                   //
mov %rdi,%r8
         { 0x4, {0xFF, 0xFF, 0xFF, 0xFF}, {0x40, 0x0F, 0xBE, 0xCE} },       //
movsbl %sil,%ecx
+        { 0x7, {0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00},
+               {0x48, 0x8D, 0x05, 0x00, 0x00, 0x00, 0x00} },  // lea
$imm(%rip),%rax
         { 0x3, {0xFF, 0xFF, 0xFF}, {0x0F, 0xBE, 0xCE} },  // movsbl, %dh, %ecx
         { 0x3, {0xFF, 0xFF, 0x00}, {0xFF, 0x77, 0x00} },  // pushq $imm(%rdi)
         { 0x2, {0xFF, 0xFF}, {0xDB, 0xE3} }, // fninit


===================================================================
Please also make sure the interceptors work in 32-bit programs

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #30 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 16:54:06 UTC ---
(In reply to comment #29)
Thanks with the patch applied from comment 29, now the use-after-free testcase
works without errors...

howarth% ./use-after-free
=================================================================
==30656== ERROR: AddressSanitizer heap-use-after-free on address 0x000108843f45
at pc 0x105b8bf12 bp 0x7fff5a0748e0 sp 0x7fff5a0748d8
READ of size 1 at 0x000108843f45 thread T0
    #0 0x105b8bf11 (/Users/howarth/./use-after-free+0x100000f11)
    #1 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #2 0x0
0x000108843f45 is located 5 bytes inside of 80-byte region
[0x000108843f40,0x000108843f90)
freed by thread T0 here:
    #0 0x105b9a7a4 (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb7a4)
    #1 0x105b9a92a (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb92a)
    #2 0x105b8bee5 (/Users/howarth/./use-after-free+0x100000ee5)
    #3 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #4 0x0
previously allocated by thread T0 here:
    #0 0x105b9a255 (/sw/lib/gcc4.8/lib/libasan.0.dylib+0xb255)
    #1 0x7fff94c3b152 (/usr/lib/system/libsystem_c.dylib+0x2d152)
    #2 0x7fff94c3bba6 (/usr/lib/system/libsystem_c.dylib+0x2dba6)
    #3 0x105b8beda (/Users/howarth/./use-after-free+0x100000eda)
    #4 0x7fff8bd827e0 (/usr/lib/system/libdyld.dylib+0x27e0)
Shadow byte and word:
  0x1000211087e8: fd
  0x1000211087e8: fd fd fd fd fd fd fd fd
More shadow bytes:
  0x1000211087c8: fa fa fa fa fa fa fa fa
  0x1000211087d0: fa fa fa fa fa fa fa fa
  0x1000211087d8: fa fa fa fa fa fa fa fa
  0x1000211087e0: fa fa fa fa fa fa fa fa
=>0x1000211087e8: fd fd fd fd fd fd fd fd
  0x1000211087f0: fd fd fd fd fd fd fd fd
  0x1000211087f8: fa fa fa fa fa fa fa fa
  0x100021108800: fa fa fa fa fa fa fa fa
  0x100021108808: fa fa fa fa fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 8:2047; 
  mallocs by size class: 8:1; 
  frees   by size class: 8:1; 
  rfrees  by size class: 
Stats: malloc large: 0 small slow: 1
==30656== ABORTING

I'll try to test a i386-apple-darwin10 build of gcc trunk later today (unless
the multilib for libsanitizer gets fixed first and I can test -m32 on a
x86_64-apple-darwin12 build).

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #31 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 19:10:15 UTC ---
Also fine on i386-apple-darwin10...

howarth% ./use-after-free
=================================================================
==82550== ERROR: AddressSanitizer heap-use-after-free on address 0x0207afc5 at
pc 0x1f0b bp 0xbfffe3a8 sp 0xbfffe39c
READ of size 1 at 0x0207afc5 thread T0
    #0 0x1f0a (/Users/howarth/./use-after-free+0x1f0a)
    #1 0x1ec0 (/Users/howarth/./use-after-free+0x1ec0)
    #2 0x0 (/Users/howarth/./use-after-free+0x0)
0x0207afc5 is located 5 bytes inside of 40-byte region [0x0207afc0,0x0207afe8)
freed by thread T0 here:
    #0 0x10b34 (/sw_i386/lib/gcc4.8/lib/libasan.0.dylib+0xcb34)
    #1 0x10cdb (/sw_i386/lib/gcc4.8/lib/libasan.0.dylib+0xccdb)
    #2 0x1ee6 (/Users/howarth/./use-after-free+0x1ee6)
    #3 0x1ec0 (/Users/howarth/./use-after-free+0x1ec0)
    #4 0x0 (/Users/howarth/./use-after-free+0x0)
previously allocated by thread T0 here:
    #0 0x10515 (/sw_i386/lib/gcc4.8/lib/libasan.0.dylib+0xc515)
    #1 0x99041ba7 (/usr/lib/libSystem.B.dylib+0x4ba7)
    #2 0x9903fc77 (/usr/lib/libSystem.B.dylib+0x2c77)
    #3 0x1edc (/Users/howarth/./use-after-free+0x1edc)
    #4 0x1ec0 (/Users/howarth/./use-after-free+0x1ec0)
    #5 0x0 (/Users/howarth/./use-after-free+0x0)
Shadow byte and word:
  0x2040f5f8: fd
  0x2040f5f8: fd fd fd fd
More shadow bytes:
  0x2040f5e8: fa fa fa fa
  0x2040f5ec: fa fa fa fa
  0x2040f5f0: fa fa fa fa
  0x2040f5f4: fa fa fa fa
=>0x2040f5f8: fd fd fd fd
  0x2040f5fc: fd fd fd fd
  0x2040f600: fa fa fa fa
  0x2040f604: fa fa fa fa
  0x2040f608: fa fa fa fa
Stats: 0M malloced (0M for red zones) by 1 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 1 calls
Stats: 0M really freed by 0 calls
Stats: 0M (128 full pages) mmaped in 1 calls
  mmaps   by size class: 7:4095; 
  mallocs by size class: 7:1; 
  frees   by size class: 7:1; 
  rfrees  by size class: 
Stats: malloc large: 0 small slow: 1
==82550== ABORTING

Alexander, can you check in
libsanitizer/interception/mach_override/mach_override.c (patched as in comment
229) and libsanitizer/interception/mach_override/mach_override.h with the build
infrastructure patch from
http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01119.html? It would be nice to be
able to start testing libsanitizer on intel darwin.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[konstantin.s.serebryany at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #32 from Konstantin Serebryany <konstantin.s.serebryany at gmail dot com> 2012-11-14 20:21:19 UTC ---
Just want to repeat, that any work on mach_override may end up being wasted
time
because we plan to get rid of mach_override *really* soon.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #33 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 20:56:50 UTC ---
(In reply to comment #32)
> Just want to repeat, that any work on mach_override may end up being wasted
> time
> because we plan to get rid of mach_override *really* soon.

Is that certain to be soon enough to insure that darwin has usable asan support
for gcc 4.8?
I would rather have the option of continuing to be able to test libsanitizer on
darwin than having no support at all.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #34 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-14 22:09:37 UTC ---
(In reply to comment #32)
> Just want to repeat, that any work on mach_override may end up being wasted
> time
> because we plan to get rid of mach_override *really* soon.

One other question. Will the replacement of mach_override also depend on the
Core Foundation framework?

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[konstantin.s.serebryany at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #35 from Konstantin Serebryany <konstantin.s.serebryany at gmail dot com> 2012-11-14 23:10:00 UTC ---
>> Is that certain to be soon enough 
Not 100%. I am just warning you.

>> Will the replacement of mach_override also depend on the Core Foundation framework?

That's the question to Alex.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #36 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-14 23:49:25 UTC ---
Jack,

Could you please post a patch with the regenerated files. I have lost my
struggle with the auto* versions and I am giving up.

TIA

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #37 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 02:06:45 UTC ---
Created attachment 28692
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=28692
revised patch with regenerated files and proposed darwin.h link spec change

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #38 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 02:10:33 UTC ---
(In reply to comment #35)
> >> Is that certain to be soon enough 
> Not 100%. I am just warning you.

Considering that the release schedule for llvm 3.2 is close to that of gcc 4.8,
it seems unrealistic to expect to be able to replace mach_override in time for
the gcc 4.8.0 release. If the existing mach_override is sufficient for llvm
3.2, I suspect we can also live with it (at least until gcc 4.8.1).

> 
> >> Will the replacement of mach_override also depend on the Core Foundation framework?
> 
> That's the question to Alex.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[glider at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #39 from Alexander Potapenko <glider at google dot com> 2012-11-15 08:18:01 UTC ---
I agree with Kostya that no major changes to mach_override are necessary
because we are really going to dump it. However minor fixes required for ASan
in GCC to work on Darwin right now should be perfectly ok.

As there currently are issues blocking the dynamic ASan runtime (e.g.
https://code.google.com/p/address-sanitizer/issues/detail?id=124), and we still
depend on CoreFoundation I can't promise that we'll get rid of mach_override
before the GCC 4.8 release.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #40 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 13:46:32 UTC ---
(In reply to comment #39)
> I agree with Kostya that no major changes to mach_override are necessary
> because we are really going to dump it. However minor fixes required for ASan
> in GCC to work on Darwin right now should be perfectly ok.
> 
> As there currently are issues blocking the dynamic ASan runtime (e.g.
> https://code.google.com/p/address-sanitizer/issues/detail?id=124), and we still
> depend on CoreFoundation I can't promise that we'll get rid of mach_override
> before the GCC 4.8 release.

Now that you have committed the patch from comment 29 to upstream
compiler-rt...

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20121112/155989.html

can we at least get missing mach_override/mach_override.h and
mach_override/mach_override.c files imported into the asan branch, if not gcc
trunk, as well as  the approved patch from...

http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01119.html

so that we can start building libsanitizer on intel darwin? Thanks in advance
for any help on these issues.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #41 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 13:51:11 UTC ---
I will also post a small patch to gcc-patches to allow darwin to automatically
pass the required linker flags on the link spec in darwin.h...

ndex: gcc/config/darwin.h
===================================================================
--- gcc/config/darwin.h    (revision 193503)
+++ gcc/config/darwin.h    (working copy)
@@ -180,6 +180,8 @@ extern GTY(()) int darwin_ms_struct;
     %{L*} %(link_libgcc) %o
%{fprofile-arcs|fprofile-generate*|coverage:-lgcov} \
     %{fopenmp|ftree-parallelize-loops=*: \
       %{static|static-libgcc|static-libstdc++|static-libgfortran: libgomp.a%s;
: -lgomp } } \
+    %{faddress-sanitizer: \
+      %{static|static-libgcc|static-libstdc++|static-libgfortran: -framework
CoreFoundation libasan.a%s; : -framework CoreFoundation -lasan } } \
     %{fgnu-tm: \
       %{static|static-libgcc|static-libstdc++|static-libgfortran: libitm.a%s;
: -litm } } \
     %{!nostdlib:%{!nodefaultlibs:\

It works fine for the dynamic linkage although we do seem to have an issue with
the static libasan.a that will require a new PR to be opened...

% gcc-fsf-4.8 -faddress-sanitizer -O1 -fno-omit-frame-pointer -static-libgcc 
-g -o use-after-free use-after-free.c
Undefined symbols for architecture x86_64:
  "___cxa_throw", referenced from:
      __asan::InitializeAsanInterceptors()      in
libasan.a(asan_interceptors.o)
     (maybe you meant: __interception::real___cxa_throw, _wrap___cxa_throw )
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

However I don't see that as a blocker to getting libsanitizer building on intel
darwin.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[dominiq at lps dot ens.fr]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #42 from Dominique d'Humieres <dominiq at lps dot ens.fr> 2012-11-15 15:03:53 UTC ---
Jack,

Thanks for the patch in comment #37. I had to do some minor surgery to adjust
it to revision 193528, but otherwise it worked as expected for both -m32 and
-m64 even with gfortran.

I think it should go to trunk as an experimental feature (see pr55341).

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #43 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 15:12:31 UTC ---
Posted updated patch to adjust for bit-rot from the libsanitizer multilib
support checkin and add required spec handling to gcc/config/darwin.h. Ignore
my previous comment about the static build failing. This was a glitch in my
fink packaging where I deleted the libstdc++.la from dependency_libs= in
libasan.la. So we have no issues with using libasan.a.

http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01251.html

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #44 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 17:53:40 UTC ---
Posted the final revision of the patch that now properly handles static linkage
options with -faddress-sanitizer...

http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01285.html

For example...

% gcc-fsf-4.8 -faddress-sanitizer -O1 -fno-omit-frame-pointer -g
-static-libstdc++ -o use-after-free use-after-free.c
% otool -L ./use-after-free
./use-after-free:
   
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
(compatibility version 150.0.0, current version 744.12.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
169.3.0)
    /sw/lib/gcc4.8/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
version 1.0.0)

links in libstdc++.a while...

% gcc-fsf-4.8 -faddress-sanitizer -O1 -fno-omit-frame-pointer -g -static-libgcc
-o use-after-free use-after-free.c

% otool -L ./use-after-free
./use-after-free:
   
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
(compatibility version 150.0.0, current version 744.12.0)
    /sw/lib/gcc4.8/lib/libstdc++.6.dylib (compatibility version 7.0.0, current
version 7.17.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
169.3.0)

links libstdc++.dylib. The standard linkage with...

% gcc-fsf-4.8 -faddress-sanitizer -O1 -fno-omit-frame-pointer -g -o
use-after-free use-after-free.c

% otool -L ./use-after-free
./use-after-free:
   
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
(compatibility version 150.0.0, current version 744.12.0)
    /sw/lib/gcc4.8/lib/libasan.0.dylib (compatibility version 1.0.0, current
version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
169.3.0)
    /sw/lib/gcc4.8/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
version 1.0.0)

doesn't need to be directly linked against libstdc++ because libasan.0.dylib
already is...

% otool -L /sw/lib/gcc4.8/lib/libasan.0.dylib 
/sw/lib/gcc4.8/lib/libasan.0.dylib:
    /sw/lib/gcc4.8/lib/libasan.0.dylib (compatibility version 1.0.0, current
version 1.0.0)
    /sw/lib/gcc4.8/lib/libstdc++.6.dylib (compatibility version 7.0.0, current
version 7.17.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
169.3.0)
    /sw/lib/gcc4.8/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
version 1.0.0)

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[iains at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #45 from Iain Sandoe <iains at gcc dot gnu.org> 2012-11-15 17:58:21 UTC ---
(In reply to comment #35)
> >> Is that certain to be soon enough 
> Not 100%. I am just warning you.

apologies for not much input to this - somewhat tied up with other things.

will the new version (did I read somewhere, library interposing?) support the
older versions of Darwin/OSX (10.5, 10.6) ?  
(if not, then there's a place for the existing method).

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #46 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 18:28:44 UTC ---
(In reply to comment #45)
> (In reply to comment #35)
> > >> Is that certain to be soon enough 
> > Not 100%. I am just warning you.
> 
> apologies for not much input to this - somewhat tied up with other things.
> 
> will the new version (did I read somewhere, library interposing?) support the
> older versions of Darwin/OSX (10.5, 10.6) ?  
> (if not, then there's a place for the existing method).

See...

http://code.google.com/p/address-sanitizer/issues/detail?id=64

The interpose example code works fine here under 10.6.8.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #47 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 19:09:00 UTC ---
(In reply to comment #46)
> (In reply to comment #45)
> > (In reply to comment #35)
> > > >> Is that certain to be soon enough 
> > > Not 100%. I am just warning you.
> > 
> > apologies for not much input to this - somewhat tied up with other things.
> > 
> > will the new version (did I read somewhere, library interposing?) support the
> > older versions of Darwin/OSX (10.5, 10.6) ?  
> > (if not, then there's a place for the existing method).
> 
> See...
> 
> http://code.google.com/p/address-sanitizer/issues/detail?id=64
> 
> The interpose example code works fine here under 10.6.8.

The interpose example code also works fine under 10.5.8 on a dual G5.

[Bug bootstrap/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #48 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-15 19:19:06 UTC ---
As proof that the current mach_override.h and mach_override.c from current
compiler-rt svn are  reasonable to use for gcc 4.8.0, I compiled the complete
Polyhedron 2005 benchmark with...

gfortran-fsf-4.8 -O1 -g -fno-omit-frame-pointer -faddress-sanitizer  %n.f90 -o
%n

and

gfortran-fsf-4.8 -O3 -funroll-loops -ffast-math -g -fno-omit-frame-pointer
-faddress-sanitizer  %n.f90 -o %n

and in both cases each benchmark runs without any error output from
mach_override.c.

[Bug sanitizer/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[glider at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

--- Comment #49 from Alexander Potapenko <glider at google dot com> 2012-11-22 10:43:03 UTC ---
The new interposition library should work on 10.6.
But ASan itself doesn't work on 10.5, at least I remember some problems with
its compilation. We're not really interested in supporting 10.5, as it's
completely unsupported by Apple.

[Bug sanitizer/55289] darwin bootstrap fails due to missing libsanitizer/interception/mach_override directory and files

[howarth at nitro dot med.uc.edu]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289

Jack Howarth <howarth at nitro dot med.uc.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #50 from Jack Howarth <howarth at nitro dot med.uc.edu> 2012-11-24 22:37:22 UTC ---
i386 and x86_64 darwin now supports libsanitizer with...

Author: kcc
Date: Sat Nov 24 18:24:13 2012
New Revision: 193781

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=193781
Log:
[libsanitizer] add mach_override and enable libsanitizer on darwin

Added:
    trunk/libsanitizer/interception/mach_override/
    trunk/libsanitizer/interception/mach_override/LICENSE.txt
    trunk/libsanitizer/interception/mach_override/mach_override.c
    trunk/libsanitizer/interception/mach_override/mach_override.h
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/darwin.h
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/configure
    trunk/libsanitizer/configure.ac
    trunk/libsanitizer/configure.tgt
    trunk/libsanitizer/interception/Makefile.am
    trunk/libsanitizer/interception/Makefile.in