From: Jack Howarth <howarth@bromo.med.uc.edu>
To: dodji@redhat.com
Cc: gcc-patches@gcc.gnu.org, dnovillo@google.com, jakub@redhat.com,
wmi@google.com, davidxl@google.com,
konstantin.s.serebryany@gmail.com,
Dodji Seketeli <dodji@seketeli.org>
Subject: Re: [PATCH 00/13] Request to merge Address Sanitizer in
Date: Thu, 15 Nov 2012 19:42:00 -0000 [thread overview]
Message-ID: <20121115194154.GA16078@bromo.med.uc.edu> (raw)
In-Reply-To: <1351799566-31447-1-git-send-email-dodji@redhat.com>
On Thu, Nov 01, 2012 at 08:52:33PM +0100, dodji@redhat.com wrote:
> From: Dodji Seketeli <dodji@seketeli.org>
>
> Hello,
>
> The set of patches following this message represents the work that
> happened on the asan branch to build up the Address Sanitizer work
> started in the Google branch.
>
> Address Sanitizer (aka asan) is a memory error detector. It finds
> use-after-free and {heap,stack,global}-buffer overflow bugs in C/C++
> programs.
>
> One can learn about the way it works by reading the pdf slides at [1],
> or by reading the documentation on the wiki page of the project at [2].
>
> To make a long story short, it works by associating each memory region
> of eight consecutive bytes with a shadow byte that tells whether if
> each byte of the memory region is addressable or not. So,
> conceptually, there is a function 'MemToShadow' which, for each set of
> contiguous eight bytes of memory returns a shadow byte that tells
> whether if each byte is accessible or not.
>
> Then, each memory access is instrumented by the asan pass to retrieve
> the shadow byte of the accessed memory; if the access is to a memory
> address that is deemed non-accessible, a call to an asan runtime
> library function is issued to report a meaningful error to the user,
> and the access is performed, letting the user program proceed despite
> the error.
>
> The advantage of this approach, compared to say, Valgrind[4] is the
> lower time and space overhead. Eventually, when this tool becomes
> more solid, it'll become complementary to Valgrind.
>
> Apart from the compiler components, asan needs a runtime library to
> function. We share that library with the LLVM implementation of asan
> that is described at [3]. The last patch of the set imports this
> library in its pristine form into our tree. The plan is to regularly
> synchronize it with its LLVM upstream repository.
>
> On behalf of the GCC asan developers listed below, I am thus proposing
> these patches for inclusion into trunk. I chose to follow the
> chronological commits that happened on the [asan] branch, to ease the
> authorship propagation. Except for some few exceptions, each of these
> commits are reasonably logically atomic, so they hopefully shouldn't
> be too hard to review.
>
> The first patch is the initial import of the asan state from the
> Google branch into the [asan] branch. Subsequent patches clean the
> code up, add features like protection of stack and global variables,
> instrumentation of memory access through built-in functions, and, last
> but not least, the import of the runtime library.
>
> Please note that the ChangeLog.asan is meant to disappear at commit
> time, as its content will be updated (for the dates) and prepended to
> the normal ChangeLog file.
>
> One noticeable shortcoming that we have at the moment is the lack of a
> DejaGNU test harness for this. This is planned to be addressed as
> soon as possible.
>
> Please find below is a summary of the patches of the set.
>
> Thanks.
>
> [1]: http://gcc.gnu.org/wiki/cauldron2012?action=AttachFile&do=get&target=kcc.pdf
> [2]: http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm
> [3]: http://code.google.com/p/address-sanitizer/w/list
> [4]: http://www.valgrind.org
>
> Diego Novillo (2):
> Initial import of asan from the Google branch
> Rename tree-asan.[ch] to asan.[ch]
>
> Dodji Seketeli (3):
> Make build_check_stmt accept an SSA_NAME for its base
> Factorize condition insertion code out of build_check_stmt
> Instrument built-in memory access function calls
>
> Jakub Jelinek (6):
> Initial asan cleanups
> Emit GIMPLE directly instead of gimplifying GENERIC.
> Allow asan at -O0
> Implement protection of stack variables
> Implement protection of global variables
> Fix a couple of ICEs.
>
> Wei Mi (2):
> Don't forget to protect 32 bytes aligned global variables.
> Import the asan runtime library into GCC tree
>
> ChangeLog.asan | 7 +
> Makefile.def | 2 +
> Makefile.in | 487 +-
> configure | 1 +
> configure.ac | 1 +
> gcc/ChangeLog.asan | 175 +
> gcc/Makefile.in | 10 +-
> gcc/asan.c | 1495 ++
> gcc/asan.h | 70 +
> gcc/cfgexpand.c | 165 +-
> gcc/common.opt | 4 +
> gcc/config/i386/i386.c | 11 +
> gcc/doc/invoke.texi | 8 +-
> gcc/doc/tm.texi | 6 +
> gcc/doc/tm.texi.in | 2 +
> gcc/gcc.c | 1 +
> gcc/passes.c | 2 +
> gcc/target.def | 11 +
> gcc/toplev.c | 14 +
> gcc/tree-pass.h | 2 +
> gcc/varasm.c | 22 +
> libasan/ChangeLog.asan | 3 +
> libasan/LICENSE.TXT | 97 +
> libasan/Makefile.am | 98 +
> libasan/Makefile.in | 992 ++
> libasan/README.gcc | 4 +
> libasan/aclocal.m4 | 9645 ++++++++++
> libasan/asan_allocator.cc | 1045 ++
> libasan/asan_allocator.h | 177 +
> libasan/asan_flags.h | 103 +
> libasan/asan_globals.cc | 206 +
> libasan/asan_intercepted_functions.h | 217 +
> libasan/asan_interceptors.cc | 704 +
> libasan/asan_interceptors.h | 39 +
> libasan/asan_internal.h | 169 +
> libasan/asan_linux.cc | 150 +
> libasan/asan_lock.h | 40 +
> libasan/asan_mac.cc | 526 +
> libasan/asan_mac.h | 54 +
> libasan/asan_malloc_linux.cc | 142 +
> libasan/asan_malloc_mac.cc | 427 +
> libasan/asan_malloc_win.cc | 140 +
> libasan/asan_mapping.h | 120 +
> libasan/asan_new_delete.cc | 54 +
> libasan/asan_poisoning.cc | 151 +
> libasan/asan_posix.cc | 118 +
> libasan/asan_report.cc | 492 +
> libasan/asan_report.h | 51 +
> libasan/asan_rtl.cc | 404 +
> libasan/asan_stack.cc | 35 +
> libasan/asan_stack.h | 52 +
> libasan/asan_stats.cc | 86 +
> libasan/asan_stats.h | 65 +
> libasan/asan_thread.cc | 153 +
> libasan/asan_thread.h | 103 +
> libasan/asan_thread_registry.cc | 188 +
> libasan/asan_thread_registry.h | 83 +
> libasan/asan_win.cc | 190 +
> libasan/config.guess | 1530 ++
> libasan/config.sub | 1773 ++
> libasan/configure | 17515 +++++++++++++++++++
> libasan/configure.ac | 67 +
> libasan/depcomp | 630 +
> libasan/include/sanitizer/asan_interface.h | 197 +
> libasan/include/sanitizer/common_interface_defs.h | 66 +
> libasan/install-sh | 527 +
> libasan/interception/interception.h | 195 +
> libasan/interception/interception_linux.cc | 28 +
> libasan/interception/interception_linux.h | 35 +
> libasan/interception/interception_mac.cc | 29 +
> libasan/interception/interception_mac.h | 47 +
> libasan/interception/interception_win.cc | 149 +
> libasan/interception/interception_win.h | 43 +
> libasan/libtool-version | 6 +
> libasan/ltmain.sh | 9661 ++++++++++
> libasan/missing | 376 +
> libasan/sanitizer_common/sanitizer_allocator.cc | 83 +
> libasan/sanitizer_common/sanitizer_allocator64.h | 573 +
> libasan/sanitizer_common/sanitizer_atomic.h | 63 +
> libasan/sanitizer_common/sanitizer_atomic_clang.h | 120 +
> libasan/sanitizer_common/sanitizer_atomic_msvc.h | 134 +
> libasan/sanitizer_common/sanitizer_common.cc | 151 +
> libasan/sanitizer_common/sanitizer_common.h | 181 +
> libasan/sanitizer_common/sanitizer_flags.cc | 95 +
> libasan/sanitizer_common/sanitizer_flags.h | 25 +
> libasan/sanitizer_common/sanitizer_internal_defs.h | 186 +
> libasan/sanitizer_common/sanitizer_libc.cc | 189 +
> libasan/sanitizer_common/sanitizer_libc.h | 69 +
> libasan/sanitizer_common/sanitizer_linux.cc | 296 +
> libasan/sanitizer_common/sanitizer_list.h | 118 +
> libasan/sanitizer_common/sanitizer_mac.cc | 249 +
> libasan/sanitizer_common/sanitizer_mutex.h | 106 +
> libasan/sanitizer_common/sanitizer_placement_new.h | 31 +
> libasan/sanitizer_common/sanitizer_posix.cc | 187 +
> libasan/sanitizer_common/sanitizer_printf.cc | 196 +
> libasan/sanitizer_common/sanitizer_procmaps.h | 95 +
> libasan/sanitizer_common/sanitizer_stackdepot.cc | 194 +
> libasan/sanitizer_common/sanitizer_stackdepot.h | 27 +
> libasan/sanitizer_common/sanitizer_stacktrace.cc | 245 +
> libasan/sanitizer_common/sanitizer_stacktrace.h | 73 +
> libasan/sanitizer_common/sanitizer_symbolizer.cc | 311 +
> libasan/sanitizer_common/sanitizer_symbolizer.h | 97 +
> .../sanitizer_common/sanitizer_symbolizer_linux.cc | 162 +
> .../sanitizer_common/sanitizer_symbolizer_mac.cc | 31 +
> .../sanitizer_common/sanitizer_symbolizer_win.cc | 33 +
> libasan/sanitizer_common/sanitizer_win.cc | 205 +
> 106 files changed, 57193 insertions(+), 25 deletions(-)
> create mode 100644 ChangeLog.asan
> create mode 100644 gcc/ChangeLog.asan
> create mode 100644 gcc/asan.c
> create mode 100644 gcc/asan.h
> create mode 100644 libasan/ChangeLog.asan
> create mode 100644 libasan/LICENSE.TXT
> create mode 100644 libasan/Makefile.am
> create mode 100644 libasan/Makefile.in
> create mode 100644 libasan/README.gcc
> create mode 100644 libasan/aclocal.m4
> create mode 100644 libasan/asan_allocator.cc
> create mode 100644 libasan/asan_allocator.h
> create mode 100644 libasan/asan_flags.h
> create mode 100644 libasan/asan_globals.cc
> create mode 100644 libasan/asan_intercepted_functions.h
> create mode 100644 libasan/asan_interceptors.cc
> create mode 100644 libasan/asan_interceptors.h
> create mode 100644 libasan/asan_internal.h
> create mode 100644 libasan/asan_linux.cc
> create mode 100644 libasan/asan_lock.h
> create mode 100644 libasan/asan_mac.cc
> create mode 100644 libasan/asan_mac.h
> create mode 100644 libasan/asan_malloc_linux.cc
> create mode 100644 libasan/asan_malloc_mac.cc
> create mode 100644 libasan/asan_malloc_win.cc
> create mode 100644 libasan/asan_mapping.h
> create mode 100644 libasan/asan_new_delete.cc
> create mode 100644 libasan/asan_poisoning.cc
> create mode 100644 libasan/asan_posix.cc
> create mode 100644 libasan/asan_report.cc
> create mode 100644 libasan/asan_report.h
> create mode 100644 libasan/asan_rtl.cc
> create mode 100644 libasan/asan_stack.cc
> create mode 100644 libasan/asan_stack.h
> create mode 100644 libasan/asan_stats.cc
> create mode 100644 libasan/asan_stats.h
> create mode 100644 libasan/asan_thread.cc
> create mode 100644 libasan/asan_thread.h
> create mode 100644 libasan/asan_thread_registry.cc
> create mode 100644 libasan/asan_thread_registry.h
> create mode 100644 libasan/asan_win.cc
> create mode 100644 libasan/config.guess
> create mode 100644 libasan/config.sub
> create mode 100644 libasan/configure
> create mode 100644 libasan/configure.ac
> create mode 100644 libasan/depcomp
> create mode 100644 libasan/include/sanitizer/asan_interface.h
> create mode 100644 libasan/include/sanitizer/common_interface_defs.h
> create mode 100644 libasan/install-sh
> create mode 100644 libasan/interception/interception.h
> create mode 100644 libasan/interception/interception_linux.cc
> create mode 100644 libasan/interception/interception_linux.h
> create mode 100644 libasan/interception/interception_mac.cc
> create mode 100644 libasan/interception/interception_mac.h
> create mode 100644 libasan/interception/interception_win.cc
> create mode 100644 libasan/interception/interception_win.h
> create mode 100644 libasan/libtool-version
> create mode 100644 libasan/ltmain.sh
> create mode 100644 libasan/missing
> create mode 100644 libasan/sanitizer_common/sanitizer_allocator.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_allocator64.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic_clang.h
> create mode 100644 libasan/sanitizer_common/sanitizer_atomic_msvc.h
> create mode 100644 libasan/sanitizer_common/sanitizer_common.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_common.h
> create mode 100644 libasan/sanitizer_common/sanitizer_flags.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_flags.h
> create mode 100644 libasan/sanitizer_common/sanitizer_internal_defs.h
> create mode 100644 libasan/sanitizer_common/sanitizer_libc.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_libc.h
> create mode 100644 libasan/sanitizer_common/sanitizer_linux.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_list.h
> create mode 100644 libasan/sanitizer_common/sanitizer_mac.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_mutex.h
> create mode 100644 libasan/sanitizer_common/sanitizer_placement_new.h
> create mode 100644 libasan/sanitizer_common/sanitizer_posix.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_printf.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_procmaps.h
> create mode 100644 libasan/sanitizer_common/sanitizer_stackdepot.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_stackdepot.h
> create mode 100644 libasan/sanitizer_common/sanitizer_stacktrace.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_stacktrace.h
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer.h
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_linux.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_mac.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_symbolizer_win.cc
> create mode 100644 libasan/sanitizer_common/sanitizer_win.cc
>
Dodji,
The Google branch is missing the required interception/mach_override/mach_override.h
and interception/mach_override/mach_override.c files from compiler-rt svn for darwin. I have
posted what I believe to be the final patch which eanbles libsanitizer on darwin...
http://gcc.gnu.org/ml/gcc-patches/2012-11/msg01285.html
which has been tested with the existing asan testsuite, the use-after-free.c testcase as
well as the Polyhedron 2005 benchmarks for -O1 -g -fno-omit-frame-pointer -faddress-sanitizer
and -O3 -funroll-loops -ffast-math -g -fno-omit-frame-pointer -faddress-sanitizer
to prove that the current mach_override from upstream is sufficient for darwin to use.
Due to the large number of maintainers for libsanitizer, it is unclear who is the person
responsible for upstream merges to lobby for these files to be ported into gcc trunk.
With Alexander Potapenko's commit of the bug fix to mach_override/mach_override.c
required for FSF gcc...
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20121112/155989.html
...there really is no reason to continue to delay (as the interpose code simply won't
be completed in time for gcc 4.8.0). Can we please get some movement on importing
these missing files from upstream? Thanks.
Jack
next prev parent reply other threads:[~2012-11-15 19:42 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-01 19:53 dodji
2012-11-01 19:53 ` [PATCH 03/13] Initial asan cleanups dodji
2012-11-01 19:53 ` [PATCH 08/13] Fix a couple of ICEs dodji
2012-11-01 19:53 ` [PATCH 01/13] Initial import of asan from the Google branch dodji
2012-11-01 19:53 ` [PATCH 02/13] Rename tree-asan.[ch] to asan.[ch] dodji
2012-11-01 21:54 ` Joseph S. Myers
2012-11-02 22:44 ` Dodji Seketeli
2012-11-01 19:53 ` [PATCH 06/13] Implement protection of stack variables dodji
[not found] ` <CAGQ9bdweH8Pn=8vLTNa8FSzAh92OYrWScxK78n9znCodADJUvw@mail.gmail.com>
2012-11-02 4:35 ` Xinliang David Li
2012-11-02 15:25 ` Dodji Seketeli
2012-11-02 14:44 ` Dodji Seketeli
[not found] ` <CAGQ9bdxQG3i=BrSYmaN-ssdv4omW6F5VTg50viskKNcYrF-8BQ@mail.gmail.com>
2012-11-02 16:02 ` Dodji Seketeli
2012-11-01 19:53 ` [PATCH 10/13] Make build_check_stmt accept an SSA_NAME for its base dodji
2012-11-01 19:53 ` [PATCH 09/13] Don't forget to protect 32 bytes aligned global variables dodji
2012-11-01 19:53 ` [PATCH 05/13] Allow asan at -O0 dodji
2012-11-01 19:53 ` [PATCH 11/13] Factorize condition insertion code out of build_check_stmt dodji
2012-11-01 19:53 ` [PATCH 07/13] Implement protection of global variables dodji
2012-11-01 19:53 ` [PATCH 12/13] Instrument built-in memory access function calls dodji
2012-11-01 19:54 ` [PATCH 04/13] Emit GIMPLE directly instead of gimplifying GENERIC dodji
2012-11-02 22:53 ` [PATCH 00/13] Request to merge Address Sanitizer in Dodji Seketeli
2012-11-02 22:56 ` [PATCH 01/10] Initial import of asan from the Google branch into trunk Dodji Seketeli
2012-11-06 17:04 ` Diego Novillo
2012-11-09 13:14 ` Tobias Burnus
2012-11-09 13:58 ` Jakub Jelinek
2012-11-09 16:53 ` Xinliang David Li
2012-11-09 17:13 ` Tobias Burnus
2012-11-09 17:18 ` Wei Mi
2012-11-12 11:09 ` [PATCH 03/11] Emit GIMPLE directly instead of gimplifying GENERIC Dodji Seketeli
2012-11-12 11:20 ` [PATCH 01/10] Initial import of asan from the Google branch into trunk Dodji Seketeli
2012-11-02 22:57 ` [PATCH 02/10] Initial asan cleanups Dodji Seketeli
2012-11-06 17:04 ` Diego Novillo
2012-11-12 11:12 ` Dodji Seketeli
2012-11-02 22:58 ` [PATCH 03/10] Emit GIMPLE directly instead of gimplifying GENERIC Dodji Seketeli
2012-11-06 17:08 ` Diego Novillo
2012-11-02 22:59 ` [PATCH 04/10] Allow asan at -O0 Dodji Seketeli
2012-11-06 17:12 ` Diego Novillo
2012-11-02 23:00 ` [PATCH 05/10] Implement protection of stack variables Dodji Seketeli
2012-11-06 17:22 ` Diego Novillo
2012-11-12 11:31 ` Dodji Seketeli
2012-11-12 11:51 ` Jakub Jelinek
2012-11-12 16:08 ` Dodji Seketeli
2012-11-02 23:01 ` [PATCH 06/10] Implement protection of global variables Dodji Seketeli
2012-11-06 17:27 ` Diego Novillo
2012-11-12 11:32 ` Dodji Seketeli
2012-11-02 23:02 ` [PATCH 07/10] Make build_check_stmt accept an SSA_NAME for its base Dodji Seketeli
2012-11-06 17:28 ` Diego Novillo
2012-11-02 23:03 ` [PATCH 08/10] Factorize condition insertion code out of build_check_stmt Dodji Seketeli
2012-11-05 15:50 ` Jakub Jelinek
2012-11-05 20:25 ` Dodji Seketeli
2012-11-06 17:30 ` Diego Novillo
2012-11-02 23:05 ` [PATCH 09/10] Instrument built-in memory access function calls Dodji Seketeli
2012-11-06 17:37 ` Diego Novillo
2012-11-12 11:40 ` Dodji Seketeli
2012-11-03 8:22 ` [PATCH 10/10] Import the asan runtime library into GCC tree Dodji Seketeli
[not found] ` <87fw4r7g8w.fsf_-_@redhat.com>
2012-11-06 17:41 ` Diego Novillo
2012-11-12 11:47 ` Dodji Seketeli
2012-11-12 18:59 ` H.J. Lu
2012-11-14 11:11 ` H.J. Lu
2012-11-14 11:42 ` H.J. Lu
2012-11-12 16:07 ` [PATCH 00/13] Request to merge Address Sanitizer in Dodji Seketeli
2012-11-12 16:21 ` Jakub Jelinek
2012-11-12 16:45 ` Tobias Burnus
2012-11-12 16:51 ` Konstantin Serebryany
2012-11-12 17:20 ` Jack Howarth
2012-11-12 17:34 ` Jack Howarth
2012-11-12 17:37 ` Tobias Burnus
2012-11-12 17:57 ` Jack Howarth
2012-11-12 17:55 ` Dodji Seketeli
2012-11-12 18:40 ` Jack Howarth
2012-11-12 20:39 ` H.J. Lu
2012-11-12 22:15 ` Ian Lance Taylor
2012-11-15 19:42 ` Jack Howarth [this message]
2012-11-15 23:42 ` Konstantin Serebryany
2012-11-16 8:27 ` Dodji Seketeli
2012-11-16 14:03 ` Jack Howarth
2012-11-16 15:57 ` Jack Howarth
2012-11-16 16:02 ` Jakub Jelinek
2012-11-16 16:47 ` Jack Howarth
2012-11-16 16:56 ` Alexander Potapenko
2012-11-16 17:06 ` Jack Howarth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121115194154.GA16078@bromo.med.uc.edu \
--to=howarth@bromo.med.uc.edu \
--cc=davidxl@google.com \
--cc=dnovillo@google.com \
--cc=dodji@redhat.com \
--cc=dodji@seketeli.org \
--cc=gcc-patches@gcc.gnu.org \
--cc=jakub@redhat.com \
--cc=konstantin.s.serebryany@gmail.com \
--cc=wmi@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).