* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
@ 2014-01-09 9:45 ` rguenth at gcc dot gnu.org
2014-01-09 15:30 ` hjl.tools at gmail dot com
` (24 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: rguenth at gcc dot gnu.org @ 2014-01-09 9:45 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
Richard Biener <rguenth at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|--- |4.9.0
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
2014-01-09 9:45 ` [Bug sanitizer/59733] " rguenth at gcc dot gnu.org
@ 2014-01-09 15:30 ` hjl.tools at gmail dot com
2014-01-09 15:34 ` jakub at gcc dot gnu.org
` (23 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-09 15:30 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
H.J. Lu <hjl.tools at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEW
Last reconfirmed| |2014-01-09
CC| |kirill.yukhin at intel dot com
Ever confirmed|0 |1
--- Comment #1 from H.J. Lu <hjl.tools at gmail dot com> ---
It is caused by r206221.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
2014-01-09 9:45 ` [Bug sanitizer/59733] " rguenth at gcc dot gnu.org
2014-01-09 15:30 ` hjl.tools at gmail dot com
@ 2014-01-09 15:34 ` jakub at gcc dot gnu.org
2014-01-09 18:25 ` hjl.tools at gmail dot com
` (22 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: jakub at gcc dot gnu.org @ 2014-01-09 15:34 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #2 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
It doesn't matter which change triggered it, this looks like libsanitizer
internal error, it assumes no internal allocations will need to be large, but
apparently some are. So, the question is what is the backtrace for this error.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (2 preceding siblings ...)
2014-01-09 15:34 ` jakub at gcc dot gnu.org
@ 2014-01-09 18:25 ` hjl.tools at gmail dot com
2014-01-09 19:30 ` jakub at gcc dot gnu.org
` (21 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-09 18:25 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> ---
Breakpoint 6, __sanitizer::Report (
format=format@entry=0x2821768 "ERROR: %s failed to allocate 0x%zx (%zd)
bytes of %s: %d\n")
---Type <return> to continue, or q <return> to quit---
at
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_printf.cc:276
276 void Report(const char *format, ...) {
(gdb) bt
#0 __sanitizer::Report (
format=format@entry=0x2821768 "ERROR: %s failed to allocate 0x%zx (%zd)
bytes of %s: %d\n")
at
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_printf.cc:276
#1 0x000000000062a660 in __sanitizer::MmapOrDie (size=<optimized out>,
size@entry=151552, mem_type=mem_type@entry=0x2817d54 "LargeMmapAllocator")
at
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_posix.cc:64
#2 0x00000000005e1cf4 in Allocate (alignment=8, size=146992,
stat=0x36d0d58 <__asan::allocator+2097752>,
this=0x34d0b08 <__asan::allocator+8>)
at
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_allocator.h:1009
#3 Allocate (cleared=false, alignment=8, size=146992, cache=0x7ffff7ab00c8,
this=0x34d0b00 <__asan::allocator>)
at
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_allocator.h:1251
#4 __asan::Allocate (size=<optimized out>, size@entry=142896,
alignment=<optimized out>, alignment@entry=8,
stack=stack@entry=0x7fffffffc640,
alloc_type=alloc_type@entry=__asan::FROM_MALLOC,
can_fill=can_fill@entry=true)
---Type <return> to continue, or q <return> to quit---
at /export/gnu/import/git/gcc/libsanitizer/asan/asan_allocator2.cc:352
#5 0x00000000005e2578 in __asan::asan_malloc (size=size@entry=142896,
stack=stack@entry=0x7fffffffc640)
at /export/gnu/import/git/gcc/libsanitizer/asan/asan_allocator2.cc:619
#6 0x000000000061ad92 in __interceptor_malloc (size=size@entry=142896)
at /export/gnu/import/git/gcc/libsanitizer/asan/asan_malloc_linux.cc:73
#7 0x0000000002808a88 in xmalloc (size=142896)
at /export/gnu/import/git/gcc/libiberty/xmalloc.c:147
#8 0x0000000000ea4e1d in calc_dfs_tree_nonrec (di=0x7fffffffd090,
bb=<basic_block 0x7fffde747410 (3598)>, reverse=<optimized out>)
at /export/gnu/import/git/gcc/gcc/dominance.c:237
#9 0x0000000000ea5c0c in calc_dfs_tree (di=di@entry=0x7fffffffd090,
reverse=reverse@entry=true)
at /export/gnu/import/git/gcc/gcc/dominance.c:374
#10 0x0000000000ea98b1 in calculate_dominance_info (
dir=dir@entry=CDI_POST_DOMINATORS)
at /export/gnu/import/git/gcc/gcc/dominance.c:662
#11 0x0000000001bc8a3f in execute_late_warn_uninitialized ()
at /export/gnu/import/git/gcc/gcc/tree-ssa-uninit.c:2297
#12 (anonymous namespace)::pass_late_warn_uninitialized::execute (
this=<optimized out>)
at /export/gnu/import/git/gcc/gcc/tree-ssa-uninit.c:2388
#13 0x00000000014ff145 in execute_one_pass (
---Type <return> to continue, or q <return> to quit---
pass=pass@entry=<opt_pass* 0x608000007720 "uninit"(145)>)
at /export/gnu/import/git/gcc/gcc/passes.c:2226
#14 0x00000000014ffb51 in execute_pass_list (
pass=<opt_pass* 0x608000007720 "uninit"(145)>)
at /export/gnu/import/git/gcc/gcc/passes.c:2279
#15 0x00000000014ffb77 in execute_pass_list (
pass=<opt_pass* 0x60800000a420 "*all_optimizations"(-1)>)
at /export/gnu/import/git/gcc/gcc/passes.c:2280
#16 0x0000000000dff83b in expand_function (
node=node@entry=<cgraph_node* 0x7fffe0b567b0 "peephole2_insns">)
at /export/gnu/import/git/gcc/gcc/cgraphunit.c:1764
#17 0x0000000000e04c11 in expand_all_functions ()
at /export/gnu/import/git/gcc/gcc/cgraphunit.c:1898
#18 compile () at /export/gnu/import/git/gcc/gcc/cgraphunit.c:2242
#19 0x0000000000e065eb in finalize_compilation_unit ()
at /export/gnu/import/git/gcc/gcc/cgraphunit.c:2319
#20 0x000000000089cb53 in cp_write_global_declarations ()
at /export/gnu/import/git/gcc/gcc/cp/decl2.c:4431
#21 0x00000000016f0135 in compile_file ()
at /export/gnu/import/git/gcc/gcc/toplev.c:562
#22 0x00000000016f505c in do_compile ()
at /export/gnu/import/git/gcc/gcc/toplev.c:1914
#23 toplev_main (argc=76, argv=0x7fffffffd9d8)
---Type <return> to continue, or q <return> to quit---
at /export/gnu/import/git/gcc/gcc/toplev.c:1990
#24 0x0000003cdda21b45 in __libc_start_main () from /lib64/libc.so.6
#25 0x00000000005ded71 in _start ()
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (3 preceding siblings ...)
2014-01-09 18:25 ` hjl.tools at gmail dot com
@ 2014-01-09 19:30 ` jakub at gcc dot gnu.org
2014-01-09 20:11 ` hjl.tools at gmail dot com
` (20 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: jakub at gcc dot gnu.org @ 2014-01-09 19:30 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
BTW, are you sure you don't have some ulimit imposed limits (say on virtual
address space)?
Can you strace it (perhaps just strace -o log -e mmap,munmap )?
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (4 preceding siblings ...)
2014-01-09 19:30 ` jakub at gcc dot gnu.org
@ 2014-01-09 20:11 ` hjl.tools at gmail dot com
2014-01-09 20:52 ` hjl.tools at gmail dot com
` (19 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-09 20:11 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #5 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Jakub Jelinek from comment #4)
> BTW, are you sure you don't have some ulimit imposed limits (say on virtual
> address space)?
[hjl@gnu-mic-2 gcc]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 192280
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 192280
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[hjl@gnu-mic-2 gcc]$
> Can you strace it (perhaps just strace -o log -e mmap,munmap )?
I will do it.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (5 preceding siblings ...)
2014-01-09 20:11 ` hjl.tools at gmail dot com
@ 2014-01-09 20:52 ` hjl.tools at gmail dot com
2014-01-23 19:38 ` jakub at gcc dot gnu.org
` (18 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-09 20:52 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #6 from H.J. Lu <hjl.tools at gmail dot com> ---
Created attachment 31788
--> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31788&action=edit
strace -o log -e mmap,munmap
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (6 preceding siblings ...)
2014-01-09 20:52 ` hjl.tools at gmail dot com
@ 2014-01-23 19:38 ` jakub at gcc dot gnu.org
2014-01-23 20:48 ` hjl.tools at gmail dot com
` (17 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: jakub at gcc dot gnu.org @ 2014-01-23 19:38 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #7 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Can't reproduce this, at least not if I use sufficiently lower make -jN factor
(used -j2). With that
../configure --enable-languages=c,c++,fortran,java,lto,objc,obj-c++,go
--with-build-config=bootstrap-asan --disable-werror
built all the way through compare and died only on fixincl not being asan
bootstrap compatible.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (7 preceding siblings ...)
2014-01-23 19:38 ` jakub at gcc dot gnu.org
@ 2014-01-23 20:48 ` hjl.tools at gmail dot com
2014-01-24 12:21 ` kcc at gcc dot gnu.org
` (16 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-23 20:48 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #8 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Jakub Jelinek from comment #7)
> Can't reproduce this, at least not if I use sufficiently lower make -jN
> factor (used -j2). With that
> ../configure --enable-languages=c,c++,fortran,java,lto,objc,obj-c++,go
> --with-build-config=bootstrap-asan --disable-werror
> built all the way through compare and died only on fixincl not being asan
> bootstrap compatible.
It still fails for me. It has nothing to do with "make -jN". When I
ran it by hand, I got
/export/build/gnu/gcc-asan/build-x86_64-linux/./prev-gcc/xg++
-B/export/build/gnu/gcc-asan/build-x86_64-linux/./prev-gcc/
-B/usr/local/x86_64-unknown-linux-gnu/bin/ -nostdinc++
-B/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs
-B/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs
-I/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu
-I/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include
-I/export/gnu/import/git/gcc-misc/libstdc++-v3/libsupc++
-L/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs
-L/export/build/gnu/gcc-asan/build-x86_64-linux/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs
-c -O2 -g -fsanitize=address -DIN_GCC -fno-exceptions -fno-rtti
-fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long
-Wno-variadic-macros -Wno-overlength-strings -fno-common -DHAVE_CONFIG_H -I.
-I. -I/export/gnu/import/git/gcc-misc/gcc
-I/export/gnu/import/git/gcc-misc/gcc/.
-I/export/gnu/import/git/gcc-misc/gcc/../include
-I/export/gnu/import/git/gcc-misc/gcc/../libcpp/include
-I/export/gnu/import/git/gcc-misc/gcc/../libdecnumber
-I/export/gnu/import/git/gcc-misc/gcc/../libdecnumber/bid -I../libdecnumber
-I/export/gnu/import/git/gcc-misc/gcc/../libbacktrace -o insn-recog.o -MT
insn-recog.o -MMD -MP -MF ./.deps/insn-recog.TPo insn-recog.c
==19786==ERROR: AddressSanitizer failed to allocate 0x24000 (147456) bytes of
LargeMmapAllocator: 12
ERROR: Failed to mmap
I did it on a Fedora 19 machine with 24GB RAM and 48GB swap. I configured GCC
with
--enable-clocale=gnu --with-system-zlib --with-demangler-in-ld
--enable-languages=c,c++ --prefix=/usr/local --enable-gnu-indirect-function
--disable-werror --with-arch=corei7 --with-cpu=slm
--with-build-config=bootstrap-asan --with-fpmath=sse
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (8 preceding siblings ...)
2014-01-23 20:48 ` hjl.tools at gmail dot com
@ 2014-01-24 12:21 ` kcc at gcc dot gnu.org
2014-01-24 12:39 ` kcc at gcc dot gnu.org
` (15 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 12:21 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #9 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
> It still fails for me. It has nothing to do with "make -jN". When I
I tried running this on the fresh gcc trunk.
During the 3-rd stage I get this:
checking how to run the C preprocessor... ../libiberty/libiberty.a(regex.o): In
function `byte_compile_range':
... gcc-boot/libiberty/../../gcc/libiberty/regex.c:4495: undefined reference to
`__asan_report_load1'
Before that the builds works fine, but compilation of insn-recog.c
with asan-ified gcc is indeed expensive: takes ~8 minutes and 4+Gb RAM
This excessive memory usage may be the reason of your OOM in case you
have some sort of limit (Are you by any chance building in x32 mode?)
non-asanified build of insn-recog.c takes ~3.5 minutes and 700M RAM.
The asan slowdown here is reasonable, but the memory overhead is surprisingly
large...
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (9 preceding siblings ...)
2014-01-24 12:21 ` kcc at gcc dot gnu.org
@ 2014-01-24 12:39 ` kcc at gcc dot gnu.org
2014-01-24 12:46 ` hjl.tools at gmail dot com
` (14 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 12:39 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #10 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
H.J., can you send the contents of /proc/PID/{maps,smaps,status}
of the failing process before it dies?
(you can use ASAN_OPTIONS=sleep_before_dying=1000)
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (10 preceding siblings ...)
2014-01-24 12:39 ` kcc at gcc dot gnu.org
@ 2014-01-24 12:46 ` hjl.tools at gmail dot com
2014-01-24 13:16 ` hjl.tools at gmail dot com
` (13 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-24 12:46 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #11 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Kostya Serebryany from comment #9)
> > It still fails for me. It has nothing to do with "make -jN". When I
>
> I tried running this on the fresh gcc trunk.
>
> During the 3-rd stage I get this:
> checking how to run the C preprocessor... ../libiberty/libiberty.a(regex.o):
> In function `byte_compile_range':
> ... gcc-boot/libiberty/../../gcc/libiberty/regex.c:4495: undefined reference
> to `__asan_report_load1'
Please try hjl/asan branch at
http://gcc.gnu.org/git/?p=gcc.git;a=summary
It fixes this problem among others.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (11 preceding siblings ...)
2014-01-24 12:46 ` hjl.tools at gmail dot com
@ 2014-01-24 13:16 ` hjl.tools at gmail dot com
2014-01-24 13:17 ` kcc at gcc dot gnu.org
` (12 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-24 13:16 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #12 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Kostya Serebryany from comment #10)
> H.J., can you send the contents of /proc/PID/{maps,smaps,status}
> of the failing process before it dies?
> (you can use ASAN_OPTIONS=sleep_before_dying=1000)
If you run cc1plus under GDB, you will get
Breakpoint 6, __sanitizer::Report (
format=format@entry=0x284f6f0 "ERROR: %s failed to allocate 0x%zx (%zd)
bytes of %s: %d\n")
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (12 preceding siblings ...)
2014-01-24 13:16 ` hjl.tools at gmail dot com
@ 2014-01-24 13:17 ` kcc at gcc dot gnu.org
2014-01-24 13:19 ` hjl.tools at gmail dot com
` (11 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 13:17 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #13 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
> If you run cc1plus under GDB, you will get
>
> Breakpoint 6, __sanitizer::Report (
> format=format@entry=0x284f6f0 "ERROR: %s failed to allocate 0x%zx (%zd)
> bytes of %s: %d\n")
This happens due to OOM, we need to understand why OOM happens.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (13 preceding siblings ...)
2014-01-24 13:17 ` kcc at gcc dot gnu.org
@ 2014-01-24 13:19 ` hjl.tools at gmail dot com
2014-01-24 13:25 ` kcc at gcc dot gnu.org
` (10 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-24 13:19 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #14 from H.J. Lu <hjl.tools at gmail dot com> ---
Created attachment 31946
--> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31946&action=edit
/proc/cc1plus/{maps,smaps,status}
This is the output of "cat /proc/cc1plus/{maps,smaps,status}".
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (14 preceding siblings ...)
2014-01-24 13:19 ` hjl.tools at gmail dot com
@ 2014-01-24 13:25 ` kcc at gcc dot gnu.org
2014-01-24 13:28 ` hjl.tools at gmail dot com
` (9 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 13:25 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #15 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
This looks very weird:
602001200000-602001210000 rw-p 00000000 00:00 0
602001210000-602001220000 rw-p 00000000 00:00 0
We have two adjacent mappings with the same perms
which are not merged by the kernel into one.
This is not what I typically see on my linux.
Is this some kind of modified kernel config or something?
If your kernel behaves this way no winder that mmap eventually fails.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (15 preceding siblings ...)
2014-01-24 13:25 ` kcc at gcc dot gnu.org
@ 2014-01-24 13:28 ` hjl.tools at gmail dot com
2014-01-24 14:37 ` kcc at gcc dot gnu.org
` (8 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: hjl.tools at gmail dot com @ 2014-01-24 13:28 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #16 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to Kostya Serebryany from comment #15)
> This looks very weird:
> 602001200000-602001210000 rw-p 00000000 00:00 0
> 602001210000-602001220000 rw-p 00000000 00:00 0
>
> We have two adjacent mappings with the same perms
> which are not merged by the kernel into one.
> This is not what I typically see on my linux.
> Is this some kind of modified kernel config or something?
>
> If your kernel behaves this way no winder that mmap eventually fails.
I am running 3.12.8-200.fc19.x86_64 from Fedora 19.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (16 preceding siblings ...)
2014-01-24 13:28 ` hjl.tools at gmail dot com
@ 2014-01-24 14:37 ` kcc at gcc dot gnu.org
2014-01-24 14:40 ` kcc at gcc dot gnu.org
` (7 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 14:37 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #17 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
When I look at my /proc/PID/maps of cc1plus, I see this:
...
610000000000-610000030000 rw-p 00000000 00:00 0
610000030000-611000000000 ---p 00000000 00:00 0
611000000000-611000980000 rw-p 00000000 00:00 0
611000980000-612000000000 ---p 00000000 00:00 0
612000000000-6120001b0000 rw-p 00000000 00:00 0
6120001b0000-613000000000 ---p 00000000 00:00 0
613000000000-613000370000 rw-p 00000000 00:00 0
613000370000-614000000000 ---p 00000000 00:00 0
614000000000-6140001a0000 rw-p 00000000 00:00 0
6140001a0000-615000000000 ---p 00000000 00:00 0
615000000000-615000460000 rw-p 00000000 00:00 0
...
As you can see, there are no adjacent mappings with same perms.
Here is a tiny test:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
int main() {
static volatile void *x;
for (int i = 0; i < 1000000; i++)
x = malloc(1000);
printf("PID %d\n", getpid());
sleep(100);
}
% clang -fsanitize=address nmalloc.c ; ./a.out
The mapping at the end looks like this:
...
2008fff7000-10007fff8000 rw-p 00000000 00:00 0
600000000000-619000000000 ---p 00000000 00:00 0
619000000000-61904c4c0000 rw-p 00000000 00:00 0
61904c4c0000-640000000000 ---p 00000000 00:00 0
640000000000-640000003000 rw-p 00000000 00:00 0
7f434fff7000-7f434fffb000 rw-p 00000000 00:00 0
...
Note: only 4 mappings that start with 6.
I bet that on your OS this test will fail under asan; please check.
I don't know what makes Linux behave this way, we haven't seen it before.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (17 preceding siblings ...)
2014-01-24 14:37 ` kcc at gcc dot gnu.org
@ 2014-01-24 14:40 ` kcc at gcc dot gnu.org
2014-01-24 14:43 ` kcc at gcc dot gnu.org
` (6 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 14:40 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
Kostya Serebryany <kcc at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |eugeni.stepanov at gmail dot com
--- Comment #18 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
eugenis@ says he sees something similar on Ubuntu 13.10.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (18 preceding siblings ...)
2014-01-24 14:40 ` kcc at gcc dot gnu.org
@ 2014-01-24 14:43 ` kcc at gcc dot gnu.org
2014-01-24 14:52 ` eugeni.stepanov at gmail dot com
` (5 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 14:43 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #19 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
(In reply to Kostya Serebryany from comment #18)
> eugenis@ says he sees something similar on Ubuntu 13.10.
Actually, only on Ubuntu 14.04 candidate.
Looks like a fresh regression in Kernel.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (19 preceding siblings ...)
2014-01-24 14:43 ` kcc at gcc dot gnu.org
@ 2014-01-24 14:52 ` eugeni.stepanov at gmail dot com
2014-01-24 14:56 ` eugeni.stepanov at gmail dot com
` (4 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: eugeni.stepanov at gmail dot com @ 2014-01-24 14:52 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #20 from Evgeniy Stepanov <eugeni.stepanov at gmail dot com> ---
I can confirm that the issue can be reproduced on 3.13.0-5-generic (ubuntu
14.04 candidate), and can not be reproduced on 3.11.0-15-generic (ubuntu
13.10).
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (20 preceding siblings ...)
2014-01-24 14:52 ` eugeni.stepanov at gmail dot com
@ 2014-01-24 14:56 ` eugeni.stepanov at gmail dot com
2014-01-24 15:16 ` kcc at gcc dot gnu.org
` (3 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: eugeni.stepanov at gmail dot com @ 2014-01-24 14:56 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #21 from Evgeniy Stepanov <eugeni.stepanov at gmail dot com> ---
A reproducer without ASan:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
int main() {
char * p = (char*)0x61904c1e0000;
for (int i = 0; i < 100; ++i)
mmap(p + i * 4096, 4096, PROT_WRITE | PROT_READ, MAP_PRIVATE |
MAP_ANONYMOUS, 0, 0);
printf("PID %d\n", getpid());
sleep(100);
}
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (21 preceding siblings ...)
2014-01-24 14:56 ` eugeni.stepanov at gmail dot com
@ 2014-01-24 15:16 ` kcc at gcc dot gnu.org
2014-01-27 7:21 ` kcc at gcc dot gnu.org
` (2 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-24 15:16 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #22 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
A quick workaround would be to change
static const uptr kUserMapSize = 1 << 16;
in sanitizer_common/sanitizer_allocator.h
to something like 17 or 18.
We can also try using mremap.
However I'd like to hear from kernel experts whether this is considered
a kernel bug or a feature.
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (22 preceding siblings ...)
2014-01-24 15:16 ` kcc at gcc dot gnu.org
@ 2014-01-27 7:21 ` kcc at gcc dot gnu.org
2014-01-27 10:00 ` kcc at gcc dot gnu.org
2014-01-27 13:15 ` kcc at gcc dot gnu.org
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-27 7:21 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
--- Comment #23 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
Relevant Kernel bug: https://bugzilla.kernel.org/show_bug.cgi?id=67651
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (23 preceding siblings ...)
2014-01-27 7:21 ` kcc at gcc dot gnu.org
@ 2014-01-27 10:00 ` kcc at gcc dot gnu.org
2014-01-27 13:15 ` kcc at gcc dot gnu.org
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-27 10:00 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
Kostya Serebryany <kcc at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED
Assignee|unassigned at gcc dot gnu.org |kcc at gcc dot gnu.org
Target Milestone|4.9.0 |---
--- Comment #24 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
I increased the mmap granularity in sanitizer's allocator to
allow our bot on ubuntu pre-14.04 to pass.
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/sanitizer_allocator.h?r1=200197&r2=200196&pathrev=200197
Feel free to apply the same change to GCC trunk.
Once the kernel bug is fixed and the fix reaches ubuntu 14.04 tree
we will likely revert this change.
There is nothing else we can do on sanitizer side -- the kernel bug
is significant (e.g. it makes gimp crash) and so such system is unusable
for many other purposes.
Let's keep this bug open so that we don't forget to revert the workaround.
FTR, here is a program that detects the presence of a broken kernel:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
#include <assert.h>
int main() {
char *p = (char*)0x600000000000;
size_t i;
for (i = 0; i < 100000; i++) {
void *addr = p + i * 4096;
void *ret = mmap(addr, 4096, PROT_WRITE | PROT_READ,
MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
if (addr != ret) {
fprintf(stderr, "failed at iteration %zd\n", i);
char command[100];
snprintf(command, sizeof(command), "cat /proc/%d/maps | head -30",
getpid());
system(command);
return 1;
}
}
}
On a broken kernel it will print something like this:
failed at iteration 65514
00400000-00401000 r-xp 00000000 fc:00 20991316 /tmp/a.out
00600000-00601000 r--p 00000000 fc:00 20991316 /tmp/a.out
00601000-00602000 rw-p 00001000 fc:00 20991316 /tmp/a.out
600000000000-600000001000 rw-p 00000000 00:00 0
600000001000-600000002000 rw-p 00000000 00:00 0
600000002000-600000003000 rw-p 00000000 00:00 0
600000003000-600000004000 rw-p 00000000 00:00 0
600000004000-600000005000 rw-p 00000000 00:00 0
...
^ permalink raw reply [flat|nested] 27+ messages in thread* [Bug sanitizer/59733] [4.9 Regression] bootstrap-asan failed to bootstrap
2014-01-09 1:01 [Bug sanitizer/59733] New: [4.9 Regression] bootstrap-asan failed to bootstrap hjl.tools at gmail dot com
` (24 preceding siblings ...)
2014-01-27 10:00 ` kcc at gcc dot gnu.org
@ 2014-01-27 13:15 ` kcc at gcc dot gnu.org
25 siblings, 0 replies; 27+ messages in thread
From: kcc at gcc dot gnu.org @ 2014-01-27 13:15 UTC (permalink / raw)
To: gcc-bugs
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59733
Kostya Serebryany <kcc at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|ASSIGNED |RESOLVED
Resolution|--- |WONTFIX
--- Comment #25 from Kostya Serebryany <kcc at gcc dot gnu.org> ---
Even with this workaround asan is almost unusable with the buggy kernel.
I suspect that when a process has too many mappings it gets too slow.
The machine where we have our clang-asan bootstrap gets very slow and
unreliable.
We have reverted the kernel on that bot
(http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/)
and reverted the above workaround in llvm trunk.
http://llvm.org/viewvc/llvm-project?rev=200217&view=rev
The kernel fix went into Linus's tree for 3.14 last week,
and was tagged for stable backport. Nothing else we could do here.
^ permalink raw reply [flat|nested] 27+ messages in thread