public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug sanitizer/105336] New: truncated address sanitizer stack traces
@ 2022-04-21 14:37 avi at scylladb dot com
2022-04-21 14:53 ` [Bug sanitizer/105336] " avi at scylladb dot com
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-21 14:37 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
Bug ID: 105336
Summary: truncated address sanitizer stack traces
Product: gcc
Version: 11.3.1
Status: UNCONFIRMED
Severity: normal
Priority: P3
Component: sanitizer
Assignee: unassigned at gcc dot gnu.org
Reporter: avi at scylladb dot com
CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
jakub at gcc dot gnu.org, kcc at gcc dot gnu.org, marxin at gcc dot gnu.org
Target Milestone: ---
Trying to debug a program with gcc 11 branch
(d26c3e4f733fcb07d90680491dd1d7a9d08c4705), I get truncated asan stack traces:
seastar::internal::repeater<replica::table::seal_active_memtable(flush_permit&&)::{lambda(auto:1&)#2}::operator()<flush_permit>(flush_permit&)
const::{lambda()#1}>
=================================================================
==313819==ERROR: AddressSanitizer: heap-use-after-free on address
0x61400003f848 at pc 0x0000040627a3 bp 0x7fff62f15fb0 sp 0x7fff62f15fa8
READ of size 8 at 0x61400003f848 thread T0
#0 0x40627a2 in seastar::debug_shared_ptr_counter_type::check() const
seastar/include/seastar/core/shared_ptr_debug_helper.hh:63
#1 0x505eab6 in seastar::debug_shared_ptr_counter_type::operator long()
const seastar/include/seastar/core/shared_ptr_debug_helper.hh:40
#2 0x505eab6 in seastar::lw_shared_ptr<replica::memtable>::use_count()
const seastar/include/seastar/core/shared_ptr.hh:356
#3 0x505eab6 in operator() replica/table.cc:620
#4 0x5061947 in
invoke<replica::table::seal_active_memtable(flush_permit&&)::<lambda(seastar::future<>)>&,
seastar::future<void> > seastar/include/seastar/core/future.hh:2141
#5 0x5061947 in operator() seastar/include/seastar/core/future.hh:1658
#6 0x5061947 in call
seastar/include/seastar/util/noncopyable_function.hh:153
#7 0x45d1383 in seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>::operator()(seastar::future<void>&&) const
seastar/include/seastar/util/noncopyable_function.hh:209
#8 0x45d1383 in
seastar::future<void>::then_wrapped_nrvo<seastar::future<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>
>(seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)
const::{lambda()#1}::operator()() const
seastar/include/seastar/core/future.hh:1674
#9 0x45d1383 in void seastar::futurize<seastar::future<void>
>::satisfy_with_result_of<seastar::future<void>::then_wrapped_nrvo<seastar::future<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>
>(seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)
const::{lambda()#1}>(seastar::internal::promise_base_with_type<void>&&,
seastar::future<void>::then_wrapped_nrvo<seastar::future<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>
>(seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&) const::{lambda()#1}&&)
seastar/include/seastar/core/future.hh:2126
#10 0x45d2191 in
seastar::future<void>::then_wrapped_nrvo<seastar::future<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>
>(seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&) const
seastar/include/seastar/core/future.hh:1673
#11 0x45d2191 in
seastar::continuation<seastar::internal::promise_base_with_type<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>,
seastar::future<void>::then_wrapped_nrvo<seastar::future<void>,
seastar::noncopyable_function<seastar::future<void> (seastar::future<void>&&)>
>(seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&,
seastar::noncopyable_function<seastar::future<void>
(seastar::future<void>&&)>&,
seastar::future_state<seastar::internal::monostate>&&)#1},
void>::run_and_dispose() seastar/include/seastar/core/future.hh:773
#12 0x17fc8b74 in
seastar::reactor::run_tasks(seastar::reactor::task_queue&)
seastar/src/core/reactor.cc:2344
#13 0x17fcd0ec in seastar::reactor::run_some_tasks()
seastar/src/core/reactor.cc:2754
#14 0x17fd2b00 in seastar::reactor::do_run()
seastar/src/core/reactor.cc:2923
#15 0x17fceba8 in seastar::reactor::run() seastar/src/core/reactor.cc:2806
#16 0x17d0a3e0 in seastar::app_template::run_deprecated(int, char**,
std::function<void ()>&&) seastar/src/core/app-template.cc:265
#17 0x17d07eb0 in seastar::app_template::run(int, char**,
std::function<seastar::future<int> ()>&&) seastar/src/core/app-template.cc:156
#18 0x3d67f67 in scylla_main /home/avi/scylla/main.cc:531
#19 0x3dd04f2 in int std::__invoke_impl<int, int (*&)(int, char**), int,
char**>(std::__invoke_other, int (*&)(int, char**), int&&, char**&&)
/home/avi/gcc.coroutines/include/c++/11.3.1/bits/invoke.h:61
#20 0x3dd04f2 in std::enable_if<is_invocable_r_v<int, int (*&)(int,
char**), int, char**>, int>::type std::__invoke_r<int, int (*&)(int, char**),
int, char**>(int (*&)(int, char**), int&&, char**&&)
/home/avi/gcc.coroutines/include/c++/11.3.1/bits/invoke.h:114
#21 0x3dd04f2 in std::_Function_handler<int (int, char**), int (*)(int,
char**)>::_M_invoke(std::_Any_data const&, int&&, char**&&)
/home/avi/gcc.coroutines/include/c++/11.3.1/bits/std_function.h:290
#22 0x3d48f4b in std::function<int (int, char**)>::operator()(int, char**)
const /home/avi/gcc.coroutines/include/c++/11.3.1/bits/std_function.h:590
#23 0x3d48f4b in main /home/avi/scylla/main.cc:1577
#24 0x7f394d66eb74 in __libc_start_main (/lib64/libc.so.6+0x27b74)
#25 0x3c1642d in _start (/home/avi/scylla/build/debug/scylla+0x3c1642d)
0x61400003f848 is located 8 bytes inside of 408-byte region
[0x61400003f840,0x61400003f9d8)
freed by thread T0 here:
#0 0x7f394fb52f07 in operator delete(void*, unsigned long)
(/lib64/libasan.so.6+0xb0f07)
#1 0x4cf9bec in
seastar::internal::lw_shared_ptr_accessors_esft<replica::memtable>::dispose(replica::memtable*)
seastar/include/seastar/core/shared_ptr.hh:199
previously allocated by thread T0 here:
#0 0x7f394fb52087 in operator new(unsigned long)
(/lib64/libasan.so.6+0xb0087)
#1 0x494b54b in seastar::lw_shared_ptr<replica::memtable>
seastar::lw_shared_ptr<replica::memtable>::make<seastar::lw_shared_ptr<schema
const>, dirty_memory_manager&, replica::table_stats&, replica::memtable_list*,
seastar::scheduling_group&>(seastar::lw_shared_ptr<schema const>&&,
dirty_memory_manager&, replica::table_stats&, replica::memtable_list*&&,
seastar::scheduling_group&) seastar/include/seastar/core/shared_ptr.hh:267
#2 0x494b54b in seastar::lw_shared_ptr<replica::memtable>
seastar::make_lw_shared<replica::memtable, seastar::lw_shared_ptr<schema
const>, dirty_memory_manager&, replica::table_stats&, replica::memtable_list*,
seastar::scheduling_group&>(seastar::lw_shared_ptr<schema const>&&,
dirty_memory_manager&, replica::table_stats&, replica::memtable_list*&&,
seastar::scheduling_group&) seastar/include/seastar/core/shared_ptr.hh:417
#3 0x494b54b in replica::memtable_list::new_memtable()
replica/database.cc:1575
#4 0x60d000024217 (<unknown module>)
SUMMARY: AddressSanitizer: heap-use-after-free
seastar/include/seastar/core/shared_ptr_debug_helper.hh:63 in
seastar::debug_shared_ptr_counter_type::check() const
Shadow bytes around the buggy address:
0x0c287ffffeb0: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
0x0c287ffffec0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c287ffffed0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c287ffffee0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c287ffffef0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c287fffff00: fa fa fa fa fa fa fa fa fd[fd]fd fd fd fd fd fd
0x0c287fffff10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c287fffff20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c287fffff30: fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa
0x0c287fffff40: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c287fffff50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==313819==ABORTING
While the first trace is full, terminating in main(), the second is immediately
truncated and the third leads to a caller that is on the heap (the program does
not JIT).
Something is wrong in stack backtracing. I realize this is not enough
information to debug, but I can't think of what else I can provide.
Compiled with -Og -g -gz.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
@ 2022-04-21 14:53 ` avi at scylladb dot com
2022-04-22 6:07 ` rguenth at gcc dot gnu.org
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-21 14:53 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #1 from Avi Kivity <avi at scylladb dot com> ---
I guess I should mention the programs uses C++20 coroutines, and it could be
the case that debug information for coroutines is generated incorrectly.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
2022-04-21 14:53 ` [Bug sanitizer/105336] " avi at scylladb dot com
@ 2022-04-22 6:07 ` rguenth at gcc dot gnu.org
2022-04-23 16:34 ` avi at scylladb dot com
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: rguenth at gcc dot gnu.org @ 2022-04-22 6:07 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Can you provide a testcase that exhibits such behavior?
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
2022-04-21 14:53 ` [Bug sanitizer/105336] " avi at scylladb dot com
2022-04-22 6:07 ` rguenth at gcc dot gnu.org
@ 2022-04-23 16:34 ` avi at scylladb dot com
2022-04-24 12:16 ` avi at scylladb dot com
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-23 16:34 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #3 from Avi Kivity <avi at scylladb dot com> ---
I have a multi-gigabyte reproducer. Unfortunately it's part of a huge program
that didn't build with gcc until very recently. It will be quite a task to
reduce it.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
` (2 preceding siblings ...)
2022-04-23 16:34 ` avi at scylladb dot com
@ 2022-04-24 12:16 ` avi at scylladb dot com
2022-04-24 12:17 ` avi at scylladb dot com
2022-04-24 12:18 ` avi at scylladb dot com
5 siblings, 0 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-24 12:16 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #4 from Avi Kivity <avi at scylladb dot com> ---
Created attachment 52856
--> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52856&action=edit
intentionally buggy reproducer
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
` (3 preceding siblings ...)
2022-04-24 12:16 ` avi at scylladb dot com
@ 2022-04-24 12:17 ` avi at scylladb dot com
2022-04-24 12:18 ` avi at scylladb dot com
5 siblings, 0 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-24 12:17 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #5 from Avi Kivity <avi at scylladb dot com> ---
I reduced it to a few lines (attached, intentionally triggers use-after-free).
The culprit is -Og.
With
g++ coroutine-asan.cc -o coroutine-asan --std=c++20 -fsanitize=address -Og
I see
READ of size 8 at 0x607000000020 thread T0
#0 0x4018e2 in test() (/home/avi/tests/coroutine-asan+0x4018e2)
#1 0x40192b in main (/home/avi/tests/coroutine-asan+0x40192b)
#2 0x7fb2e0a1d58f in __libc_start_call_main (/lib64/libc.so.6+0x2d58f)
#3 0x7fb2e0a1d648 in __libc_start_main_alias_1 (/lib64/libc.so.6+0x2d648)
#4 0x401144 in _start (/home/avi/tests/coroutine-asan+0x401144)
0x607000000020 is located 0 bytes inside of 80-byte region
[0x607000000020,0x607000000070)
freed by thread T0 here:
#0 0x7fb2e0fdebc8 in operator delete(void*) (/lib64/libasan.so.8+0xbbbc8)
#1 0x40164f in
test_coroutine(test_coroutine(std::__n4861::coroutine_handle<void>&)::_Z14test_coroutineRNSt7__n486116coroutine_handleIvEE.Frame*)
[clone .actor] (/home/avi/tests/coroutine-asan+0x40164f)
#2 0x40200f (/home/avi/tests/coroutine-asan+0x40200f)
previously allocated by thread T0 here:
#0 0x7fb2e0fde188 in operator new(unsigned long)
(/lib64/libasan.so.8+0xbb188)
#1 0x4016c0 in test_coroutine(std::__n4861::coroutine_handle<void>&)
(/home/avi/tests/coroutine-asan+0x4016c0)
The stack traces are truncated, compared to
g++ coroutine-asan.cc -o coroutine-asan --std=c++20 -fsanitize=address
which yields
READ of size 8 at 0x607000000020 thread T0
#0 0x401cef in std::__n4861::coroutine_handle<void>::resume() const
(/home/avi/tests/coroutine-asan+0x401cef)
#1 0x401b0f in test() (/home/avi/tests/coroutine-asan+0x401b0f)
#2 0x401b85 in main (/home/avi/tests/coroutine-asan+0x401b85)
#3 0x7f79b8be958f in __libc_start_call_main (/lib64/libc.so.6+0x2d58f)
#4 0x7f79b8be9648 in __libc_start_main_alias_1 (/lib64/libc.so.6+0x2d648)
#5 0x4011a4 in _start (/home/avi/tests/coroutine-asan+0x4011a4)
0x607000000020 is located 0 bytes inside of 80-byte region
[0x607000000020,0x607000000070)
freed by thread T0 here:
#0 0x7f79b91aabc8 in operator delete(void*) (/lib64/libasan.so.8+0xbbbc8)
#1 0x4019a5 in
test_coroutine(test_coroutine(std::__n4861::coroutine_handle<void>&)::_Z14test_coroutineRNSt7__n486116coroutine_handleIvEE.Frame*)
[clone .actor] (/home/avi/tests/coroutine-asan+0x4019a5)
#2 0x401cf7 in std::__n4861::coroutine_handle<void>::resume() const
(/home/avi/tests/coroutine-asan+0x401cf7)
#3 0x401b0f in test() (/home/avi/tests/coroutine-asan+0x401b0f)
#4 0x401b85 in main (/home/avi/tests/coroutine-asan+0x401b85)
#5 0x7f79b8be958f in __libc_start_call_main (/lib64/libc.so.6+0x2d58f)
previously allocated by thread T0 here:
#0 0x7f79b91aa188 in operator new(unsigned long)
(/lib64/libasan.so.8+0xbb188)
#1 0x401293 in test_coroutine(std::__n4861::coroutine_handle<void>&)
(/home/avi/tests/coroutine-asan+0x401293)
#2 0x401af9 in test() (/home/avi/tests/coroutine-asan+0x401af9)
#3 0x401b85 in main (/home/avi/tests/coroutine-asan+0x401b85)
#4 0x7f79b8be958f in __libc_start_call_main (/lib64/libc.so.6+0x2d58f)
The traces go all the away to main.
^ permalink raw reply [flat|nested] 7+ messages in thread
* [Bug sanitizer/105336] truncated address sanitizer stack traces
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
` (4 preceding siblings ...)
2022-04-24 12:17 ` avi at scylladb dot com
@ 2022-04-24 12:18 ` avi at scylladb dot com
5 siblings, 0 replies; 7+ messages in thread
From: avi at scylladb dot com @ 2022-04-24 12:18 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105336
--- Comment #6 from Avi Kivity <avi at scylladb dot com> ---
(the reproducer was executed by gcc 12 prerelease, not gcc 11)
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-04-24 12:18 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-21 14:37 [Bug sanitizer/105336] New: truncated address sanitizer stack traces avi at scylladb dot com
2022-04-21 14:53 ` [Bug sanitizer/105336] " avi at scylladb dot com
2022-04-22 6:07 ` rguenth at gcc dot gnu.org
2022-04-23 16:34 ` avi at scylladb dot com
2022-04-24 12:16 ` avi at scylladb dot com
2022-04-24 12:17 ` avi at scylladb dot com
2022-04-24 12:18 ` avi at scylladb dot com
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).