From: "agentzh at gmail dot com" <sourceware-bugzilla@sourceware.org>
To: systemtap@sourceware.org
Subject: [Bug runtime/31013] New: Use of sleeping _stp_stat_del() operations in atomic contexts when cleaning up for -t
Date: Mon, 30 Oct 2023 21:17:16 +0000 [thread overview]
Message-ID: <bug-31013-6586@http.sourceware.org/bugzilla/> (raw)
https://sourceware.org/bugzilla/show_bug.cgi?id=31013
Bug ID: 31013
Summary: Use of sleeping _stp_stat_del() operations in atomic
contexts when cleaning up for -t
Product: systemtap
Version: unspecified
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: runtime
Assignee: systemtap at sourceware dot org
Reporter: agentzh at gmail dot com
Target Milestone: ---
When stap's -t option is specified, the stat data needed by the timing stats
are freed in atomic contexts (with preemption disabled), which may cause kernel
deadlocks.
A debug kernel reports this:
```
[ 2437.320105] BUG: sleeping function called from invalid context at
mm/vmalloc.c:2359
[ 2437.320146] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 516958,
name: stapio
[ 2437.320152] 3 locks held by stapio/516958:
[ 2437.320157] #0: ffff888179939500 (&f->f_pos_lock){+.+.}-{3:3}, at:
__fdget_pos+0xa7/0xc0
[ 2437.320188] #1: ffff88812214a498 (sb_writers#3){.+.+}-{0:0}, at:
ksys_write+0xe9/0x1b0
[ 2437.320218] #2: ffffffffc1157a90 (cmd_mutex#6){+.+.}-{3:3}, at:
_stp_ctl_write_cmd+0x230/0x3004 [stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320253] CPU: 9 PID: 516958 Comm: stapio Tainted: G W OE
5.11.22-100.orinc.fc32.x86_64+debug #1
[ 2437.320259] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
1.15.0-1.fc35 04/01/2014
[ 2437.320264] Call Trace:
[ 2437.320270] dump_stack+0xae/0xe5
[ 2437.320282] ___might_sleep.cold+0x150/0x17e
[ 2437.320292] vfree+0x4b/0x60
[ 2437.320299] _stp_mem_debug_free+0x46f/0x4b7
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320314] _stp_vfree+0xf/0x10
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320322] _stp_stat_free+0xc2/0x163
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320338] _stp_stat_del+0xf/0x10
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320354] _stp_cleanup_and_exit+0x56e/0xa97
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320372] ? stap_stop_task_finder+0x73c/0x73c
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320395] _stp_ctl_write_cmd+0x3be/0x3004
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320409] ? lock_acquire+0x1cc/0x780
[ 2437.320416] ? ksys_write+0xe9/0x1b0
[ 2437.320428] ? _stp_cleanup_and_exit+0xa97/0xa97
[stap_471641e12a7967ee269f819958d850_516958]
[ 2437.320437] ? lockdep_hardirqs_on_prepare+0x3f0/0x3f0
[ 2437.320442] ? _cond_resched+0x15/0x30
[ 2437.320448] ? inode_security+0x56/0xf0
[ 2437.320457] ? selinux_file_permission+0x30a/0x3f0
[ 2437.320471] proc_reg_write+0x1ac/0x270
[ 2437.320485] vfs_write+0x180/0x810
[ 2437.320500] ksys_write+0xe9/0x1b0
[ 2437.320507] ? __ia32_sys_read+0xb0/0xb0
[ 2437.320516] ? rcu_read_lock_sched_held+0x3f/0x80
[ 2437.320530] do_syscall_64+0x33/0x40
[ 2437.320537] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 2437.320543] RIP: 0033:0x7f789872b90f
[ 2437.320550] Code: 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 29 fd ff ff 48
8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 b8 01 00 00 00 0f 05 <48> 3d 00
f0 ff ff 77 31 44 89 c7 48 89 44 24 08 e8 5c fd ff ff 48
[ 2437.320555] RSP: 002b:00007ffc1c3ce360 EFLAGS: 00000246 ORIG_RAX:
0000000000000001
[ 2437.320564] RAX: ffffffffffffffda RBX: ffffffffffffffff RCX:
00007f789872b90f
[ 2437.320568] RDX: 0000000000000004 RSI: 00007ffc1c3ce398 RDI:
0000000000000004
[ 2437.320572] RBP: 0000000000000000 R08: 0000000000000002 R09:
00007ffc1c3cf100
[ 2437.320577] R10: 0000000000000000 R11: 0000000000000246 R12:
00007ffc1c3cf170
[ 2437.320581] R13: 0000000000000001 R14: 000000000000000b R15:
00007ffc1c3cf1f4
```
This is because the `_stp_cleanup_and_exit` function in the runtime explicitly
disables preemption for `_stp_printf`. But we should really temporarily
re-enable premption for sleeping operations like `_stp_stat_del()`.
--
You are receiving this mail because:
You are the assignee for the bug.
next reply other threads:[~2023-10-30 21:17 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-30 21:17 agentzh at gmail dot com [this message]
2023-10-30 22:06 ` [Bug runtime/31013] " agentzh at gmail dot com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bug-31013-6586@http.sourceware.org/bugzilla/ \
--to=sourceware-bugzilla@sourceware.org \
--cc=systemtap@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).