From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 97CA43858D28; Mon, 30 Oct 2023 21:17:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 97CA43858D28 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1698700636; bh=KSCuKaGLqCKnoSyQItElSP22AppuGTdNAYKYNBX48v0=; h=From:To:Subject:Date:From; b=bCcTaGeKI4HFHFul4cZnfM+Vpsk/ycTDp+bKnrSPckR/G+POZKYSe/o0zSbGHWxpy xd5EThtbxOfYHUeqybkykuyP/2207ZdM9Hfi8UNhkc5XPsnROoMe/QRiAKad2DjYF4 w6f3x461yRsvnofTH0zm4Gpo5P6O3TPdZqcBwTPE= From: "agentzh at gmail dot com" To: systemtap@sourceware.org Subject: [Bug runtime/31013] New: Use of sleeping _stp_stat_del() operations in atomic contexts when cleaning up for -t Date: Mon, 30 Oct 2023 21:17:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: systemtap X-Bugzilla-Component: runtime X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: agentzh at gmail dot com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: systemtap at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 List-Id: https://sourceware.org/bugzilla/show_bug.cgi?id=3D31013 Bug ID: 31013 Summary: Use of sleeping _stp_stat_del() operations in atomic contexts when cleaning up for -t Product: systemtap Version: unspecified Status: UNCONFIRMED Severity: normal Priority: P2 Component: runtime Assignee: systemtap at sourceware dot org Reporter: agentzh at gmail dot com Target Milestone: --- When stap's -t option is specified, the stat data needed by the timing stats are freed in atomic contexts (with preemption disabled), which may cause ke= rnel deadlocks. A debug kernel reports this: ``` [ 2437.320105] BUG: sleeping function called from invalid context at mm/vmalloc.c:2359 [ 2437.320146] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 51695= 8, name: stapio [ 2437.320152] 3 locks held by stapio/516958: [ 2437.320157] #0: ffff888179939500 (&f->f_pos_lock){+.+.}-{3:3}, at: __fdget_pos+0xa7/0xc0 [ 2437.320188] #1: ffff88812214a498 (sb_writers#3){.+.+}-{0:0}, at: ksys_write+0xe9/0x1b0 [ 2437.320218] #2: ffffffffc1157a90 (cmd_mutex#6){+.+.}-{3:3}, at: _stp_ctl_write_cmd+0x230/0x3004 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320253] CPU: 9 PID: 516958 Comm: stapio Tainted: G W OE=20= =20=20=20 5.11.22-100.orinc.fc32.x86_64+debug #1 [ 2437.320259] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1.fc35 04/01/2014 [ 2437.320264] Call Trace: [ 2437.320270] dump_stack+0xae/0xe5 [ 2437.320282] ___might_sleep.cold+0x150/0x17e [ 2437.320292] vfree+0x4b/0x60 [ 2437.320299] _stp_mem_debug_free+0x46f/0x4b7 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320314] _stp_vfree+0xf/0x10 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320322] _stp_stat_free+0xc2/0x163 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320338] _stp_stat_del+0xf/0x10 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320354] _stp_cleanup_and_exit+0x56e/0xa97 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320372] ? stap_stop_task_finder+0x73c/0x73c [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320395] _stp_ctl_write_cmd+0x3be/0x3004 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320409] ? lock_acquire+0x1cc/0x780 [ 2437.320416] ? ksys_write+0xe9/0x1b0 [ 2437.320428] ? _stp_cleanup_and_exit+0xa97/0xa97 [stap_471641e12a7967ee269f819958d850_516958] [ 2437.320437] ? lockdep_hardirqs_on_prepare+0x3f0/0x3f0 [ 2437.320442] ? _cond_resched+0x15/0x30 [ 2437.320448] ? inode_security+0x56/0xf0 [ 2437.320457] ? selinux_file_permission+0x30a/0x3f0 [ 2437.320471] proc_reg_write+0x1ac/0x270 [ 2437.320485] vfs_write+0x180/0x810 [ 2437.320500] ksys_write+0xe9/0x1b0 [ 2437.320507] ? __ia32_sys_read+0xb0/0xb0 [ 2437.320516] ? rcu_read_lock_sched_held+0x3f/0x80 [ 2437.320530] do_syscall_64+0x33/0x40 [ 2437.320537] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 2437.320543] RIP: 0033:0x7f789872b90f [ 2437.320550] Code: 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 29 fd ff ff = 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 b8 01 00 00 00 0f 05 <48> 3= d 00 f0 ff ff 77 31 44 89 c7 48 89 44 24 08 e8 5c fd ff ff 48 [ 2437.320555] RSP: 002b:00007ffc1c3ce360 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 2437.320564] RAX: ffffffffffffffda RBX: ffffffffffffffff RCX: 00007f789872b90f [ 2437.320568] RDX: 0000000000000004 RSI: 00007ffc1c3ce398 RDI: 0000000000000004 [ 2437.320572] RBP: 0000000000000000 R08: 0000000000000002 R09: 00007ffc1c3cf100 [ 2437.320577] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc1c3cf170 [ 2437.320581] R13: 0000000000000001 R14: 000000000000000b R15: 00007ffc1c3cf1f4 ``` This is because the `_stp_cleanup_and_exit` function in the runtime explici= tly disables preemption for `_stp_printf`. But we should really temporarily re-enable premption for sleeping operations like `_stp_stat_del()`. --=20 You are receiving this mail because: You are the assignee for the bug.=