From: "agentzh at gmail dot com" <sourceware-bugzilla@sourceware.org>
To: systemtap@sourceware.org
Subject: [Bug runtime/25853] New: Stap's own dwarf-based stack unwinder does not give very accurate kernel backtraces
Date: Sat, 18 Apr 2020 19:51:25 +0000 [thread overview]
Message-ID: <bug-25853-6586@http.sourceware.org/bugzilla/> (raw)
https://sourceware.org/bugzilla/show_bug.cgi?id=25853
Bug ID: 25853
Summary: Stap's own dwarf-based stack unwinder does not give
very accurate kernel backtraces
Product: systemtap
Version: unspecified
Status: UNCONFIRMED
Severity: normal
Priority: P2
Component: runtime
Assignee: systemtap at sourceware dot org
Reporter: agentzh at gmail dot com
Target Milestone: ---
Intuitively, stap's own dwarf-based kernel stack unwinder should behave better
than kernel's `dump_stack()` function, but it seems that the opposite is true:
Using the current git master of stap on Fedora 30 x86_64:
```
$ /opt/stap/bin/stap -g -v -e 'probe oneshot { print_backtrace() dump_stack()
}'
Pass 1: parsed user script and 486 library scripts using
378940virt/154300res/8128shr/146396data kb, in 1190usr/40sys/1238real ms.
Pass 2: analyzed script: 1 probe, 3 functions, 0 embeds, 0 globals using
383824virt/159440res/8264shr/151280data kb, in 70usr/0sys/66real ms.
Pass 3: translated to C into
"/tmp/stapiqhPvJ/stap_15a72d9462813416bf28386f595c0677_1270_src.c" using
383824virt/159440res/8264shr/151280data kb, in 0usr/0sys/0real ms.
Pass 4: compiled C into "stap_15a72d9462813416bf28386f595c0677_1270.ko" in
10840usr/2210sys/2457real ms.
Pass 5: starting run.
WARNING: Missing unwind data for a module, rerun with 'stap -d kernel'
0xffffffffc0ace13b [stap_15a72d9462813416bf28386f595c06_108017+0x813b/0x0]
0xffffffffc0acf68f [stap_15a72d9462813416bf28386f595c06_108017+0x968f/0x0]
0xffffffffc0acf92e [stap_15a72d9462813416bf28386f595c06_108017+0x992e/0x0]
0xffffffffc0ad0f13 [stap_15a72d9462813416bf28386f595c06_108017+0xaf13/0x0]
0xffffffffa042f803
0xffffffffa042f803 (inexact)
0xffffffffa030d2c6 (inexact)
0xffffffffa030d56f (inexact)
0xffffffffa000438b (inexact)
0xffffffffa0c0008c (inexact)
Pass 5: run completed in 10usr/30sys/421real ms.
$ dmesg|tail -n13
[4739989.625769] stap_15a72d9462813416bf28386f595c06_108017 (<input>):
systemtap: 4.3/0.177, base: 0000000090fcb81d, memory:
160data/48text/268ctx/2063net/914alloc kb, probes: 1
[4739989.625812] CPU: 55 PID: 108017 Comm: stapio Tainted: G D OE
5.4.19-100.fc30.x86_64 #1
[4739989.625813] Hardware name: Gigabyte Technology Co., Ltd. TRX40 AORUS
MASTER/TRX40 AORUS MASTER, BIOS F4 01/20/2020
[4739989.625813] Call Trace:
[4739989.625817] dump_stack+0x66/0x90
[4739989.625821] probe_19154.cold+0x5/0x7e
[stap_15a72d9462813416bf28386f595c06_108017]
[4739989.625824] ? enter_be_probe.isra.0.constprop.0+0x11e/0x230
[stap_15a72d9462813416bf28386f595c06_108017]
[4739989.625826] ? _stp_ctl_write_cmd.cold+0x10b/0x11f8
[stap_15a72d9462813416bf28386f595c06_108017]
[4739989.625828] ? full_proxy_write+0x53/0x80
[4739989.625830] ? vfs_write+0xb6/0x1a0
[4739989.625831] ? ksys_write+0x5f/0xe0
[4739989.625832] ? do_syscall_64+0x5b/0x1a0
[4739989.625835] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9
```
More details:
```
$ /opt/stap/bin/stap -V
Systemtap translator/driver (version 4.3/0.177, commit
release-4.2-65-g6de9136a2794)
Copyright (C) 2005-2019 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.32 ... 5.4-rc6
enabled features: BPF PYTHON3 LIBSQLITE3 NLS READLINE
$ uname -r
5.4.19-100.fc30.x86_64
```
Also tried on an older system, Fedora 28 x86_64. It's the same thing:
```
$ /opt/stap/bin/stap -g -v -e 'probe oneshot { print_backtrace() dump_stack()
}'
Pass 1: parsed user script and 480 library scripts using
332760virt/89532res/7364shr/82644data kb, in 390usr/30sys/420real ms.
Pass 2: analyzed script: 1 probe, 3 functions, 0 embeds, 0 globals using
334344virt/91864res/7884shr/84228data kb, in 10usr/0sys/12real ms.
Pass 3: translated to C into
"/tmp/stap1aI32F/stap_4add63f3ab8162b5a22e27aee890f56b_1270_src.c" using
334344virt/91864res/7884shr/84228data kb, in 0usr/0sys/0real ms.
Pass 4: compiled C into "stap_4add63f3ab8162b5a22e27aee890f56b_1270.ko" in
11110usr/1740sys/3196real ms.
Pass 5: starting run.
WARNING: Missing unwind data for a module, rerun with 'stap -d kernel'
0xffffffffc0ded3ab [stap_4add63f3ab8162b5a22e27aee890f56_22901+0x83ab/0x0]
0xffffffffc0dee68a [stap_4add63f3ab8162b5a22e27aee890f56_22901+0x968a/0x0]
0xffffffffc0dee95e [stap_4add63f3ab8162b5a22e27aee890f56_22901+0x995e/0x0]
0xffffffffc0deff16 [stap_4add63f3ab8162b5a22e27aee890f56_22901+0xaf16/0x0]
0xffffffff813edbf3
0xffffffff813edbf3 (inexact)
0xffffffff812e3b86 (inexact)
0xffffffff812e3e95 (inexact)
0xffffffff812e4137 (inexact)
0xffffffff810041ab (inexact)
0xffffffff81a0008c (inexact)
Pass 5: run completed in 10usr/20sys/371real ms.
$ dmesg|tail -n15
[66680.072659] CPU: 7 PID: 22901 Comm: stapio Tainted: G OE
5.0.16-100.fc28.x86_64 #1
[66680.072659] Hardware name: Gigabyte Technology Co., Ltd. Z390 AORUS
MASTER/Z390 AORUS MASTER-CF, BIOS F6 11/08/2018
[66680.072660] Call Trace:
[66680.072662] dump_stack+0x5c/0x80
[66680.072666] probe_6406.cold.78+0x5/0x7e
[stap_4add63f3ab8162b5a22e27aee890f56_22901]
[66680.072667] ? enter_be_probe.isra.35.constprop.67+0x11e/0x230
[stap_4add63f3ab8162b5a22e27aee890f56_22901]
[66680.072669] ? _stp_ctl_write_cmd.cold.79+0x5a/0x1144
[stap_4add63f3ab8162b5a22e27aee890f56_22901]
[66680.072671] ? full_proxy_write+0x53/0x80
[66680.072672] ? __vfs_write+0x36/0x1b0
[66680.072673] ? selinux_file_permission+0xf0/0x130
[66680.072674] ? security_file_permission+0x2e/0xf0
[66680.072675] ? vfs_write+0xa5/0x1a0
[66680.072676] ? ksys_write+0x57/0xd0
[66680.072677] ? do_syscall_64+0x5b/0x160
[66680.072678] ? entry_SYSCALL_64_after_hwframe+0x44/0xa9
```
More details:
```
$ /opt/stap/bin/stap -V
Systemtap translator/driver (version 4.3/0.177, commit
release-4.2-65-g6de9136a2794)
Copyright (C) 2005-2019 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.32 ... 5.4-rc6
enabled features: BPF LIBSQLITE3 NLS
$ uname -r
5.0.16-100.fc28.x86_64
```
--
You are receiving this mail because:
You are the assignee for the bug.
next reply other threads:[~2020-04-18 19:51 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-18 19:51 agentzh at gmail dot com [this message]
2020-09-28 21:22 ` [Bug runtime/25853] " agentzh at gmail dot com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bug-25853-6586@http.sourceware.org/bugzilla/ \
--to=sourceware-bugzilla@sourceware.org \
--cc=systemtap@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).