From: Chris Dunlop <chris@onthe.net.au>
To: systemtap@sourceware.org
Subject: Error removing module: Device or resource busy
Date: Fri, 23 Dec 2011 08:21:00 -0000 [thread overview]
Message-ID: <20111223050020.GA11829@onthe.net.au> (raw)
Hi,
Linux-3.0.13, systemtap 6.1 and now HEAD(b6a3da4)
I've just started using systemtap to help diagnose a problem with zfsonlinux[1].
Whenever I run stap[2], the output ends with:
Error removing module 'stap_fcac0085842418e34d8094455dc203e8_1_21605': Device or resource busy.
(obviously the module name changes) and the module is still loaded:
# lsmod | grep stap
stap_fcac0085842418e34d8094455dc203e8_1_21605 2896285 1027571582 [permanent]
Using "rmmod -f" on that module results in a kernel NULL pointer dereference[3].
Any suggestions about how to avoid the unloadable module?
Cheers,
Chris
[1] https://github.com/zfsonlinux/zfs/issues/503
----------------------------------------------------------------------
[2] stap usage
----------------------------------------------------------------------
# stap \
-c "getfattr -d file" \
para-callgraph.stp \
'module("zfs").function("*@module/zfs/*.c")' \
'module("zfs").function("zpl_xattr_list")'
# cat para-callgraph.stp
#! /usr/bin/env stap
#
# http://sourceware.org/systemtap/examples/general/para-callgraph.stp
#
function trace(entry_p, extra) {
%( $# > 1 %? if (tid() in trace) %)
printf("%s%s%s %s\n",
thread_indent (entry_p),
(entry_p>0?"->":"<-"),
probefunc (),
extra)
}
%( $# > 1 %?
global trace
probe $2.call {
trace[tid()] = 1
}
probe $2.return {
delete trace[tid()]
}
%)
probe $1.call { trace(1, $$parms) }
probe $1.return { trace(-1, $$return) }
----------------------------------------------------------------------
[3] kernel message
----------------------------------------------------------------------
kernel: [ 292.856177] BUG: unable to handle kernel NULL pointer dereference at 0000000000000020
kernel: [ 292.856392] IP: [<ffffffff810cf584>] free_percpu+0xa4/0x112
kernel: [ 292.856531] PGD 611554067 PUD 5e1622067 PMD 0
kernel: [ 292.856754] Oops: 0000 [#1] SMP
kernel: [ 292.856927] CPU 5
kernel: [ 292.856980] Modules linked in: ext4 mbcache jbd2 crc16 loop fuse nfsd exportfs zfs(P) zcommon(P) znvpair(P) zavl(P) zunicode(P) spl zlib_deflate sha256_generic aesni_intel cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod nfs lockd auth_rpcgss nfs_acl sunrpc bridge stp llc sg sd_mod usbhid hid uhci_hcd mpt2sas psmouse scsi_transport_sas raid_class scsi_mod i2c_i801 i2c_core ioatdma ehci_hcd igb dca button processor thermal_sys [last unloaded: stap_152d5b1673c7d1a7749e69173a687f9a_12_6436]
kernel: [ 292.859810]
kernel: [ 292.859886] Pid: 6444, comm: rmmod Tainted: P R 3.0.13-otn-00016-g1306024 #1 Supermicro X8DTH-i/6/iF/6F/X8DTH
kernel: [ 292.860152] RIP: 0010:[<ffffffff810cf584>] [<ffffffff810cf584>] free_percpu+0xa4/0x112
kernel: [ 292.860319] RSP: 0018:ffff8805d4371e58 EFLAGS: 00010046
kernel: [ 292.860406] RAX: 0000000000000000 RBX: 0000000000000287 RCX: ffff880be0261dde
kernel: [ 292.860498] RDX: 00003ffffffff000 RSI: ffff880000000000 RDI: 0000000000000000
kernel: [ 292.860591] RBP: ffff8805d4371e78 R08: ffffffff8117c1ab R09: ffffffff8100da91
kernel: [ 292.860684] R10: ffffffff8107487f R11: ffff88062763dfa0 R12: ffff880c3ffdde40
kernel: [ 292.860777] R13: ffff8805e0221dde R14: ffffffffa0728a28 R15: 0000000000000008
kernel: [ 292.860870] FS: 00007f9150eb8700(0000) GS:ffff880c3fc40000(0000) knlGS:0000000000000000
kernel: [ 292.860989] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
kernel: [ 292.861078] CR2: 0000000000000020 CR3: 000000060ffa1000 CR4: 00000000000006e0
kernel: [ 292.861171] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kernel: [ 292.861263] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
kernel: [ 292.861356] Process rmmod (pid: 6444, threadinfo ffff8805d4370000, task ffff88062155c350)
kernel: [ 292.861476] Stack:
kernel: [ 292.861551] ffffffff8117c1ab ffffffffa0728a18 ffffffffa0728830 ffffffffa0728a18
kernel: [ 292.861853] ffff8805d4371ea8 ffffffff810758f1 ffffffffa0728830 ffffffffa0728880
kernel: [ 292.862154] 0000000000000002 ffff8805d533e670 ffff8805d4371ed8 ffffffff81075a6f
kernel: [ 292.862456] Call Trace:
kernel: [ 292.862539] [<ffffffff8117c1ab>] ? kobject_del+0x38/0x38
kernel: [ 292.862631] [<ffffffff810758f1>] module_unload_free+0xca/0xd3
kernel: [ 292.862721] [<ffffffff81075a6f>] free_module+0x175/0x1c6
kernel: [ 292.862810] [<ffffffff8107725b>] sys_delete_module+0x1f3/0x226
kernel: [ 292.862903] [<ffffffff812c2312>] system_call_fastpath+0x16/0x1b
kernel: [ 292.862992] Code: c2 85 d2 4c 0f 45 25 04 52 54 00 eb 22 48 8b 05 9b f7 3e 00 4c 89 ef 65 8b 14 25 f8 db 00 00 48 63 d2 48 03 3c d0 e8 eb e4 00 00 <4c> 8b 60 20 44 89 ee 41 2b 74 24 18 4c 89 e7 e8 dc fd ff ff 41
kernel: [ 292.866166] RIP [<ffffffff810cf584>] free_percpu+0xa4/0x112
kernel: [ 292.866302] RSP <ffff8805d4371e58>
kernel: [ 292.866383] CR2: 0000000000000020
kernel: [ 292.866464] ---[ end trace 2b4d7238c919f6da ]---
----------------------------------------------------------------------
next reply other threads:[~2011-12-23 5:00 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-23 8:21 Chris Dunlop [this message]
2011-12-23 21:10 ` Josh Stone
2011-12-26 18:33 ` Chris Dunlop
2011-12-27 22:06 ` Josh Stone
2011-12-28 17:08 ` Chris Dunlop
2012-01-21 1:18 ` Josh Stone
2012-01-21 1:36 ` Chris Dunlop
2012-01-21 2:30 ` Chris Dunlop
2012-01-21 7:27 ` Josh Stone
2012-01-23 7:25 ` Chris Dunlop
2012-01-25 12:31 ` Frank Ch. Eigler
2012-01-26 0:30 ` Chris Dunlop
2012-01-26 1:34 ` Josh Stone
2012-01-26 2:28 ` Chris Dunlop
2012-01-26 18:47 ` Josh Stone
2012-01-26 23:32 ` Chris Dunlop
2012-01-21 1:40 ` Chris Dunlop
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20111223050020.GA11829@onthe.net.au \
--to=chris@onthe.net.au \
--cc=systemtap@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).