[Bug runtime/22124] New: RHEL6 ppc64 system crash when running the perf.exp test case

[Bug runtime/22124] New: RHEL6 ppc64 system crash when running the perf.exp test case

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22124

            Bug ID: 22124
           Summary: RHEL6 ppc64 system crash when running the perf.exp
                    test case
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: runtime
          Assignee: systemtap at sourceware dot org
          Reporter: dsmith at redhat dot com
  Target Milestone: ---

On 2.6.32-696.el6.ppc64, I'm seeing the following crash when running the
systemtap.base/perf.exp test case:

====
Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xd000000007dfa90c
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=1024 NUMA pSeries
Modules linked in: stap_79164cf446ea6b9c763989fcbbe91f8_13958(U) uprobes(U)
ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ib_mad
ib_core ib_addr ipv6 sg ibmveth ext4 jbd2 mbcache sr_mod cdrom sd_mod
crc_t10dif ibmvscsic scsi_transport_srp scsi_tgt dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: stap_7a693eb26517ff1145067a1cd652257_13485]
NIP: d000000007dfa90c LR: d000000007df75e8 CTR: d000000007dfa8c0
REGS: c00000043370f550 TRAP: 0300   Not tainted  (2.6.32-696.el6.ppc64)
MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 24000482  XER: 00000000
DAR: 0000000000000000, DSISR: 0000000040000000
TASK = c00000042eed0580[13959] 'towers.x' THREAD: c00000043370c000 CPU: 0
GPR00: d000000007df75e8 c00000043370f7d0 d000000007e0b438 d000000007dfe7e0 
GPR04: 0000000000000002 0000000000000001 c00000043370fce0 c000000433885688 
GPR08: 0000000d00000000 0000000000000000 d000000008060000 d000000007dfaed0 
GPR12: 0000000024000488 c000000001092500 0000000000000000 0000000000000000 
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
GPR20: c00000043370fad8 0000000000000000 00000fffc910000c 0000000000100100 
GPR24: 0000000000200200 0000000000100100 c00000043370fea0 c000000429267780 
GPR28: d000000007dfe690 d000000008060000 d000000007e0a8e0 d000000008060278 
NIP [d000000007dfa90c] .probe_3790+0x4c/0x610
[stap_79164cf446ea6b9c763989fcbbe91f8_13958]
LR [d000000007df75e8] .enter_uretprobe_probe+0x238/0x410
[stap_79164cf446ea6b9c763989fcbbe91f8_13958]
Call Trace:
[c00000043370f890] [d000000007df75e8] .enter_uretprobe_probe+0x238/0x410
[stap_79164cf446ea6b9c763989fcbbe91f8_13958]
[c00000043370f940] [d000000007944b24] .uprobe_report_signal+0xa04/0xfcc
[uprobes]
[c00000043370fa60] [c000000000102f58] .utrace_get_signal+0x378/0x820
[c00000043370fb60] [c0000000000b4230] .get_signal_to_deliver+0x3f0/0x590
[c00000043370fc50] [c000000000015f94] .do_signal_pending.clone.0+0x64/0x3e0
[c00000043370fda0] [c000000000016398] .do_signal+0x88/0xb0
[c00000043370fe30] [c000000000008b60] do_work+0x24/0x28
Instruction dump:
f8010010 fb21ffc8 fb41ffd0 fb61ffd8 f821ff41 ebc28000 7c7d1b78 38800002 
3be30278 eb9e8000 e9230270 387c0150 <e8090000> f81d0278 4bff740d 2fa30000
====

(Note that this happens with the pre-bpf code also.)

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22124] RHEL6 ppc64 system crash when running the perf.exp test case

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22124

--- Comment #1 from David Smith <dsmith at redhat dot com> ---
There are several subtests in perf.exp. Here are the manual steps to get the
crash:

====
# cd /root/rhel6-ppc64/testsuite
# gcc /root/src.copy/testsuite/systemtap.base/towers.c  -g  -lm   -o
/root/rhel6-ppc64/testsuite/towers.x
# stap -W /root/src.copy/testsuite/systemtap.base/perf04.stp ./towers.x -c
./towers.x
====

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22124] RHEL6 ppc64 system crash when running the perf.exp test case

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22124

--- Comment #2 from David Smith <dsmith at redhat dot com> ---
After a bit more debugging, I believe this would happen on any system with
"classic" uprobes (not inode-based uprobes) with support for perf hardware
instruction counters. Only the ppc64 seems to have both of these.

The uretprobe probe handler is accessing c->perf_read_values without
initializing the value first.

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22124] RHEL6 ppc64 system crash when running the perf.exp test case

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22124

David Smith <dsmith at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #3 from David Smith <dsmith at redhat dot com> ---
Fixed in commit 958a33dbd. This issue actually affected every platform, it is
just that other platforms were OK with the uninitialized variable (which could
have had an old stack address in it assuming the same context structure was
used for the function entry and exit probes).

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/22124] RHEL6 ppc64 system crash when running the perf.exp test case

[dsmith at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=22124

--- Comment #4 from David Smith <dsmith at redhat dot com> ---
*** Bug 21876 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are the assignee for the bug.