[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[jistone at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11249

Josh Stone <jistone at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jistone at redhat dot com

--- Comment #10 from Josh Stone <jistone at redhat dot com> 2010-11-04 01:11:06 UTC ---
I think I'm hitting this same problem, easily reproduced on i686 but not
x86_64.
http://sourceware.org/ml/systemtap/2010-q4/msg00168.html

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[jistone at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11249

--- Comment #11 from Josh Stone <jistone at redhat dot com> 2010-11-17 00:19:44 UTC ---
Belated update... later in that thread, I theorized that the GPF may be the
CPU's check that the call target is within the segment boundaries.  Roland
helped me dig into this, and after "sysctl -w kernel.print-fatal-signals=1", I
got:

> [  259.933657] #GPF(0[seg:0]) at bfb19020, CPU#0.
> [  259.934104] exec_limit: bfb1a000, user_cs: 0000fb19/00cbfb00.
> [  259.934707] stap/1684: potentially unexpected fatal signal 11.
> [  259.934709] code at bfb19020: e8 05 ea 00 00 8d 85 a8 fa ff ff 89 5c 24 08 89 
> [  259.934768] Pid: 1684, comm: stap Not tainted 2.6.35.6-45.fc14.i686 #1 /Bochs
> [  259.934771] EIP: 0073:[<bfb19020>] EFLAGS: 00010346 CPU: 0
> [  259.934779] EIP is at 0xbfb19020
> [  259.934781] EAX: 00000000 EBX: bfb169f4 ECX: bfb16950 EDX: bfb163e0
> [  259.934783] ESI: 00000002 EDI: 00000000 EBP: bfb16938 ESP: bfb16140
> [  259.934785]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
> [  259.934788] Process stap (pid: 1684, ti=dce12000 task=ddcf9940 task.ti=dce12000)
> [  259.935379] 
> [  259.935381] Call Trace:
> [  259.937563] Task died at uprobe probepoint:  pid/tgid = 1684/1684, probepoint = 0x80538f6

Notice in particular the exec_limit at bfb1a000.  Our instruction at bfb19020
is 5 bytes, "e8 05 ea 00 00 : call +0xea05", which puts the call target at
0xbfb27a2a.  So indeed, we appear to be leaving the segment.  That's before
uprobes corrects the IP, of course, but the CPU doesn't give us that chance.

This is on a non-PAE kernel.  Roland informed me that i686.PAE and x86_64
kernels don't put any bounds on the code segment, so I tried on a PAE kernel
and could not reproduce the issue. (And we already knew x86_64 was ok).

(In reply to comment #4)
> I was using 2.6.29.4-167.fc11.i686.PAE in my analysis.
> 
> Whats interesting is if we run a 2.6.33-rc4 git kernel from branch
> utrace-gdbstub-uprobes from location git://web.elastic.org/~fche/utrace-ext.git
> on the same machine, the problem disappears.

Srikar's PAE result casts this into doubt, unless in F11 times the PAE kernel
still kept segment limits.  Or he misreported. ;)  But I tried on F12-14 and
RHEL5, and consistently the non-PAE would fail while the PAE worked fine. 
RHEL6 is always PAE, and also works fine.

So - if we care about non-PAE kernels, we'll need to either refuse to probes
these relative instructions, or figure out a pre_ssout correction to keep the
target in bounds.  But at least on i686.PAE and x86_64, it seems ok.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=11249

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |fche at redhat dot com
         Resolution|---                         |FIXED

--- Comment #12 from Frank Ch. Eigler <fche at redhat dot com> ---
uprobes on 32-bit -calls- to .get_pc_thunk. functions appear to be working on
rhel7+.
.return probes don't, and this is fixed via commit 2303f9f86

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug runtime/11249] New: Tracking executable plus library fails on i386

[mjw at redhat dot com]

The following works fine on x86_64:

$ stap -e 'probe process("/bin/ls").function("*").call, probe
process("/lib64/libc.so.6").function("*").call { log(pp()) }' -c /bin/ls

producing logs of function calls in both the process and the library.
But on i386 (use /lib/libc.so.6) it only produces some of the calls in the
library, none in the executable:

$ stap -e 'probe process("/bin/ls").function("*").call,
process("/lib/libc.so.6").function("*").call { log(pp()) }' -c /bin/ls
process("/lib/libc.so.6").function("__sigprocmask@../sysdeps/unix/sysv/linux/sigprocmask.c:38").call
process("/lib/libc.so.6").function("execvp@/usr/src/debug/glibc-2.11.1/posix/execvp.c:26").call
process("/lib/libc.so.6").function("__execvpe@/usr/src/debug/glibc-2.11.1/posix/execvpe.c:50").call
process("/lib/libc.so.6").function("__execve@../sysdeps/unix/sysv/linux/execve.c:28").call
process("/lib/libc.so.6").function("__new_getrlimit@../sysdeps/unix/sysv/linux/i386/getrlimit.c:40").call
process("/lib/libc.so.6").function("__sysconf@../sysdeps/unix/sysv/linux/x86_64/sysconf.c:32").call
process("/lib/libc.so.6").function("__getpagesize@../sysdeps/unix/sysv/linux/getpagesize.c:27").call

And that is it. Only probing the process itself and not the library correctly
logs all executable function calls.

-- 
           Summary: Tracking executable plus library fails on i386
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: runtime
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: mjw at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=11249

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[fche at redhat dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|runtime                     |uprobes
            Summary|Tracking executable plus    |uprobes fails on glibc get-
                   |library fails on i386       |pc-thunk call insn probe


http://sourceware.org/bugzilla/show_bug.cgi?id=11249

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/11249] uprobes fails on glibc get-pc-thunk call insn probe

[jkenisto at us dot ibm dot com]

------- Additional Comments From jkenisto at us dot ibm dot com  2010-03-13 22:27 -------
After applying the patch from PR #11376, to get stap to emit correct code for
stap scripts generated by probe*.awk, I ran some a couple of simple tests, and
verified that uprobes still seems to support probing of relative jumps and
calls, at least in executable binaries.  I didn't test shared libs, where
addressing is more complicated.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=11249

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.