[Bug uprobes/10836] New: uprobes-provided pt_regs* are unreliable

[Bug uprobes/10836] New: uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

Several registers appearing in a utrace-oriented pt_regs do not accurately
represent the state of the user-space task.  As per bug #10601, this breaks
some dwarf-based $context variables.

It seems to me that either uprobes should not pass pt_regs at all to its
callbacks, or else populate it with synthetic (but accurate) values pulled
out of utrace regset calls.  (The latter would be slower and require per-arch
code, but would make it more convenient for the clients.  The former would
be faster, and still require per-arch code at the client, but would be less
convenient.)

-- 
           Summary: uprobes-provided pt_regs* are unreliable
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: uprobes
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: fche at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[srikar at linux dot vnet dot ibm dot com]

------- Additional Comments From srikar at linux dot vnet dot ibm dot com  2009-10-27 11:44 -------
I am not sure if I understand the problem completely.

> Several registers appearing in a utrace-oriented pt_regs do not accurately
> represent the state of the user-space task. 

Is this concern about instruction pointer pointing past the breakpoint?
or do we have additional concerns? If yes do we have specific registers in mind?
                                        
uprobes passes the pt_regs it gets from utrace's report_signal callback as is to
the handler.

> As per bug #10601, this breaks
> some dwarf-based $context variables.

This bug refers to two other bugs which point to problems in user space markers.
 So is this problem only seen on user space markers? or can we see this problem
on plain uprobes probe points too.

> It seems to me that either uprobes should not pass pt_regs at all to its
> callbacks, or else populate it with synthetic (but accurate) values pulled
> out of utrace regset calls.  (The latter would be slower and require per-arch
> code, but would make it more convenient for the clients.  The former would
> be faster, and still require per-arch code at the client, but would be less
> convenient.)

Is there any reason why this synthesis should be done at the uprobes end and not
at the client end?  Do you see all uprobe clients facing this problem?
If its a problem faced by all uprobe clients, then is it worth checking if
utrace should send the synthesized pt_regs as a parameter to report_signal.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |roland at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[srikar at linux dot vnet dot ibm dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |srikar at linux dot vnet dot
                   |                            |ibm dot com, jkenisto at us
                   |                            |dot ibm dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-10-27 12:27 -------
(In reply to comment #1)
> > Several registers appearing in a utrace-oriented pt_regs do not accurately
> > represent the state of the user-space task. 
> 
> Is this concern about instruction pointer pointing past the breakpoint?

No, not only.

> or do we have additional concerns? If yes do we have specific registers in mind?

It depends.  Sometimes esp, sometimes ebp, and I think I've seen other
registers with inconsistent values too.  Compare a systemtap print_regs
with a gdb breakpoint "info regs" at the same spot.

                                     
> uprobes passes the pt_regs it gets from utrace's report_signal callback as is to
> the handler.

Yes.  Unfortunately, these registers do not completely & correctly match the
state of the user-space thread.


> This bug refers to two other bugs which point to problems in user space markers.
>  So is this problem only seen on user space markers? or can we see this problem
> on plain uprobes probe points too.

In this context, user-space markers are a special case of uprobes.  Statement
uprobes (bypassing prologue heuristics) at the function entry point also
display this problem.


> Is there any reason why this synthesis should be done at the uprobes end and not
> at the client end?

I believe I summarized some pros & cons already.  Hiding the regset
stuff from uprobes clients would be the main benefit.  Perhaps the
run-time costs of doing this could be controlled by a struct-uprobes
flag that tells uprobes whether the client is interested in raw
pt_regs, nothing, or regset-filled pt_regs.

> Do you see all uprobe clients facing this problem?

Yes.

> If its a problem faced by all uprobe clients, then is it worth checking if
> utrace should send the synthesized pt_regs as a parameter to report_signal.

Roland?


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[roland at gnu dot org]

------- Additional Comments From roland at gnu dot org  2009-10-27 18:40 -------
"synthesized pt_regs" is a loony concept considered only by systemtap.
It has no place in the general utrace world.  user_regset is there, it's what
should be used.  The only reason pt_regs is passed into some callbacks is
because the pointer is handy and in some circumstances a few of its fields might
be sufficiently useful for particular code that knows exactly what it is looking
at.  General-case code can use the asm/syscall.h macros on it, for example.  For
any generalized register access, user_regset is the only right thing to use.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-10-27 19:16 -------
irc transcript:

mjw So, for syscall things, like getting at the arguments, we can use asm/syscall.h
mjw But that is only enough for syscall related callbacks isn't it?
mjw For anything else, like uprobes, we need to go through linux/regset.h
mjw and some arch specific mapping of the regsets to arch specific register 
values

roland i think instruction_pointer(regs) and user_stack_pointer(regs) can always
be used
roland for other particular things where you know arch-specifically that the reg
is ok you can use it
roland but yes, the general case is user_regset


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[mjw at redhat dot com]

------- Additional Comments From mjw at redhat dot com  2009-10-28 12:29 -------
This is also a problem for the dwarf based unwinder.

In runtime/unwind.c we feed unwind_frame() a pt_regs struct describing the
current state of the registers (plus some unwind tables, stp_module, etc.) Then
that uses its internal unwind_state regs stack to keep track of what is going to
happen with the pt_regs given the cfa ops found in the table. And when it is all
processed it spits out a new pt_regs struct describing the state of one unwind
step. In runtime/stack-<arch>.c we are feeding unwind_frame() initially the
pt_regs given by either kprobes or uprobes (arch_unw_init_frame_info(&info,
regs)). But the uprobes pt_regs are the "wrong kind" for user space although the
do work most of the time.

This could be fixed without having to change much by having the "loony concept"
of synthesized pt_regs without having to change too much of this code. Otherwise
we would need to make arch_unw_init_frame_info() smarter and detect we want the
user space registers (which we know in stp_stack_print because tsk != NULL) and
use the appropriate regsets to instantiate the unwind_info->regs.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[jkenisto at us dot ibm dot com]

------- Additional Comments From jkenisto at us dot ibm dot com  2009-10-28 16:53 -------
(In reply to comment #3)
> "synthesized pt_regs" is a loony concept considered only by systemtap.
> It has no place in the general utrace world.  user_regset is there, it's what
> should be used.  The only reason pt_regs is passed into some callbacks is
> because the pointer is handy and in some circumstances a few of its fields might
> be sufficiently useful for particular code that knows exactly what it is looking
> at.  General-case code can use the asm/syscall.h macros on it, for example.  For
> any generalized register access, user_regset is the only right thing to use.

Previous advice (as I understood it) from Roland during the uprobes 2 port
(~August '08) was to continue using the pt_regs pointer passed to
uprobe_report_signal().  So before Srikar adds a lot of user_regset code to
uprobes, it'd be nice clarify what Roland means by "any generalized register
access."  For example, at least some architectures' user_regset code boils down
to references to the pt_regs pointer provided by task_pt_regs().  For the
registers that SystemTap actually references, would the pointer provided by
task_pt_regs() be sufficient?

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-10-28 16:56 -------
(In reply to comment #6)

> [...] For the registers that SystemTap actually references, would the pointer 
> provided by task_pt_regs() be sufficient?

For sake of print_regs(), unwinding, or general dwarf-based (eg. fbreg-relative)
$context variable access, it looks like utrace signal pt_regs == task_pt_regs ==
not accurate.  So we do have a pervasive problem.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-10-28 16:58 -------
(In reply to comment #6)
> [...] So before Srikar adds a lot of user_regset code to
> uprobes, [...]

By the way, systemtap could do its own user_regset thing
independent of uprobes -- we'd just ignore the incoming pt_regs
almost entirely and add a bunch of code to uprobe/utrace probe
prologues and/or multiple places the runtime.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[roland at gnu dot org]

------- Additional Comments From roland at gnu dot org  2009-10-28 18:22 -------
uprobes machinery is not "generalized register access".  It is arch-specific
magic that knows plenty of deep magic about machine and kernel arch
implementation details.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[ananth at in dot ibm dot com]

------- Additional Comments From ananth at in dot ibm dot com  2009-10-29 06:20 -------
What are the main registers one needs to know here? Helpers like
regset_instruction_pointer(task), regset_stack_pointer(task) would be a useful
addition with or without uprobes, since they can hide the underlying
architecture details and offsets.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-10-29 14:29 -------
(In reply to comment #10)
> What are the main registers one needs to know here?

In the context of uprobes, apprx. all of them, since they may
contain $variable addresses / values.

> Helpers like regset_instruction_pointer(task), regset_stack_pointer(task)
> would be a useful addition with or without uprobes [...]

While I've been thinking about this mainly for uprobes, we get related
complications from utrace & timer.profile type probes, if someone were
to invoke print_ubacktrace().


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
OtherBugsDependingO|                            |10907
              nThis|                            |


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug uprobes/10836] uprobes-provided pt_regs* are unreliable

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-11-30 20:35 -------
systemtap will learn to ignore uprobes-provided pt_regs for most purposes as a
part of bug #10601.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX


http://sourceware.org/bugzilla/show_bug.cgi?id=10836

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.