* Single-stepping with interrupts enabled.
@ 2006-02-27 11:29 Richard J Moore
2006-02-27 13:51 ` Frank Ch. Eigler
2006-02-27 17:13 ` James Dickens
0 siblings, 2 replies; 6+ messages in thread
From: Richard J Moore @ 2006-02-27 11:29 UTC (permalink / raw)
To: systemtap
A couple of weeks ago the topic of single-stepping came up on the
conference call. I said, I'd document the technical consideration over
allowing that to happen - and why it has not been implemented in the
general case.
The problem is down to the need to save state over the single-step, so that
pre- and post-processing either side of the single-step can be correctly
associated with each other. The difficulties that arise when interrupts are
enabled are two-fold, but only manifest themselves when a user-space
instruction is single-stepped.
1) To save state, and to cater for possible recursion one needs a stack.
Since single-stepping a user-space instruction requires a return to
user-space and a stack-switch, there is no natural stack that can be used
to save and restore state. In general a transition from high privilege to
low privilege can only be done using a return mechanism, not a call. On
re-entering the kernel, all state on the kernel stack is lost since we
always enter from a base stack setting as ultimately governed by the TSS in
IA32 or some other equivalent structure in other stack-based architectures.
The answer to this is to implement a hand-managed stack, where we need to
detect the out-of stack condition and revert back to single-stepping with
interrupts disabled until stack space is freed up.
2) the second problem is due to the possibility that the ordering of data
saved on the single-step stack can become invalidated. For example:
Consider the following events concerning two probed locations in user-space
(probe A and probe B).
a) Encounter probe A
b) Save probe processing state
b) Schedule single-stepping of probe A's instruction,
d) Timer interrupt occurs
e) Thread context switch to thread with probe B
f) Encounter probe B
g) Save probe processing state
h) Schedule single-stepping of probe B's instruction.
i) Timer interrupt
j) Thread context switch to thread with probe A
k) Single-step instruction at probe A
l) Enter probe post-processing.
m) Attempt to restore probe processing state.
Here we have a problem: the most recently saved state is that of probe B.
If on the other hand, had probe B single-stepping not been interrupted with
a context switch then we would have found the correct sate, namely that of
probe B.
So, the only options are to disable pre-emption over the single-step or to
implement a per-thread single-step stack.
I can be done, but the implementation is messy and it's a general problem
for all architectures.
- -
Richard J Moore
IBM Advanced Linux Response Team - Linux Technology Centre
MOBEX: 264807; Mobile (+44) (0)7739-875237
Office: (+44) (0)1962-817072
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Single-stepping with interrupts enabled.
2006-02-27 11:29 Single-stepping with interrupts enabled Richard J Moore
@ 2006-02-27 13:51 ` Frank Ch. Eigler
2006-02-27 14:05 ` Richard J Moore
2006-02-27 17:13 ` James Dickens
1 sibling, 1 reply; 6+ messages in thread
From: Frank Ch. Eigler @ 2006-02-27 13:51 UTC (permalink / raw)
To: Richard J Moore; +Cc: systemtap
richardj_moore wrote:
> [...] The problem is down to the need to save state over the
> single-step, so that pre- and post-processing either side of the
> single-step can be correctly associated with each other. [...]
How much of this would be simplified if we did away with support for
post_handlers (which we don't use in systemtap)?
- FChE
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Single-stepping with interrupts enabled.
2006-02-27 13:51 ` Frank Ch. Eigler
@ 2006-02-27 14:05 ` Richard J Moore
0 siblings, 0 replies; 6+ messages in thread
From: Richard J Moore @ 2006-02-27 14:05 UTC (permalink / raw)
To: Frank Ch. Eigler; +Cc: systemtap
You can't do without post-handlers. SystemTap might not use them explicitly
but post-handling of the single-step (in kprobes) is required whether or
not a user handler is registered. For example for correcting the outcome of
an instructions execution (certain adjustments may have to be made - E49,
etc..); for handling exceptions from the single-stepped instruction
correctly; for giving the correct 'logical' view at a higher level of a
trace event following the generation of a recoverable page-fault on
single-step; for correcting IF manipulating instructions where the state of
the interrupt flag is not actually as the probed program logically thinks
it is; for correcting the TRAP Flag for the same reasons as IF.
There must be more.
- -
Richard J Moore
IBM Advanced Linux Response Team - Linux Technology Centre
MOBEX: 264807; Mobile (+44) (0)7739-875237
Office: (+44) (0)1962-817072
fche@redhat.com
(Frank Ch.
Eigler) To
Sent by: Richard J Moore/UK/IBM@IBMGB
fche@redhat.com cc
systemtap@sources.redhat.com
bcc
27/02/2006
13:51 Subject
Re: Single-stepping with interrupts
enabled.
richardj_moore wrote:
> [...] The problem is down to the need to save state over the
> single-step, so that pre- and post-processing either side of the
> single-step can be correctly associated with each other. [...]
How much of this would be simplified if we did away with support for
post_handlers (which we don't use in systemtap)?
- FChE
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Single-stepping with interrupts enabled.
2006-02-27 11:29 Single-stepping with interrupts enabled Richard J Moore
2006-02-27 13:51 ` Frank Ch. Eigler
@ 2006-02-27 17:13 ` James Dickens
2006-02-27 17:34 ` Richard J Moore
1 sibling, 1 reply; 6+ messages in thread
From: James Dickens @ 2006-02-27 17:13 UTC (permalink / raw)
To: Richard J Moore; +Cc: systemtap
On 2/27/06, Richard J Moore <richardj_moore@uk.ibm.com> wrote:
>
>
>
>
> A couple of weeks ago the topic of single-stepping came up on the
> conference call. I said, I'd document the technical consideration over
> allowing that to happen - and why it has not been implemented in the
> general case.
>
>
> The problem is down to the need to save state over the single-step, so that
> pre- and post-processing either side of the single-step can be correctly
> associated with each other. The difficulties that arise when interrupts are
> enabled are two-fold, but only manifest themselves when a user-space
> instruction is single-stepped.
A lot of problems can be solved if you use some of the methodology
that dtrace uses. Where they do most single stepping in a scratch
space and defers signals till after were done with our probing.
http://cvs.opensolaris.org/source/xref/on/usr/src/uts/intel/dtrace/fasttrap_isa.c
44 * Most instructions' execution is not dependent on their
address; for these
45 * instrutions it is sufficient to copy them into the user
process's address
46 * space and execute them. To effectively single-step an instruction in
47 * user-land, we copy out the following sequence of
instructions to scratch
48 * space in the user thread's ulwp_t structure.
49 *
50 * We then set the program counter (%eip or %rip) to point to
this scratch
51 * space. Once execution resumes, the original instruction is
executed and
52 * then control flow is redirected to what was originally the subsequent
53 * instruction. If the kernel attemps to deliver a signal while single-
54 * stepping, the signal is deferred and the program counter is
moved into the
55 * second sequence of instructions. The second sequence ends
in a trap into
56 * the kernel where the deferred signal is then properly
handled and delivered.
57 *
58 * For instructions whose execute is position dependent, we
perform simple
59 * emulation. These instructions are limited to control transfer
60 * instructions in 32-bit mode, but in 64-bit mode there's the
added wrinkle
61 * of %rip-relative addressing that means that almost any
instruction can be
62 * position dependent. For all the details on how we emulate generic
63 * instructions included %rip-relative instructions, see the code in
64 * fasttrap_pid_probe() below where we handle instructions of type
65 * FASTTRAP_T_COMMON (under the header: Generic Instruction Tracing).
66 */
James Dickens
uadmin.blogspot.com
[snipped]
>
> - -
> Richard J Moore
> IBM Advanced Linux Response Team - Linux Technology Centre
> MOBEX: 264807; Mobile (+44) (0)7739-875237
> Office: (+44) (0)1962-817072
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Single-stepping with interrupts enabled.
2006-02-27 17:13 ` James Dickens
@ 2006-02-27 17:34 ` Richard J Moore
2006-02-27 18:18 ` James Dickens
0 siblings, 1 reply; 6+ messages in thread
From: Richard J Moore @ 2006-02-27 17:34 UTC (permalink / raw)
To: James Dickens; +Cc: systemtap
Effectively we do that (and I note that the Dtrace guys read my paper on
how to do this ;-)), however to do better than that one might consider
enabling interrupts or allowing signals during the single-step.
- -
Richard J Moore
IBM Advanced Linux Response Team - Linux Technology Centre
MOBEX: 264807; Mobile (+44) (0)7739-875237
Office: (+44) (0)1962-817072
"James Dickens"
<jamesd.wi@gmai
l.com> To
Richard J Moore/UK/IBM@IBMGB
27/02/2006 cc
17:13 "systemtap@sources.redhat.com"
<systemtap@sources.redhat.com>
bcc
Subject
Re: Single-stepping with interrupts
enabled.
On 2/27/06, Richard J Moore <richardj_moore@uk.ibm.com> wrote:
>
>
>
>
> A couple of weeks ago the topic of single-stepping came up on the
> conference call. I said, I'd document the technical consideration over
> allowing that to happen - and why it has not been implemented in the
> general case.
>
>
> The problem is down to the need to save state over the single-step, so
that
> pre- and post-processing either side of the single-step can be correctly
> associated with each other. The difficulties that arise when interrupts
are
> enabled are two-fold, but only manifest themselves when a user-space
> instruction is single-stepped.
A lot of problems can be solved if you use some of the methodology
that dtrace uses. Where they do most single stepping in a scratch
space and defers signals till after were done with our probing.
http://cvs.opensolaris.org/source/xref/on/usr/src/uts/intel/dtrace/fasttrap_isa.c
44 * Most instructions' execution is not dependent on their
address; for these
45 * instrutions it is sufficient to copy them into the user
process's address
46 * space and execute them. To effectively single-step an
instruction in
47 * user-land, we copy out the following sequence of
instructions to scratch
48 * space in the user thread's ulwp_t structure.
49 *
50 * We then set the program counter (%eip or %rip) to point to
this scratch
51 * space. Once execution resumes, the original instruction is
executed and
52 * then control flow is redirected to what was originally the
subsequent
53 * instruction. If the kernel attemps to deliver a signal while
single-
54 * stepping, the signal is deferred and the program counter is
moved into the
55 * second sequence of instructions. The second sequence ends
in a trap into
56 * the kernel where the deferred signal is then properly
handled and delivered.
57 *
58 * For instructions whose execute is position dependent, we
perform simple
59 * emulation. These instructions are limited to control transfer
60 * instructions in 32-bit mode, but in 64-bit mode there's the
added wrinkle
61 * of %rip-relative addressing that means that almost any
instruction can be
62 * position dependent. For all the details on how we emulate
generic
63 * instructions included %rip-relative instructions, see the code
in
64 * fasttrap_pid_probe() below where we handle instructions of type
65 * FASTTRAP_T_COMMON (under the header: Generic Instruction
Tracing).
66 */
James Dickens
uadmin.blogspot.com
[snipped]
>
> - -
> Richard J Moore
> IBM Advanced Linux Response Team - Linux Technology Centre
> MOBEX: 264807; Mobile (+44) (0)7739-875237
> Office: (+44) (0)1962-817072
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Single-stepping with interrupts enabled.
2006-02-27 17:34 ` Richard J Moore
@ 2006-02-27 18:18 ` James Dickens
0 siblings, 0 replies; 6+ messages in thread
From: James Dickens @ 2006-02-27 18:18 UTC (permalink / raw)
To: Richard J Moore; +Cc: systemtap
On 2/27/06, Richard J Moore <richardj_moore@uk.ibm.com> wrote:
>
>
>
>
> Effectively we do that (and I note that the Dtrace guys read my paper on
> how to do this ;-)), however to do better than that one might consider
> enabling interrupts or allowing signals during the single-step.
Seems like enabling interups and allowing signals during the single
step is just asking for problems, dtrace's implementation of your
ideas seems rock stable, I have tested it out by testing 32 dtrace
apps ( plockstat) running at the same time on a SMP system and had no
problems. And others have done even more testing. It would be best to
get basic functionality working, before you move to the next step,
enabling interupts and signals.
James
> - -
> Richard J Moore
> IBM Advanced Linux Response Team - Linux Technology Centre
> MOBEX: 264807; Mobile (+44) (0)7739-875237
> Office: (+44) (0)1962-817072
>
>
>
> "James Dickens"
> <jamesd.wi@gmai
> l.com> To
> Richard J Moore/UK/IBM@IBMGB
> 27/02/2006 cc
> 17:13 "systemtap@sources.redhat.com"
> <systemtap@sources.redhat.com>
> bcc
>
> Subject
> Re: Single-stepping with interrupts
> enabled.
>
>
>
>
>
>
> On 2/27/06, Richard J Moore <richardj_moore@uk.ibm.com> wrote:
> >
> >
> >
> >
> > A couple of weeks ago the topic of single-stepping came up on the
> > conference call. I said, I'd document the technical consideration over
> > allowing that to happen - and why it has not been implemented in the
> > general case.
> >
> >
> > The problem is down to the need to save state over the single-step, so
> that
> > pre- and post-processing either side of the single-step can be correctly
> > associated with each other. The difficulties that arise when interrupts
> are
> > enabled are two-fold, but only manifest themselves when a user-space
> > instruction is single-stepped.
>
> A lot of problems can be solved if you use some of the methodology
> that dtrace uses. Where they do most single stepping in a scratch
> space and defers signals till after were done with our probing.
>
>
> http://cvs.opensolaris.org/source/xref/on/usr/src/uts/intel/dtrace/fasttrap_isa.c
>
> 44 * Most instructions' execution is not dependent on their
> address; for these
> 45 * instrutions it is sufficient to copy them into the user
> process's address
> 46 * space and execute them. To effectively single-step an
> instruction in
> 47 * user-land, we copy out the following sequence of
> instructions to scratch
> 48 * space in the user thread's ulwp_t structure.
> 49 *
> 50 * We then set the program counter (%eip or %rip) to point to
> this scratch
> 51 * space. Once execution resumes, the original instruction is
> executed and
> 52 * then control flow is redirected to what was originally the
> subsequent
> 53 * instruction. If the kernel attemps to deliver a signal while
> single-
> 54 * stepping, the signal is deferred and the program counter is
> moved into the
> 55 * second sequence of instructions. The second sequence ends
> in a trap into
> 56 * the kernel where the deferred signal is then properly
> handled and delivered.
> 57 *
> 58 * For instructions whose execute is position dependent, we
> perform simple
> 59 * emulation. These instructions are limited to control transfer
> 60 * instructions in 32-bit mode, but in 64-bit mode there's the
> added wrinkle
> 61 * of %rip-relative addressing that means that almost any
> instruction can be
> 62 * position dependent. For all the details on how we emulate
> generic
> 63 * instructions included %rip-relative instructions, see the code
> in
> 64 * fasttrap_pid_probe() below where we handle instructions of type
> 65 * FASTTRAP_T_COMMON (under the header: Generic Instruction
> Tracing).
> 66 */
>
> James Dickens
> uadmin.blogspot.com
>
>
> [snipped]
> >
> > - -
> > Richard J Moore
> > IBM Advanced Linux Response Team - Linux Technology Centre
> > MOBEX: 264807; Mobile (+44) (0)7739-875237
> > Office: (+44) (0)1962-817072
> >
> >
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2006-02-27 18:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-02-27 11:29 Single-stepping with interrupts enabled Richard J Moore
2006-02-27 13:51 ` Frank Ch. Eigler
2006-02-27 14:05 ` Richard J Moore
2006-02-27 17:13 ` James Dickens
2006-02-27 17:34 ` Richard J Moore
2006-02-27 18:18 ` James Dickens
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).