Re: external systemtap meeting notes 20070816

Re: external systemtap meeting notes 20070816

[Frank Ch. Eigler]

jkenisto wrote [untrimmed for forwarding to utrace-devel]:

> [...]
> Uprobes maintains one utrace engine per probed task.  When
> instrumentation modules A and B register uprobes at the same address,
> and that probepoint gets hit, uprobes's signal (SIGTRAP) callback gets
> called (once).  It calls both A's and B's handlers (a la aggregate
> kprobes), then eats the SIGTRAP.
>  
> If A and B each got its own engine, then what would uprobes's callback
> do?  If the callback eats the SIGTRAP, then the 2nd handler never gets
> called.  If it doesn't eat the SIGTRAP, the probed process takes the
> SIGTRAP and dies.  We could work up some new way that A and B could
> coordinate this sort of thing, but then we're back to needing some sort
> of shared (among instrumentation modules) data structures.  And what we
> have now works just fine.

This is probably related to Roland's "engine interaction" note
(http://tinyurl.com/22jt2o).  It would be nice if concurrent engines
could nest transparently - or failing that, at least be aware of each
other enough to get this right.

- FChE

Re: external systemtap meeting notes 20070816

[Roland McGrath]

> > If A and B each got its own engine, then what would uprobes's callback
> > do?  If the callback eats the SIGTRAP, then the 2nd handler never gets
> > called.  If it doesn't eat the SIGTRAP, the probed process takes the
> > SIGTRAP and dies.  We could work up some new way that A and B could
> > coordinate this sort of thing, but then we're back to needing some sort
> > of shared (among instrumentation modules) data structures.  And what we
> > have now works just fine.
> 
> This is probably related to Roland's "engine interaction" note
> (http://tinyurl.com/22jt2o).  It would be nice if concurrent engines
> could nest transparently - or failing that, at least be aware of each
> other enough to get this right.

cf https://www.redhat.com/archives/utrace-devel/2007-September/msg00007.html

The future non-signal mechanism I described there can have a reporting
interface that lets an engine say "I induced this" while letting other
engines still see it, in case two have that same answer for a single
occurrence.  That is, it wouldn't cause any signal if any engines at all
claimed it, yet multiple engines could always see it.  This addresses the
part of the breakpoint interaction problem that Jim described here.

The other part of the problem is insertion/removal.  Naive non-cooperation
works if they literally nest, but not if removal order is not LIFO.  
I don't have any implicit-communication solution for that off hand.


Thanks,
Roland

Re: external systemtap meeting notes 20070816

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 925 bytes --]

Hi -

On Wed, Sep 26, 2007 at 01:42:29AM -0700, Roland McGrath wrote:

> [...]  The future non-signal mechanism I described there can have a
> reporting interface [...]  The other part of the problem is
> insertion/removal.  Naive non-cooperation works if they literally
> nest, but not if removal order is not LIFO.  I don't have any
> implicit-communication solution for that off hand.

Yeah, this is roughly why we pointed out some time back that the
utrace layer would be well situated to provide a high-level
breakpoint-related API.

What do you suggest in the interim?

Would this hack work: have the second utrace engine refuse to put a
breakpoint wherever it suspects another engine may have put one?  Or
even more pessimistically, can an engine know that another one is
already monitoring a given target process, and give up at attach time?
(That would defeat some of the promise of utrace, but so it goes.)

- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: external systemtap meeting notes 20070816

[Roland McGrath]

> Would this hack work: have the second utrace engine refuse to put a
> breakpoint wherever it suspects another engine may have put one?  

Sure, it's easy to notice the breakpoint instruction already there.

> Or even more pessimistically, can an engine know that another one is
> already monitoring a given target process, and give up at attach time?
> (That would defeat some of the promise of utrace, but so it goes.)

Yes, it can (UTRACE_ATTACH_EXCLUSIVE).


Thanks,
Roland

Re: external systemtap meeting notes 20070816

[Mark Wielaard]

On Wed, 2007-09-26 at 13:33 -0700, Roland McGrath wrote:
> > Would this hack work: have the second utrace engine refuse to put a
> > breakpoint wherever it suspects another engine may have put one?  
> 
> Sure, it's easy to notice the breakpoint instruction already there.

Would the second engine then also get an event notification when the
existing breakpoint gets hit? If so then it would be nice to have a
mechanism to detect a breakpoint being removed so an engine could say
"wait! I am still interested in monitoring when that address gets hit,
please keep that breakpoint there", and then just have the first engine
not be notified of any breakpoint events anymore, and only notify the
second engine from then on. (This is kind of what we do in Frysk, but
there of course we do all breakpoint handling in user space through one
ptrace wait monitoring thread that dispatches the events, so we are in
control of all breakpoint setting and breakpoint hit event detection.)

Cheers,

Mark