Breakpoints and queued signals

[Ananth N Mavinakayanahalli <ananth@in.ibm.com>]

Roland,

Here is a brief on what I saw happening on powerpc. The basic issue is
that a SIGTRAP caused due to a uprobes breakpoint, is being delivered
after the uprobe is unregistered, causing the application to terminate
with a "Trace/Breakpoint Trap".

The testcase being used is a simple program that calls a uprobed
function in a loop; simultaneously, the uprobe module is insmod/rmmod'ed
in a loop with a 0.5 second delay.

In the uprobes code, when a call to unregister happens, we wait for all
threads to rendezvous in the quiesce callback. One can get into the
quiesce callback for different reasons:
- A breakpoint (uprobe) hit, before the signal callback
- 'Cos the UTRACE_ACTION_QUIESCE was asserted for the thread to
  facilitate the breakpoint removal

In order to handle this, when a unregister is pending and a thread
enters quiesce, using regset calls, the PC is examined to determine if
it corresponds to a breakpoint. If not, we deem the quiesce entry to be
due to the QUIESCE flag assertion, or possibly a different SIGTRAP.

In the current issue, by some combination of events, we are getting into
the quiesce callback due to a SIGTRAP with the PC pointing to a libc
location. The SIGTRAP due to a uprobe breakpoint is the subsequent
one in the signal queue.

utrace_get_signal: pid = 3178, regs->nip = ff1a6d4, thread.regs->nip = ff1a6d4, signo = 0 <-- nip corresponds to the libc text area
utrace_get_signal: pid = 3178, regs->nip = 1000058c, thread.regs->nip =
1000058c, signo = 0 <-- this is the signal due to the uprobe

Hence, during unregister, since the PC obtained from regset doesn't
point to the uprobe breakpoint location, we just go ahead and unregister
the uprobe. The subsequent signal delivered though was caused by a
uprobe hit, but by this time, the handlers are long gone; the signal is
delivered to the application, leading to the process to terminate.

Is there a way to handle such cases, probably by "walking" the pending
signals list for the thread, during unregister? Or, do you think this
is a uprobes bug? Or, if this is just a by-product of the way signals
work, is it a realistic issue for other utrace clients also? If so, how
would you enviasge this be handled?

I must mention though that in recent tests, I haven't been able to
recreate this issue consistantly.

Thoughts?

Ananth