Implementation of different software breakpoint kinds in gdb server

Implementation of different software breakpoint kinds in gdb server

[Michal Lesniewski]

Hi All,

I have been recently analyzing parts of GDB server code. I focused mainly on
the ARM specific code and checked if it is possible to add tracepoint
support for this architecture. I found out that there is a lot of work to do
that but that's not the point. The main problem is the internal
implementation of software breakpoints. 

The most general type of tracepoints are trap tracepoints (created using the
trace command). These are implemented internally using software breakpoints.
Software breakpoints are set by simply overwriting the instruction, at which
the program should stop, by a trap instruction or illegal instruction. When
the processor reaches the instruction, a exception occurs and gdbserver
handles it. 

Now the problem is that gdbserver always uses the same trap/illegal
instruction code. The functions in mem-break.c do not provide any way to
allow using different kinds of trap instructions depending on some
additional parameters. On most architectures this is OK, but on ARM there
are two instruction sets - ARM (32 bit) and Thumb (mixed 16 and 32 bit).
Depending on the instruction set used at a specific address, the trap
instruction should be different. 

I was wondering if anybody was already considering to extend the code in
mem-break.c to add support for different kinds of breakpoints. 

Best Regards,
Michal Lesniewski

Re: Implementation of different software breakpoint kinds in gdb server

[Pedro Alves]

On 10/18/2012 10:22 AM, Michal Lesniewski wrote:
> 
> I was wondering if anybody was already considering to extend the code in
> mem-break.c to add support for different kinds of breakpoints. 

Extending mem-break.c is not the big problem, IMO.  The RSP already supports
this, with the mode encoded in the "size" field of the z0 packet.

Offhand, the main issues with tracepoints on ARM are:

#1 - gdbserver needs to know how to step over the tracepoints, without gdb's intervention.

#2 - ARM can't hw single-step, so that needs to be done the hard way, with breakpoints
    (a.k.a., software single-step).
    All the logic to do that is in gdb.  This conflicts with #1.

So we'd need to teach gdbserver to software single-step.  Maybe it's possible
to tell offline all the possible destinations of an instruction, so we could still
leave that logic in gdb, but I suspect not.

I don't know whether the current kernel can already do all that for us? (perf,
uprobes, etc?)

-- 
Pedro Alves

Re: Implementation of different software breakpoint kinds in gdb server

[Yao Qi]

On 10/18/2012 06:08 PM, Pedro Alves wrote:
> Offhand, the main issues with tracepoints on ARM are:
>
> #1 - gdbserver needs to know how to step over the tracepoints, without gdb's intervention.
>
> #2 - ARM can't hw single-step, so that needs to be done the hard way, with breakpoints
>      (a.k.a., software single-step).
>      All the logic to do that is in gdb.  This conflicts with #1.
>
> So we'd need to teach gdbserver to software single-step.  Maybe it's possible
> to tell offline all the possible destinations of an instruction, so we could still
> leave that logic in gdb, but I suspect not.

I am wondering that it might be relatively easier to implement fast 
tracepoint in which step-over is not needed, if finding a jumppad is not 
a problem :)

-- 
Yao

Re: Implementation of different software breakpoint kinds in gdb server

[Pedro Alves]

On 10/18/2012 11:28 AM, Yao Qi wrote:
> On 10/18/2012 06:08 PM, Pedro Alves wrote:
>> Offhand, the main issues with tracepoints on ARM are:
>>
>> #1 - gdbserver needs to know how to step over the tracepoints, without gdb's intervention.
>>
>> #2 - ARM can't hw single-step, so that needs to be done the hard way, with breakpoints
>>      (a.k.a., software single-step).
>>      All the logic to do that is in gdb.  This conflicts with #1.
>>
>> So we'd need to teach gdbserver to software single-step.  Maybe it's possible
>> to tell offline all the possible destinations of an instruction, so we could still
>> leave that logic in gdb, but I suspect not.
> 
> I am wondering that it might be relatively easier to implement fast tracepoint in which step-over is not needed, if finding a jumppad is not a problem :)

Maybe.  :-)

The "move out of jumppad" logic uses single-stepping.  I guess we could come up
with something clever to avoid it.  Related, linux_fast_tracepoint_collecting logic
assumes it can identify a thread from its tls base / thread area.  And this assumes
you can retrieve this address without an infcall.  This is possible on x86.  Don't know
about ARM (well, in an ABI-stable way).

-- 
Pedro Alves

RE: Implementation of different software breakpoint kinds in gdb server

[Michal Lesniewski]

On 10/18/2012 12:09 PM, Pedro Alves wrote:
> Extending mem-break.c is not the big problem, IMO.  

But we'd probably have to change its interface a bit. So there would be also
changes needed in the architecture specific files like linux-*-low.c.
However, these changes would probably trivial for most architectures because
usually there is only one kind of trap instructions.

> The RSP already supports this, with the mode encoded in the "size" field
of the z0 packet.

That's right, but the RSP does not support specifying kinds/sizes in the
QTDP packets, which are used for adding tracepoints, but that's a different
story. I added a enhancement request on bugzilla today, maybe there will be
some feedback: http://sourceware.org/bugzilla/show_bug.cgi?id=14740

> So we'd need to teach gdbserver to software single-step.  Maybe it's
possible to tell offline all the possible destinations of an instruction, so
we could still leave that logic in gdb, but I suspect not.

It's generally not possible. Of course, some instructions can never cause a
branch, so in these cases we could safely set the "reinsert-breakpoint" at
the next instruction. But some branch instructions read the branch
destination from a register. In this case we can only evaluate the next PC
value when the breakpoint is hit. 

Teaching gdbserver to single-step shouldn't be too hard. As you mentioned,
the logic is already there in gdb. I even found the code to do this in
arm-tdep.c (e.g. arm_get_next_pc). We could use it as a base to implement it
in gdbserver, but we would have some code duplication.

> I don't know whether the current kernel can already do all that for us?
(perf, uprobes, etc?)

I tried uprobes, but it works in a different way. Instead of single
stepping, uprobes simulates (or emulates) the instruction, which was
replaced by the breakpoint. (It executes the instruction out-of-line and
jumps back to the original code. If this is not possible, it just looks at
the original instruction and modifies the registers and memory itself. This
solution has the advantage that it doesn't need to stop other threads while
one thread is single-stepping). Anyway, as far as I know uprobes does not
work on ARM yet. 

Michal Lesniewski

Re: Implementation of different software breakpoint kinds in gdb server

[Pedro Alves]

On 10/18/2012 12:44 PM, Michal Lesniewski wrote:
> On 10/18/2012 12:09 PM, Pedro Alves wrote:
>> Extending mem-break.c is not the big problem, IMO.  
> 
> But we'd probably have to change its interface a bit. So there would be also
> changes needed in the architecture specific files like linux-*-low.c.
> However, these changes would probably trivial for most architectures because
> usually there is only one kind of trap instructions.

Right.

> 
>> The RSP already supports this, with the mode encoded in the "size" field
> of the z0 packet.
> 
> That's right, but the RSP does not support specifying kinds/sizes in the
> QTDP packets, which are used for adding tracepoints, but that's a different
> story. I added a enhancement request on bugzilla today, maybe there will be
> some feedback: http://sourceware.org/bugzilla/show_bug.cgi?id=14740

Ah, didn't see it.  Most of the people that would probably respond
there are already in CC here.  :-P

Right.  Although we've revamped the tracepoints support not that long
ago (making it actually useful), the tracepoint packets are quite old,
and that wasn't considered originally (even the encoding of mode/kind in the
size/length field of z0 is recent).  QTDP supports optional extensions, but
it probably best to take a look at all the packets that are affected before
deciding how to extended QTDP.

> 
>> So we'd need to teach gdbserver to software single-step.  Maybe it's
> possible to tell offline all the possible destinations of an instruction, so
> we could still leave that logic in gdb, but I suspect not.
> 
> It's generally not possible. Of course, some instructions can never cause a
> branch, so in these cases we could safely set the "reinsert-breakpoint" at
> the next instruction. But some branch instructions read the branch
> destination from a register. In this case we can only evaluate the next PC
> value when the breakpoint is hit. 
> 
> Teaching gdbserver to single-step shouldn't be too hard. As you mentioned,
> the logic is already there in gdb. I even found the code to do this in
> arm-tdep.c (e.g. arm_get_next_pc). We could use it as a base to implement it
> in gdbserver, but we would have some code duplication.

It's not the next_pc bits per se, but the run control stuff that always
gets tricky.  Well, every time I think touching run control stuff in either
gdb or gdbserver shouldn't be hard, I spend long whiles head banging.
Maybe it's just me.  :-)

It'd be nice to avoid the duplication, though that might not be easy.

>> I don't know whether the current kernel can already do all that for us?
> (perf, uprobes, etc?)
> 
> I tried uprobes, but it works in a different way. Instead of single
> stepping, uprobes simulates (or emulates) the instruction, which was
> replaced by the breakpoint. (It executes the instruction out-of-line and
> jumps back to the original code. If this is not possible, it just looks at
> the original instruction and modifies the registers and memory itself. This
> solution has the advantage that it doesn't need to stop other threads while
> one thread is single-stepping). Anyway, as far as I know uprobes does not
> work on ARM yet. 

Thanks.

Looking forward to all this.

-- 
Pedro Alves

Re: Implementation of different software breakpoint kinds in gdb server

[John Gilmore]

> It's not the next_pc bits per se, but the run control stuff that always
> gets tricky.  Well, every time I think touching run control stuff in either
> gdb or gdbserver shouldn't be hard, I spend long whiles head banging.
> Maybe it's just me.  :-)

It's not just you.  For the first few years that I maintained GDB in
the 1990s, whenever I changed something in that big wait_for_inferior
jungle, I would introduce two bugs for every bug I fixed.  I got
pretty cautious about messing around in there.  Eventually I
started factoring the huge function into smaller functions, which
reduced the conceptual complexity somewhat.  Also, I gradually
got better at learning what would break it, and we also built up
a test suite that could test for obvious breakages.

> It'd be nice to avoid the duplication, though that might not be easy.

If you DO end up moving any of this infrastructure into the
target environment, I strongly recommend using common code.
It is hard enough to debug it once -- let alone debugging it
separately on each target as they evolve separate versions of
this very complicated code.

	John

RE: Implementation of different software breakpoint kinds in gdb server

[Michal Lesniewski]

Ok, so all this really looks like a lot of work. I think first I'll have a
look at the code in mem-break.c and try to add support for different kinds
of breakpoints. This must be done first anyway, I guess. I'll let you know
about my results, observations and further ideas. 

Thank you for your time and comments :-)

Michal Lesniewski