[RFC] Removing in process agent (IPA)

[RFC] Removing in process agent (IPA)

[Andrew Burgess]

Hi,

I'd like to have a conversation about whether or not we want to keep the
in-process-agent (IPA or libinproctrace.so).

A couple of recent patches[1][2] have triggered the question of whether
it would be better if we just dropped IPA from GDB.

We do have some tests that make use of IPA in gdb.trace/, so its no
untested code.

I'd describe myself as neutral on this topic.  I've never used the IPA
myself, but I have no real objections to it continuing to live on.
However, if nobody wants to keep it I'm happy to do to the work to
remove it from GDB.

So my question to the group: would anyone like to speak in favour of
keeping the IPA around?

Thanks,
Andrew

[1] https://inbox.sourceware.org/gdb-patches/cover.1706801009.git.aburgess@redhat.com
[2] https://inbox.sourceware.org/gdb-patches/f18d8f32e1e8af423e38efd0cb58ced1a98b5efb.1718271737.git.aburgess@redhat.com

Re: [RFC] Removing in process agent (IPA)

[Luis Machado]

On 6/13/24 15:17, Andrew Burgess via Gdb wrote:
> 
> Hi,
> 
> I'd like to have a conversation about whether or not we want to keep the
> in-process-agent (IPA or libinproctrace.so).
> 
> A couple of recent patches[1][2] have triggered the question of whether
> it would be better if we just dropped IPA from GDB.
> 
> We do have some tests that make use of IPA in gdb.trace/, so its no
> untested code.
> 
> I'd describe myself as neutral on this topic.  I've never used the IPA
> myself, but I have no real objections to it continuing to live on.
> However, if nobody wants to keep it I'm happy to do to the work to
> remove it from GDB.
> 
> So my question to the group: would anyone like to speak in favour of
> keeping the IPA around?
> 
> Thanks,
> Andrew
> 
> [1] https://inbox.sourceware.org/gdb-patches/cover.1706801009.git.aburgess@redhat.com
> [2] https://inbox.sourceware.org/gdb-patches/f18d8f32e1e8af423e38efd0cb58ced1a98b5efb.1718271737.git.aburgess@redhat.com
> 

Thanks for starting this Andrew.

For context into this rather obscure (nowadays) library, [1] shows the
original intended design and commit fa593d66d5696018bc8fb166f9e2a960d484ccd0
is the first pushed change to add the IPA, from what I could track down.

[1] https://sourceware.org/legacy-ml/gdb/2011-12/msg00004.html
[2] fa593d66d5696018bc8fb166f9e2a960d484ccd0

I think some of the design bits never made it into upstream gdb (itsets?), and
the tracepoints mechanism was mostly used by non-gdbserver debugging servers,
with gdbserver seemingly being more of a reference implementation.

Fast-forward to today, we have a lot of multi-core/multi-threaded systems
(even heterogeneous), but I can't really say the IPA library is being used
more (or at all), or the way it was intended back in the day. We do use little
bits (agent expressions), but not a lot.

Nowadays we also have better debugging tools to do the job, not necessarily in
gdb (RR, various tracing mechanisms).

I think it is important to remove maintenance burden if possible, and with
this perspective, I consider it is time for the IPA to go.

Similarly, I also think it might be time to consider dropping the tracepoints
(mainly static and fast) machinery from gdbserver, as I suspect that is another
mechanism that is not being used very often.

I recall trying to use tracepoints for practical purposes (back in the day), and
it wasn't up to the task, being too cumbersome to use, having bad failure modes
and having other security implications when trying to debug stuff in production
platforms. The remote protocol side of it is OK, but again, I bet no debugging
stubs are using that anymore.

Re: [RFC] Removing in process agent (IPA)

[Tom Tromey]

>> I'd like to have a conversation about whether or not we want to keep the
>> in-process-agent (IPA or libinproctrace.so).

Maybe you should ask on gdb@.

I personally have never used tracepoints or IPA.  However, I'm not
exactly in the target audience, either, since most of my debugging is on
gdb itself.

Luis> Similarly, I also think it might be time to consider dropping the tracepoints
Luis> (mainly static and fast) machinery from gdbserver, as I suspect that is another
Luis> mechanism that is not being used very often.

Something I found out relatively recently is that agent expressions are
useful for letting gdbserver evaluate (some) breakpoint conditions.
This happens automatically, when possible, and greatly improves
performance.

So, while I don't really mind either way about tracepoints, if we do
remove these we should be careful to keep this part.

Tom

RE: [RFC] Removing in process agent (IPA)

[Terekhov, Mikhail]

> -----Original Message-----
> From: Luis Machado <luis.machado@arm.com>
>
> Similarly, I also think it might be time to consider dropping the tracepoints
> (mainly static and fast) machinery from gdbserver, as I suspect that is another
> mechanism that is not being used very often.
>
We use tracepoints in our stub from the last millennia I guess (back from EMC days)
and plan to do so in the future.

> I recall trying to use tracepoints for practical purposes (back in the day), and it
> wasn't up to the task, being too cumbersome to use, having bad failure modes

Putting some UI/GUI on top of it makes it relatively simple to use.

> and having other security implications when trying to debug stuff in
> production platforms. The remote protocol side of it is OK, but again, I bet no
> debugging stubs are using that anymore.

We do use them but not in production though.

Regards,
Mikhail

Internal Use - Confidential

Re: [RFC] Removing in process agent (IPA)

[Luis Machado]

Hi Mikhail,

Thanks for the input.

On 6/13/24 19:11, Terekhov, Mikhail wrote:
>> -----Original Message-----
>> From: Luis Machado <luis.machado@arm.com>
>>
>> Similarly, I also think it might be time to consider dropping the tracepoints
>> (mainly static and fast) machinery from gdbserver, as I suspect that is another
>> mechanism that is not being used very often.
>>
> We use tracepoints in our stub from the last millennia I guess (back from EMC days)
> and plan to do so in the future.

Is it a custom remote debugging stub that uses tracepoints through the RSP packets gdb
sends? As opposed to using tracepoints through gdbserver itself?

> 
>> I recall trying to use tracepoints for practical purposes (back in the day), and it
>> wasn't up to the task, being too cumbersome to use, having bad failure modes
> 
> Putting some UI/GUI on top of it makes it relatively simple to use.
> 
>> and having other security implications when trying to debug stuff in
>> production platforms. The remote protocol side of it is OK, but again, I bet no
>> debugging stubs are using that anymore.
> 
> We do use them but not in production though.> 
> Regards,
> Mikhail
> 
> Internal Use - Confidential

RE: [RFC] Removing in process agent (IPA)

[Andrew Burgess]

"Terekhov, Mikhail" <Mikhail.Terekhov@dell.com> writes:

>> -----Original Message-----
>> From: Luis Machado <luis.machado@arm.com>
>>
>> Similarly, I also think it might be time to consider dropping the tracepoints
>> (mainly static and fast) machinery from gdbserver, as I suspect that is another
>> mechanism that is not being used very often.
>>
> We use tracepoints in our stub from the last millennia I guess (back from EMC days)
> and plan to do so in the future.
>
>> I recall trying to use tracepoints for practical purposes (back in the day), and it
>> wasn't up to the task, being too cumbersome to use, having bad failure modes
>
> Putting some UI/GUI on top of it makes it relatively simple to use.
>
>> and having other security implications when trying to debug stuff in
>> production platforms. The remote protocol side of it is OK, but again, I bet no
>> debugging stubs are using that anymore.
>
> We do use them but not in production though.

Just so I'm clear, you're talking about tracepoints here either in
gdbserver, or at least (if you have your own stub) in GDB and the remote
protocol, right?

This doesn't require gdb to continue shipping the IPA right?  Or have I
failed to understand your position?

Thanks,
Andrew

RE: [RFC] Removing in process agent (IPA)

[Terekhov, Mikhail]

Internal Use - Confidential
> -----Original Message-----
> From: Andrew Burgess <aburgess@redhat.com>
> Sent: Friday, June 14, 2024 4:08 AM
> To: Terekhov, Mikhail <Mikhail.Terekhov@dell.com>; Luis Machado
> <luis.machado@arm.com>; gdb-patches@sourceware.org;
> gdb@sourceware.org
> Cc: Pedro Alves <pedro@palves.net>
> Subject: RE: [RFC] Removing in process agent (IPA)
>
>
> [EXTERNAL EMAIL]
>
> "Terekhov, Mikhail" <Mikhail.Terekhov@dell.com> writes:
>
> >> -----Original Message-----
> >> From: Luis Machado <luis.machado@arm.com>
> >>
> >> Similarly, I also think it might be time to consider dropping the
> >> tracepoints (mainly static and fast) machinery from gdbserver, as I
> >> suspect that is another mechanism that is not being used very often.
> >>
> > We use tracepoints in our stub from the last millennia I guess (back
> > from EMC days) and plan to do so in the future.
> >
> >> I recall trying to use tracepoints for practical purposes (back in
> >> the day), and it wasn't up to the task, being too cumbersome to use,
> >> having bad failure modes
> >
> > Putting some UI/GUI on top of it makes it relatively simple to use.
> >
> >> and having other security implications when trying to debug stuff in
> >> production platforms. The remote protocol side of it is OK, but
> >> again, I bet no debugging stubs are using that anymore.
> >
> > We do use them but not in production though.
>
> Just so I'm clear, you're talking about tracepoints here either in gdbserver, or at
> least (if you have your own stub) in GDB and the remote protocol, right?
>
Yes, you are right, I'm talking about GDB, remote protocol and stub.
May be gdbserver in the future.

> This doesn't require gdb to continue shipping the IPA right?  Or have I failed to
> understand your position?
>
We do not use IPA explicitly and I did not check if it's code is used in the tracepoints implementation.
My replay was to the proposal to drop tracepoints.

Regards,
Mikhail

RE: [RFC] Removing in process agent (IPA)

[Terekhov, Mikhail]

> -----Original Message-----
> From: Luis Machado <luis.machado@arm.com>

Internal Use - Confidential
> On 6/13/24 19:11, Terekhov, Mikhail wrote:
> >> -----Original Message-----
> >> From: Luis Machado <luis.machado@arm.com>
> >>
> >> Similarly, I also think it might be time to consider dropping the
> >> tracepoints (mainly static and fast) machinery from gdbserver, as I
> >> suspect that is another mechanism that is not being used very often.
> >>
> > We use tracepoints in our stub from the last millennia I guess (back
> > from EMC days) and plan to do so in the future.
>
> Is it a custom remote debugging stub that uses tracepoints through the RSP
> packets gdb sends? As opposed to using tracepoints through gdbserver itself?
>
Both ways. No opposition😊

Regards,
Mikhail

Re: [RFC] Removing in process agent (IPA)

[Simon Marchi]

On 2024-06-14 04:08, Andrew Burgess wrote:
> "Terekhov, Mikhail" <Mikhail.Terekhov@dell.com> writes:
> 
>>> -----Original Message-----
>>> From: Luis Machado <luis.machado@arm.com>
>>>
>>> Similarly, I also think it might be time to consider dropping the tracepoints
>>> (mainly static and fast) machinery from gdbserver, as I suspect that is another
>>> mechanism that is not being used very often.
>>>
>> We use tracepoints in our stub from the last millennia I guess (back from EMC days)
>> and plan to do so in the future.
>>
>>> I recall trying to use tracepoints for practical purposes (back in the day), and it
>>> wasn't up to the task, being too cumbersome to use, having bad failure modes
>>
>> Putting some UI/GUI on top of it makes it relatively simple to use.
>>
>>> and having other security implications when trying to debug stuff in
>>> production platforms. The remote protocol side of it is OK, but again, I bet no
>>> debugging stubs are using that anymore.
>>
>> We do use them but not in production though.
> 
> Just so I'm clear, you're talking about tracepoints here either in
> gdbserver, or at least (if you have your own stub) in GDB and the remote
> protocol, right?
> 
> This doesn't require gdb to continue shipping the IPA right?  Or have I
> failed to understand your position?

Just to clarify one thing: the IPA is only needed for the fast
tracepoints mechanism, which allows recording trace data all in process,
without hitting a trap and the control going back to GDBserver.  We can
still use regular trap-based tracepoints without the IPA.

The possible levels of removal are:

1. Remove the IPA, thus removing the support for fast tracepoints in
   GDBserver (you could still use regular trap-based tracepoints)
2. Remove the support for tracepoints in GDBserver altogether (you could
   still use the tracepoints feature in GDB, when connecting to another
   RSP implementation than GDBserver, although the tracepoints feature
   in GDB would become untested code)
3. Remove the support for tracepoints in GDB and GDBserver completely
   (which I would not consider since Mikhail says they use that).

Simon

Re: [RFC] Removing in process agent (IPA)

[Luis Machado]

On 6/14/24 14:48, Simon Marchi wrote:
> 
> 
> On 2024-06-14 04:08, Andrew Burgess wrote:
>> "Terekhov, Mikhail" <Mikhail.Terekhov@dell.com> writes:
>>
>>>> -----Original Message-----
>>>> From: Luis Machado <luis.machado@arm.com>
>>>>
>>>> Similarly, I also think it might be time to consider dropping the tracepoints
>>>> (mainly static and fast) machinery from gdbserver, as I suspect that is another
>>>> mechanism that is not being used very often.
>>>>
>>> We use tracepoints in our stub from the last millennia I guess (back from EMC days)
>>> and plan to do so in the future.
>>>
>>>> I recall trying to use tracepoints for practical purposes (back in the day), and it
>>>> wasn't up to the task, being too cumbersome to use, having bad failure modes
>>>
>>> Putting some UI/GUI on top of it makes it relatively simple to use.
>>>
>>>> and having other security implications when trying to debug stuff in
>>>> production platforms. The remote protocol side of it is OK, but again, I bet no
>>>> debugging stubs are using that anymore.
>>>
>>> We do use them but not in production though.
>>
>> Just so I'm clear, you're talking about tracepoints here either in
>> gdbserver, or at least (if you have your own stub) in GDB and the remote
>> protocol, right?
>>
>> This doesn't require gdb to continue shipping the IPA right?  Or have I
>> failed to understand your position?
> 
> Just to clarify one thing: the IPA is only needed for the fast
> tracepoints mechanism, which allows recording trace data all in process,
> without hitting a trap and the control going back to GDBserver.  We can
> still use regular trap-based tracepoints without the IPA.
> 
> The possible levels of removal are:
> 
> 1. Remove the IPA, thus removing the support for fast tracepoints in
>    GDBserver (you could still use regular trap-based tracepoints)
> 2. Remove the support for tracepoints in GDBserver altogether (you could
>    still use the tracepoints feature in GDB, when connecting to another
>    RSP implementation than GDBserver, although the tracepoints feature
>    in GDB would become untested code)
> 3. Remove the support for tracepoints in GDB and GDBserver completely
>    (which I would not consider since Mikhail says they use that).

To be clear, I'm only proposing removing (1) at the moment. (2) might be worth
removing in the future if nobody is still using it (drops tracepoint support in
gdbserver). (3) is the more generic layer, and I suppose there are still a few
users out there.

Re: [RFC] Removing in process agent (IPA)

[Andrew Burgess]

Luis Machado <luis.machado@arm.com> writes:

> On 6/14/24 14:48, Simon Marchi wrote:
>> 
>> 
>> On 2024-06-14 04:08, Andrew Burgess wrote:
>>> "Terekhov, Mikhail" <Mikhail.Terekhov@dell.com> writes:
>>>
>>>>> -----Original Message-----
>>>>> From: Luis Machado <luis.machado@arm.com>
>>>>>
>>>>> Similarly, I also think it might be time to consider dropping the tracepoints
>>>>> (mainly static and fast) machinery from gdbserver, as I suspect that is another
>>>>> mechanism that is not being used very often.
>>>>>
>>>> We use tracepoints in our stub from the last millennia I guess (back from EMC days)
>>>> and plan to do so in the future.
>>>>
>>>>> I recall trying to use tracepoints for practical purposes (back in the day), and it
>>>>> wasn't up to the task, being too cumbersome to use, having bad failure modes
>>>>
>>>> Putting some UI/GUI on top of it makes it relatively simple to use.
>>>>
>>>>> and having other security implications when trying to debug stuff in
>>>>> production platforms. The remote protocol side of it is OK, but again, I bet no
>>>>> debugging stubs are using that anymore.
>>>>
>>>> We do use them but not in production though.
>>>
>>> Just so I'm clear, you're talking about tracepoints here either in
>>> gdbserver, or at least (if you have your own stub) in GDB and the remote
>>> protocol, right?
>>>
>>> This doesn't require gdb to continue shipping the IPA right?  Or have I
>>> failed to understand your position?
>> 
>> Just to clarify one thing: the IPA is only needed for the fast
>> tracepoints mechanism, which allows recording trace data all in process,
>> without hitting a trap and the control going back to GDBserver.  We can
>> still use regular trap-based tracepoints without the IPA.
>> 
>> The possible levels of removal are:
>> 
>> 1. Remove the IPA, thus removing the support for fast tracepoints in
>>    GDBserver (you could still use regular trap-based tracepoints)
>> 2. Remove the support for tracepoints in GDBserver altogether (you could
>>    still use the tracepoints feature in GDB, when connecting to another
>>    RSP implementation than GDBserver, although the tracepoints feature
>>    in GDB would become untested code)
>> 3. Remove the support for tracepoints in GDB and GDBserver completely
>>    (which I would not consider since Mikhail says they use that).
>
> To be clear, I'm only proposing removing (1) at the moment. (2) might be worth
> removing in the future if nobody is still using it (drops tracepoint support in
> gdbserver). (3) is the more generic layer, and I suppose there are still a few
> users out there.

While there are users of (3) I would suggest we should keep (2) so that
we (GDB project) can test (3).

My initial question was only about (1), dropping IPA and everything that
implies.  Again, I have no strong feelings either way, I asked only
because the idea was floated in some replies in other threads and I
thought the question should be asked in its own thread.

Thanks,
Andrew