[RFC] Changing gdbarch mid-execution

[RFC] Changing gdbarch mid-execution

[Luis Machado]

Hi,

I have a situation at hand and i'm thinking about how to best solve it.

AArch64 SVE has the capability of changing the vector length 
mid-execution. This can be done at the thread level.

Native GDB already supports this by looking at the ptrace data. But 
doing this for a remote target requires changes to the RSP.

Instead of changing things just for this particular case, i'm 
considering having a more general mechanism for updating the 
architecture data whenever such change is noticed by whoever is 
controlling the inferior.

My idea is to get the mechanism started by using the stop reply to send 
a new notification, say, "arch-changed".

That should trigger GDB to re-fetch the architecture data and 
reinitialize it.

In the particular case of SVE, we only need to fetch the target 
description again, so we have the proper vector length and data types set.

Does this sound like a useful feature? Or should i go for the solution 
with less impact that will only take care of re-fetching the target 
description?

Regards,
Luis

Re: [RFC] Changing gdbarch mid-execution

[Pedro Alves]

On 1/6/20 2:08 PM, Luis Machado wrote:
> Hi,
> 
> I have a situation at hand and i'm thinking about how to best solve it.
> 
> AArch64 SVE has the capability of changing the vector length mid-execution. This can be done at the thread level.
> 
> Native GDB already supports this by looking at the ptrace data. But doing this for a remote target requires changes to the RSP.
> 
> Instead of changing things just for this particular case, i'm considering having a more general mechanism for updating the architecture data whenever such change is noticed by whoever is controlling the inferior.
> 
> My idea is to get the mechanism started by using the stop reply to send a new notification, say, "arch-changed".
> 
> That should trigger GDB to re-fetch the architecture data and reinitialize it.
> 
> In the particular case of SVE, we only need to fetch the target description again, so we have the proper vector length and data types set.
> 
> Does this sound like a useful feature? Or should i go for the solution with less impact that will only take care of re-fetching the target description?

I'm not keep on the idea of potential constant re-fetching of arch data.
I'd think that "arch-changed=ARCH" with each arch having its own unique
name (can be opaque to GDB) so that GDB can cache the arch description,
and avoid refetching it over and over would be better.

Also, I don't think a state transition such a "arch changed" is the best.
I'd think making the stop reply say:

 "stopped on code running arch foo"

is better.

See this:

 https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/

In which Cagney suggested something very similar:

 T00;...;Architecture=<arch>;...
     The T packet is used to report the reason the target stopped to GDB. That packet includes information such as the processor and that processors registers. The packet can be extended to include the architecture of the processor that halted. 


Though for the SVE case, I'm not sure a target description change is the
best model, where you end up with a different target description description
for each potential different vector length.  

An alternative could be for the target description to always describe the
largest possible vector file, or explicitly describe the VLE registers as variable
length, and then gdb would handle presenting the usable registers.  GDB should
be able to tell the size of the vector registers by looking at the VQ (or was
it VL?  Or whatever it is called) register.

In effect, we can see the current native support as a middle ground,
where aarch64_linux_nat_target::thread_architecture returns a different
gdbarch, there's no target description re-fetching, I believe.

Thanks,
Pedro Alves

Re: [RFC] Changing gdbarch mid-execution

[Luis Machado]

On 1/22/20 11:56 AM, Pedro Alves wrote:
> On 1/6/20 2:08 PM, Luis Machado wrote:
>> Hi,
>>
>> I have a situation at hand and i'm thinking about how to best solve it.
>>
>> AArch64 SVE has the capability of changing the vector length mid-execution. This can be done at the thread level.
>>
>> Native GDB already supports this by looking at the ptrace data. But doing this for a remote target requires changes to the RSP.
>>
>> Instead of changing things just for this particular case, i'm considering having a more general mechanism for updating the architecture data whenever such change is noticed by whoever is controlling the inferior.
>>
>> My idea is to get the mechanism started by using the stop reply to send a new notification, say, "arch-changed".
>>
>> That should trigger GDB to re-fetch the architecture data and reinitialize it.
>>
>> In the particular case of SVE, we only need to fetch the target description again, so we have the proper vector length and data types set.
>>
>> Does this sound like a useful feature? Or should i go for the solution with less impact that will only take care of re-fetching the target description?
> 
> I'm not keep on the idea of potential constant re-fetching of arch data.
> I'd think that "arch-changed=ARCH" with each arch having its own unique
> name (can be opaque to GDB) so that GDB can cache the arch description,
> and avoid refetching it over and over would be better.

I don't like the re-fetching either, so i'm trying to minimize that.

Part of the problem is that the vector length (VL) is per-thread, ...

> 
> Also, I don't think a state transition such a "arch changed" is the best.
> I'd think making the stop reply say:
> 
>   "stopped on code running arch foo"
> 
> is better.
> 
> See this:
> 
>   https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
> 
> In which Cagney suggested something very similar:
> 
>   T00;...;Architecture=<arch>;...
>       The T packet is used to report the reason the target stopped to GDB. That packet includes information such as the processor and that processors registers. The packet can be extended to include the architecture of the processor that halted.
> 

... so the above, even though it works nicely for reporting the stop of 
a single thread, it won't carry information about potential other 
threads that stopped along with the one the caused the stop reply to be 
sent, right? We would need to fetch updates from the other threads in 
case they changed their VL during execution.

> 
> Though for the SVE case, I'm not sure a target description change is the
> best model, where you end up with a different target description description
> for each potential different vector length.

Right. A new target description comes along with new sizes for the 
particular types and aggregates it defines.

> 
> An alternative could be for the target description to always describe the
> largest possible vector file, or explicitly describe the VLE registers as variable
> length, and then gdb would handle presenting the usable registers.  GDB should
> be able to tell the size of the vector registers by looking at the VQ (or was
> it VL?  Or whatever it is called) register.

The variable length description is technically more correct, but i think 
we already opted for a different solution with multiple VL-based target 
descriptions.

My idea is to not rely on register values and, instead, focus on sizes 
of some aggregates the target description defines. That way we are not 
forced to fetch any registers and can infer the vector length from the 
sizes on the new target description.

Both native sides (GDB and gdbserver) and QEMU know how to detect VL 
changes. It is just the communication of that change to GDB that we need 
to sort out via RSP.

> 
> In effect, we can see the current native support as a middle ground,
> where aarch64_linux_nat_target::thread_architecture returns a different
> gdbarch, there's no target description re-fetching, I believe.
> 

There is no re-fetching in the sense that data doesn't get passed 
around, but new target descriptions do get created dynamically 
(aarch64_create_target_description) based on the new VL. The resulting 
gdbarch then gets cached so we don't need to recreate that particular 
variation.

My idea for a RSP-based target description update tries to mimic that as 
follows...

- Remote end notices a target description / gdbarch change and notifies 
GDB via a stop reply packet entry.

- GDB fetches the stop reply data and knows it has to query the remote 
about what particular threads had their target descriptions updated. I 
think this needs to be a new packet, maybe a qXfer one with a different 
object. The qXfer packet would handle large lists of threads (thinking 
about future use cases, GPU's etc).

- Remote sends a list of threads to GDB.

- GDB fetches the list of threads it needs to re-fetch the target 
descriptions from and proceeds to query the remote about those 
descriptions. I think we could cache the descriptions here, or have an 
opaque description that gets passed down to the target-specific code as 
you suggested.

- GDB finishes the update and caches (as much as possible) the gdbarch 
per-thread/per-regcache.

When no target description change has taken place we have nothing to do 
and no RSP overhead, so it wouldn't slow things down.

Does the above sound like an acceptable way forward?

Luis

Re: [RFC] Changing gdbarch mid-execution

[Luis Machado]

CC-ing Alan as well.

On 1/22/20 2:03 PM, Luis Machado wrote:
> On 1/22/20 11:56 AM, Pedro Alves wrote:
>> On 1/6/20 2:08 PM, Luis Machado wrote:
>>> Hi,
>>>
>>> I have a situation at hand and i'm thinking about how to best solve it.
>>>
>>> AArch64 SVE has the capability of changing the vector length 
>>> mid-execution. This can be done at the thread level.
>>>
>>> Native GDB already supports this by looking at the ptrace data. But 
>>> doing this for a remote target requires changes to the RSP.
>>>
>>> Instead of changing things just for this particular case, i'm 
>>> considering having a more general mechanism for updating the 
>>> architecture data whenever such change is noticed by whoever is 
>>> controlling the inferior.
>>>
>>> My idea is to get the mechanism started by using the stop reply to 
>>> send a new notification, say, "arch-changed".
>>>
>>> That should trigger GDB to re-fetch the architecture data and 
>>> reinitialize it.
>>>
>>> In the particular case of SVE, we only need to fetch the target 
>>> description again, so we have the proper vector length and data types 
>>> set.
>>>
>>> Does this sound like a useful feature? Or should i go for the 
>>> solution with less impact that will only take care of re-fetching the 
>>> target description?
>>
>> I'm not keep on the idea of potential constant re-fetching of arch data.
>> I'd think that "arch-changed=ARCH" with each arch having its own unique
>> name (can be opaque to GDB) so that GDB can cache the arch description,
>> and avoid refetching it over and over would be better.
> 
> I don't like the re-fetching either, so i'm trying to minimize that.
> 
> Part of the problem is that the vector length (VL) is per-thread, ...
> 
>>
>> Also, I don't think a state transition such a "arch changed" is the best.
>> I'd think making the stop reply say:
>>
>>   "stopped on code running arch foo"
>>
>> is better.
>>
>> See this:
>>
>>   https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
>>
>> In which Cagney suggested something very similar:
>>
>>   T00;...;Architecture=<arch>;...
>>       The T packet is used to report the reason the target stopped to 
>> GDB. That packet includes information such as the processor and that 
>> processors registers. The packet can be extended to include the 
>> architecture of the processor that halted.
>>
> 
> ... so the above, even though it works nicely for reporting the stop of 
> a single thread, it won't carry information about potential other 
> threads that stopped along with the one the caused the stop reply to be 
> sent, right? We would need to fetch updates from the other threads in 
> case they changed their VL during execution.
> 
>>
>> Though for the SVE case, I'm not sure a target description change is the
>> best model, where you end up with a different target description 
>> description
>> for each potential different vector length.
> 
> Right. A new target description comes along with new sizes for the 
> particular types and aggregates it defines.
> 
>>
>> An alternative could be for the target description to always describe the
>> largest possible vector file, or explicitly describe the VLE registers 
>> as variable
>> length, and then gdb would handle presenting the usable registers.  
>> GDB should
>> be able to tell the size of the vector registers by looking at the VQ 
>> (or was
>> it VL?  Or whatever it is called) register.
> 
> The variable length description is technically more correct, but i think 
> we already opted for a different solution with multiple VL-based target 
> descriptions.
> 
> My idea is to not rely on register values and, instead, focus on sizes 
> of some aggregates the target description defines. That way we are not 
> forced to fetch any registers and can infer the vector length from the 
> sizes on the new target description.
> 
> Both native sides (GDB and gdbserver) and QEMU know how to detect VL 
> changes. It is just the communication of that change to GDB that we need 
> to sort out via RSP.
> 
>>
>> In effect, we can see the current native support as a middle ground,
>> where aarch64_linux_nat_target::thread_architecture returns a different
>> gdbarch, there's no target description re-fetching, I believe.
>>
> 
> There is no re-fetching in the sense that data doesn't get passed 
> around, but new target descriptions do get created dynamically 
> (aarch64_create_target_description) based on the new VL. The resulting 
> gdbarch then gets cached so we don't need to recreate that particular 
> variation.
> 
> My idea for a RSP-based target description update tries to mimic that as 
> follows...
> 
> - Remote end notices a target description / gdbarch change and notifies 
> GDB via a stop reply packet entry.
> 
> - GDB fetches the stop reply data and knows it has to query the remote 
> about what particular threads had their target descriptions updated. I 
> think this needs to be a new packet, maybe a qXfer one with a different 
> object. The qXfer packet would handle large lists of threads (thinking 
> about future use cases, GPU's etc).
> 
> - Remote sends a list of threads to GDB.
> 
> - GDB fetches the list of threads it needs to re-fetch the target 
> descriptions from and proceeds to query the remote about those 
> descriptions. I think we could cache the descriptions here, or have an 
> opaque description that gets passed down to the target-specific code as 
> you suggested.
> 
> - GDB finishes the update and caches (as much as possible) the gdbarch 
> per-thread/per-regcache.
> 
> When no target description change has taken place we have nothing to do 
> and no RSP overhead, so it wouldn't slow things down.
> 
> Does the above sound like an acceptable way forward?
> 
> Luis
> 

Re: [RFC] Changing gdbarch mid-execution

[Alan Hayward]

> On 22 Jan 2020, at 17:04, Luis Machado <luis.machado@linaro.org> wrote:
> 
> CC-ing Alan as well.
> 
> On 1/22/20 2:03 PM, Luis Machado wrote:
>> On 1/22/20 11:56 AM, Pedro Alves wrote:
>>> On 1/6/20 2:08 PM, Luis Machado wrote:
>>>> Hi,
>>>> 
>>>> I have a situation at hand and i'm thinking about how to best solve it.
>>>> 
>>>> AArch64 SVE has the capability of changing the vector length mid-execution. This can be done at the thread level.
>>>> 
>>>> Native GDB already supports this by looking at the ptrace data. But doing this for a remote target requires changes to the RSP.
>>>> 
>>>> Instead of changing things just for this particular case, i'm considering having a more general mechanism for updating the architecture data whenever such change is noticed by whoever is controlling the inferior.
>>>> 
>>>> My idea is to get the mechanism started by using the stop reply to send a new notification, say, "arch-changed".
>>>> 
>>>> That should trigger GDB to re-fetch the architecture data and reinitialize it.
>>>> 
>>>> In the particular case of SVE, we only need to fetch the target description again, so we have the proper vector length and data types set.
>>>> 
>>>> Does this sound like a useful feature? Or should i go for the solution with less impact that will only take care of re-fetching the target description?
>>> 
>>> I'm not keep on the idea of potential constant re-fetching of arch data.
>>> I'd think that "arch-changed=ARCH" with each arch having its own unique
>>> name (can be opaque to GDB) so that GDB can cache the arch description,
>>> and avoid refetching it over and over would be better.
>> I don't like the re-fetching either, so i'm trying to minimize that.
>> Part of the problem is that the vector length (VL) is per-thread, ...
>>> 
>>> Also, I don't think a state transition such a "arch changed" is the best.
>>> I'd think making the stop reply say:
>>> 
>>>   "stopped on code running arch foo"
>>> 
>>> is better.
>>> 
>>> See this:
>>> 
>>>   https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
>>> 
>>> In which Cagney suggested something very similar:
>>> 
>>>   T00;...;Architecture=<arch>;...
>>>       The T packet is used to report the reason the target stopped to GDB. That packet includes information such as the processor and that processors registers. The packet can be extended to include the architecture of the processor that halted.
>>> 
>> ... so the above, even though it works nicely for reporting the stop of a single thread, it won't carry information about potential other threads that stopped along with the one the caused the stop reply to be sent, right? We would need to fetch updates from the other threads in case they changed their VL during execution.
>>> 
>>> Though for the SVE case, I'm not sure a target description change is the
>>> best model, where you end up with a different target description description
>>> for each potential different vector length.
>> Right. A new target description comes along with new sizes for the particular types and aggregates it defines.
>>> 
>>> An alternative could be for the target description to always describe the
>>> largest possible vector file, or explicitly describe the VLE registers as variable
>>> length, and then gdb would handle presenting the usable registers.  GDB should
>>> be able to tell the size of the vector registers by looking at the VQ (or was
>>> it VL?  Or whatever it is called) register.
>> The variable length description is technically more correct, but i think we already opted for a different solution with multiple VL-based target descriptions.


My big worry With a variable length description is that the gdbtype system would fall over.
I’ve not delved too much in that area, so I’m not sure how much would need fixing up.

You’d also want to make sure it doesn’t always allocate the maximum size register bank,
otherwise with lots of threads, that’s a large overhead.

If this does end up being the solution, then maybe a later task, after everything is working,
would be change the local target version to work in the same way.

>> My idea is to not rely on register values and, instead, focus on sizes of some aggregates the target description defines. That way we are not forced to fetch any registers and can infer the vector length from the sizes on the new target description.
>> Both native sides (GDB and gdbserver) and QEMU know how to detect VL changes. It is just the communication of that change to GDB that we need to sort out via RSP.
>>> 
>>> In effect, we can see the current native support as a middle ground,
>>> where aarch64_linux_nat_target::thread_architecture returns a different
>>> gdbarch, there's no target description re-fetching, I believe.
>>> 
>> There is no re-fetching in the sense that data doesn't get passed around, but new target descriptions do get created dynamically (aarch64_create_target_description) based on the new VL. The resulting gdbarch then gets cached so we don't need to recreate that particular variation.
>> My idea for a RSP-based target description update tries to mimic that as follows...
>> - Remote end notices a target description / gdbarch change and notifies GDB via a stop reply packet entry.

Thinking about this again... problem now is that when the remote stops, the remote now needs to do a VL read for
every thread. That’s going to horrible when we get to lots of threads. Then that information needs putting
into the stop reply thread. Imagine a program with 500 threads, and 200 of them have changed vector length.


>> - GDB fetches the stop reply data and knows it has to query the remote about what particular threads had their target descriptions updated. I think this needs to be a new packet, maybe a qXfer one with a different object. The qXfer packet would handle large lists of threads (thinking about future use cases, GPU's etc).
>> - Remote sends a list of threads to GDB.
>> - GDB fetches the list of threads it needs to re-fetch the target descriptions from and proceeds to query the remote about those descriptions. I think we could cache the descriptions here, or have an opaque description that gets passed down to the target-specific code as you suggested.
>> - GDB finishes the update and caches (as much as possible) the gdbarch per-thread/per-regcache.
>> When no target description change has taken place we have nothing to do and no RSP overhead, so it wouldn't slow things down.
>> Does the above sound like an acceptable way forward?
>> Luis

Re: [RFC] Changing gdbarch mid-execution

[Pedro Alves]

On 1/22/20 5:03 PM, Luis Machado wrote:
> On 1/22/20 11:56 AM, Pedro Alves wrote:
>> On 1/6/20 2:08 PM, Luis Machado wrote:
>>> Hi,
>>>
>>> I have a situation at hand and i'm thinking about how to best solve it.
>>>
>>> AArch64 SVE has the capability of changing the vector length mid-execution. This can be done at the thread level.
>>>
>>> Native GDB already supports this by looking at the ptrace data. But doing this for a remote target requires changes to the RSP.
>>>
>>> Instead of changing things just for this particular case, i'm considering having a more general mechanism for updating the architecture data whenever such change is noticed by whoever is controlling the inferior.
>>>
>>> My idea is to get the mechanism started by using the stop reply to send a new notification, say, "arch-changed".
>>>
>>> That should trigger GDB to re-fetch the architecture data and reinitialize it.
>>>
>>> In the particular case of SVE, we only need to fetch the target description again, so we have the proper vector length and data types set.
>>>
>>> Does this sound like a useful feature? Or should i go for the solution with less impact that will only take care of re-fetching the target description?
>>
>> I'm not keep on the idea of potential constant re-fetching of arch data.
>> I'd think that "arch-changed=ARCH" with each arch having its own unique
>> name (can be opaque to GDB) so that GDB can cache the arch description,
>> and avoid refetching it over and over would be better.
> 
> I don't like the re-fetching either, so i'm trying to minimize that.
> 
> Part of the problem is that the vector length (VL) is per-thread, ...
> 
>>
>> Also, I don't think a state transition such a "arch changed" is the best.
>> I'd think making the stop reply say:
>>
>>   "stopped on code running arch foo"
>>
>> is better.
>>
>> See this:
>>
>>   https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
>>
>> In which Cagney suggested something very similar:
>>
>>   T00;...;Architecture=<arch>;...
>>       The T packet is used to report the reason the target stopped to GDB. That packet includes information such as the processor and that processors registers. The packet can be extended to include the architecture of the processor that halted.
>>
> 
> ... so the above, even though it works nicely for reporting the stop of a single thread, it won't carry information about potential other threads that stopped along with the one the caused the stop reply to be sent, right? We would need to fetch updates from the other threads in case they changed their VL during execution.

Yes, but we also need to fetch their registers anyway too.  It's just
another piece of information that we need to retrieve.  Maybe we could
include the "current architecture" info in the 'g' packet's reply.

> 
>>
>> Though for the SVE case, I'm not sure a target description change is the
>> best model, where you end up with a different target description description
>> for each potential different vector length.
> 
> Right. A new target description comes along with new sizes for the particular types and aggregates it defines.
> 
>>
>> An alternative could be for the target description to always describe the
>> largest possible vector file, or explicitly describe the VLE registers as variable
>> length, and then gdb would handle presenting the usable registers.  GDB should
>> be able to tell the size of the vector registers by looking at the VQ (or was
>> it VL?  Or whatever it is called) register.
> 
> The variable length description is technically more correct, but i think we already opted for a different solution with multiple VL-based target descriptions.

We did?

> 
> My idea is to not rely on register values and, instead, focus on sizes of some aggregates the target description defines. That way we are not forced to fetch any registers and can infer the vector length from the sizes on the new target description.
> 
> Both native sides (GDB and gdbserver) and QEMU know how to detect VL changes. It is just the communication of that change to GDB that we need to sort out via RSP.

I don't know about QEMU, but the native side communicates the vector length with
the kernel exactly via a register..

> 
>>
>> In effect, we can see the current native support as a middle ground,
>> where aarch64_linux_nat_target::thread_architecture returns a different
>> gdbarch, there's no target description re-fetching, I believe.
>>
> 
> There is no re-fetching in the sense that data doesn't get passed around, but new target descriptions do get created dynamically (aarch64_create_target_description) based on the new VL. The resulting gdbarch then gets cached so we don't need to recreate that particular variation.

Which is still a lot more work than reporting "N" size, where N is a simple number.
Tdescs are de-dupped by hashing/comparing the whole tdesc xml text, for example.

And how are you going to force a vector length change from GDB, if you go the
tdesc route?  With the register-like approach, you can just write to a register,
or something like it with a known value.  With the tdesc route, you can't, it's
read-only.  Writes _must_ be considered, for inferior function calls,
for example.  That's one of the features of the current ptrace interface
design, see:

http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/thread.html#478941
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/479843.html

> 
> My idea for a RSP-based target description update tries to mimic that as follows...
> 
> - Remote end notices a target description / gdbarch change and notifies GDB via a stop reply packet entry.
> 
> - GDB fetches the stop reply data and knows it has to query the remote about what particular threads had their target descriptions updated. I think this needs to be a new packet, maybe a qXfer one with a different object. The qXfer packet would handle large lists of threads (thinking about future use cases, GPU's etc).
> 
> - Remote sends a list of threads to GDB.
> 
> - GDB fetches the list of threads it needs to re-fetch the target descriptions from and proceeds to query the remote about those descriptions. I think we could cache the descriptions here, or have an opaque description that gets passed down to the target-specific code as you suggested.
> 
> - GDB finishes the update and caches (as much as possible) the gdbarch per-thread/per-regcache.
> 
> When no target description change has taken place we have nothing to do and no RSP overhead, so it wouldn't slow things down.
> 
> Does the above sound like an acceptable way forward?
I'm afraid this doesn't consider the whole story.

Thanks,
Pedro Alves

Re: [RFC] Changing gdbarch mid-execution

[Luis Machado]

On 1/23/20 1:06 PM, Pedro Alves wrote:
> On 1/22/20 5:03 PM, Luis Machado wrote:
>> On 1/22/20 11:56 AM, Pedro Alves wrote:
>>> On 1/6/20 2:08 PM, Luis Machado wrote:
>>>> Hi,
>>>>
>>>> I have a situation at hand and i'm thinking about how to best solve it.
>>>>
>>>> AArch64 SVE has the capability of changing the vector length mid-execution. This can be done at the thread level.
>>>>
>>>> Native GDB already supports this by looking at the ptrace data. But doing this for a remote target requires changes to the RSP.
>>>>
>>>> Instead of changing things just for this particular case, i'm considering having a more general mechanism for updating the architecture data whenever such change is noticed by whoever is controlling the inferior.
>>>>
>>>> My idea is to get the mechanism started by using the stop reply to send a new notification, say, "arch-changed".
>>>>
>>>> That should trigger GDB to re-fetch the architecture data and reinitialize it.
>>>>
>>>> In the particular case of SVE, we only need to fetch the target description again, so we have the proper vector length and data types set.
>>>>
>>>> Does this sound like a useful feature? Or should i go for the solution with less impact that will only take care of re-fetching the target description?
>>>
>>> I'm not keep on the idea of potential constant re-fetching of arch data.
>>> I'd think that "arch-changed=ARCH" with each arch having its own unique
>>> name (can be opaque to GDB) so that GDB can cache the arch description,
>>> and avoid refetching it over and over would be better.
>>
>> I don't like the re-fetching either, so i'm trying to minimize that.
>>
>> Part of the problem is that the vector length (VL) is per-thread, ...
>>
>>>
>>> Also, I don't think a state transition such a "arch changed" is the best.
>>> I'd think making the stop reply say:
>>>
>>>    "stopped on code running arch foo"
>>>
>>> is better.
>>>
>>> See this:
>>>
>>>    https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
>>>
>>> In which Cagney suggested something very similar:
>>>
>>>    T00;...;Architecture=<arch>;...
>>>        The T packet is used to report the reason the target stopped to GDB. That packet includes information such as the processor and that processors registers. The packet can be extended to include the architecture of the processor that halted.
>>>
>>
>> ... so the above, even though it works nicely for reporting the stop of a single thread, it won't carry information about potential other threads that stopped along with the one the caused the stop reply to be sent, right? We would need to fetch updates from the other threads in case they changed their VL during execution.
> 
> Yes, but we also need to fetch their registers anyway too.  It's just
> another piece of information that we need to retrieve.  Maybe we could
> include the "current architecture" info in the 'g' packet's reply.
> 

By "current architecture" you mean a reference/id to a particular target 
description or, more explicitly, a vg value? Like "aarch64-sve-8" for a 
SVE description with vg==8?

If we go the route of using the vg value to create another target 
description locally (not passed via RSP), then it sounds like we could 
just use the expedited vg register or vg value passed in via the 
register fetch/store packets, since that is a static-sized register that 
we can always parse?

By using register fetches/stores we can keep both gdb/gdbserver updated.

Whenever gdbserver sees a change in vg, gdb will be informed of that by 
fetching registers. Similarly, whenever gdb writes to vg, gdbserver will 
know about it and will update its view of the target description.

gdb and the remote stub would need to be kept in sync regarding their 
views of the vector length.

Would this be more acceptable?

>>
>>>
>>> Though for the SVE case, I'm not sure a target description change is the
>>> best model, where you end up with a different target description description
>>> for each potential different vector length.
>>
>> Right. A new target description comes along with new sizes for the particular types and aggregates it defines.
>>
>>>
>>> An alternative could be for the target description to always describe the
>>> largest possible vector file, or explicitly describe the VLE registers as variable
>>> length, and then gdb would handle presenting the usable registers.  GDB should
>>> be able to tell the size of the vector registers by looking at the VQ (or was
>>> it VL?  Or whatever it is called) register.
>>
>> The variable length description is technically more correct, but i think we already opted for a different solution with multiple VL-based target descriptions.
> 
> We did?
> 

Well, we already have a working solution with support for changing 
vector lengths on both gdb and gdbserver. What we are missing is a way 
to do that via RSP, which would enable us to support changing vector 
lengths in QEMU, for example.

I'd rather avoid having to change some of gdb's type framework at this 
moment to support a sizeless type.

>>
>> My idea is to not rely on register values and, instead, focus on sizes of some aggregates the target description defines. That way we are not forced to fetch any registers and can infer the vector length from the sizes on the new target description.
>>
>> Both native sides (GDB and gdbserver) and QEMU know how to detect VL changes. It is just the communication of that change to GDB that we need to sort out via RSP.
> 
> I don't know about QEMU, but the native side communicates the vector length with
> the kernel exactly via a register..
> 
>>
>>>
>>> In effect, we can see the current native support as a middle ground,
>>> where aarch64_linux_nat_target::thread_architecture returns a different
>>> gdbarch, there's no target description re-fetching, I believe.
>>>
>>
>> There is no re-fetching in the sense that data doesn't get passed around, but new target descriptions do get created dynamically (aarch64_create_target_description) based on the new VL. The resulting gdbarch then gets cached so we don't need to recreate that particular variation.
> 
> Which is still a lot more work than reporting "N" size, where N is a simple number.
> Tdescs are de-dupped by hashing/comparing the whole tdesc xml text, for example.
> 
> And how are you going to force a vector length change from GDB, if you go the
> tdesc route?  With the register-like approach, you can just write to a register,
> or something like it with a known value.  With the tdesc route, you can't, it's
> read-only.  Writes _must_ be considered, for inferior function calls,
> for example.  That's one of the features of the current ptrace interface
> design, see:

I agree inferior function calls would get a bit more complicated to support.

If we use vg directly, then it will be simpler, though a little less 
generic.

> 
> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/thread.html#478941
> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/479843.html
> 
>>
>> My idea for a RSP-based target description update tries to mimic that as follows...
>>
>> - Remote end notices a target description / gdbarch change and notifies GDB via a stop reply packet entry.
>>
>> - GDB fetches the stop reply data and knows it has to query the remote about what particular threads had their target descriptions updated. I think this needs to be a new packet, maybe a qXfer one with a different object. The qXfer packet would handle large lists of threads (thinking about future use cases, GPU's etc).
>>
>> - Remote sends a list of threads to GDB.
>>
>> - GDB fetches the list of threads it needs to re-fetch the target descriptions from and proceeds to query the remote about those descriptions. I think we could cache the descriptions here, or have an opaque description that gets passed down to the target-specific code as you suggested.
>>
>> - GDB finishes the update and caches (as much as possible) the gdbarch per-thread/per-regcache.
>>
>> When no target description change has taken place we have nothing to do and no RSP overhead, so it wouldn't slow things down.
>>
>> Does the above sound like an acceptable way forward?
> I'm afraid this doesn't consider the whole story.
> 
> Thanks,
> Pedro Alves
> 

Re: [RFC] Changing gdbarch mid-execution

[Tim Newsome]

I'm coincidentally working on RISC-V vector support, where the same problem
exists. My preferred solution in that case is to export the (potentially
very wide) v0 -- v31 registers which hold the raw data and have a fixed
width. Then gdb can use the contents of vsew and vl to present this raw
data to the user, but broken up into the elements as the hardware treats it
currently.

That feels much simpler than trying to change the architecture on the fly,
but I'm far from a gdb expert.

Tim

On Tue, Jan 28, 2020 at 11:21 AM Luis Machado <luis.machado@linaro.org>
wrote:

> On 1/23/20 1:06 PM, Pedro Alves wrote:
> > On 1/22/20 5:03 PM, Luis Machado wrote:
> >> On 1/22/20 11:56 AM, Pedro Alves wrote:
> >>> On 1/6/20 2:08 PM, Luis Machado wrote:
> >>>> Hi,
> >>>>
> >>>> I have a situation at hand and i'm thinking about how to best solve
> it.
> >>>>
> >>>> AArch64 SVE has the capability of changing the vector length
> mid-execution. This can be done at the thread level.
> >>>>
> >>>> Native GDB already supports this by looking at the ptrace data. But
> doing this for a remote target requires changes to the RSP.
> >>>>
> >>>> Instead of changing things just for this particular case, i'm
> considering having a more general mechanism for updating the architecture
> data whenever such change is noticed by whoever is controlling the inferior.
> >>>>
> >>>> My idea is to get the mechanism started by using the stop reply to
> send a new notification, say, "arch-changed".
> >>>>
> >>>> That should trigger GDB to re-fetch the architecture data and
> reinitialize it.
> >>>>
> >>>> In the particular case of SVE, we only need to fetch the target
> description again, so we have the proper vector length and data types set.
> >>>>
> >>>> Does this sound like a useful feature? Or should i go for the
> solution with less impact that will only take care of re-fetching the
> target description?
> >>>
> >>> I'm not keep on the idea of potential constant re-fetching of arch
> data.
> >>> I'd think that "arch-changed=ARCH" with each arch having its own unique
> >>> name (can be opaque to GDB) so that GDB can cache the arch description,
> >>> and avoid refetching it over and over would be better.
> >>
> >> I don't like the re-fetching either, so i'm trying to minimize that.
> >>
> >> Part of the problem is that the vector length (VL) is per-thread, ...
> >>
> >>>
> >>> Also, I don't think a state transition such a "arch changed" is the
> best.
> >>> I'd think making the stop reply say:
> >>>
> >>>    "stopped on code running arch foo"
> >>>
> >>> is better.
> >>>
> >>> See this:
> >>>
> >>>    https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
> >>>
> >>> In which Cagney suggested something very similar:
> >>>
> >>>    T00;...;Architecture=<arch>;...
> >>>        The T packet is used to report the reason the target stopped to
> GDB. That packet includes information such as the processor and that
> processors registers. The packet can be extended to include the
> architecture of the processor that halted.
> >>>
> >>
> >> ... so the above, even though it works nicely for reporting the stop of
> a single thread, it won't carry information about potential other threads
> that stopped along with the one the caused the stop reply to be sent,
> right? We would need to fetch updates from the other threads in case they
> changed their VL during execution.
> >
> > Yes, but we also need to fetch their registers anyway too.  It's just
> > another piece of information that we need to retrieve.  Maybe we could
> > include the "current architecture" info in the 'g' packet's reply.
> >
>
> By "current architecture" you mean a reference/id to a particular target
> description or, more explicitly, a vg value? Like "aarch64-sve-8" for a
> SVE description with vg==8?
>
> If we go the route of using the vg value to create another target
> description locally (not passed via RSP), then it sounds like we could
> just use the expedited vg register or vg value passed in via the
> register fetch/store packets, since that is a static-sized register that
> we can always parse?
>
> By using register fetches/stores we can keep both gdb/gdbserver updated.
>
> Whenever gdbserver sees a change in vg, gdb will be informed of that by
> fetching registers. Similarly, whenever gdb writes to vg, gdbserver will
> know about it and will update its view of the target description.
>
> gdb and the remote stub would need to be kept in sync regarding their
> views of the vector length.
>
> Would this be more acceptable?
>
> >>
> >>>
> >>> Though for the SVE case, I'm not sure a target description change is
> the
> >>> best model, where you end up with a different target description
> description
> >>> for each potential different vector length.
> >>
> >> Right. A new target description comes along with new sizes for the
> particular types and aggregates it defines.
> >>
> >>>
> >>> An alternative could be for the target description to always describe
> the
> >>> largest possible vector file, or explicitly describe the VLE registers
> as variable
> >>> length, and then gdb would handle presenting the usable registers.
> GDB should
> >>> be able to tell the size of the vector registers by looking at the VQ
> (or was
> >>> it VL?  Or whatever it is called) register.
> >>
> >> The variable length description is technically more correct, but i
> think we already opted for a different solution with multiple VL-based
> target descriptions.
> >
> > We did?
> >
>
> Well, we already have a working solution with support for changing
> vector lengths on both gdb and gdbserver. What we are missing is a way
> to do that via RSP, which would enable us to support changing vector
> lengths in QEMU, for example.
>
> I'd rather avoid having to change some of gdb's type framework at this
> moment to support a sizeless type.
>
> >>
> >> My idea is to not rely on register values and, instead, focus on sizes
> of some aggregates the target description defines. That way we are not
> forced to fetch any registers and can infer the vector length from the
> sizes on the new target description.
> >>
> >> Both native sides (GDB and gdbserver) and QEMU know how to detect VL
> changes. It is just the communication of that change to GDB that we need to
> sort out via RSP.
> >
> > I don't know about QEMU, but the native side communicates the vector
> length with
> > the kernel exactly via a register..
> >
> >>
> >>>
> >>> In effect, we can see the current native support as a middle ground,
> >>> where aarch64_linux_nat_target::thread_architecture returns a different
> >>> gdbarch, there's no target description re-fetching, I believe.
> >>>
> >>
> >> There is no re-fetching in the sense that data doesn't get passed
> around, but new target descriptions do get created dynamically
> (aarch64_create_target_description) based on the new VL. The resulting
> gdbarch then gets cached so we don't need to recreate that particular
> variation.
> >
> > Which is still a lot more work than reporting "N" size, where N is a
> simple number.
> > Tdescs are de-dupped by hashing/comparing the whole tdesc xml text, for
> example.
> >
> > And how are you going to force a vector length change from GDB, if you
> go the
> > tdesc route?  With the register-like approach, you can just write to a
> register,
> > or something like it with a known value.  With the tdesc route, you
> can't, it's
> > read-only.  Writes _must_ be considered, for inferior function calls,
> > for example.  That's one of the features of the current ptrace interface
> > design, see:
>
> I agree inferior function calls would get a bit more complicated to
> support.
>
> If we use vg directly, then it will be simpler, though a little less
> generic.
>
> >
> >
> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/thread.html#478941
> >
> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/479843.html
> >
> >>
> >> My idea for a RSP-based target description update tries to mimic that
> as follows...
> >>
> >> - Remote end notices a target description / gdbarch change and notifies
> GDB via a stop reply packet entry.
> >>
> >> - GDB fetches the stop reply data and knows it has to query the remote
> about what particular threads had their target descriptions updated. I
> think this needs to be a new packet, maybe a qXfer one with a different
> object. The qXfer packet would handle large lists of threads (thinking
> about future use cases, GPU's etc).
> >>
> >> - Remote sends a list of threads to GDB.
> >>
> >> - GDB fetches the list of threads it needs to re-fetch the target
> descriptions from and proceeds to query the remote about those
> descriptions. I think we could cache the descriptions here, or have an
> opaque description that gets passed down to the target-specific code as you
> suggested.
> >>
> >> - GDB finishes the update and caches (as much as possible) the gdbarch
> per-thread/per-regcache.
> >>
> >> When no target description change has taken place we have nothing to do
> and no RSP overhead, so it wouldn't slow things down.
> >>
> >> Does the above sound like an acceptable way forward?
> > I'm afraid this doesn't consider the whole story.
> >
> > Thanks,
> > Pedro Alves
> >
>

Re: [RFC] Changing gdbarch mid-execution

[Luis Machado]

Hi Tim,

On 1/28/20 4:32 PM, Tim Newsome wrote:
> I'm coincidentally working on RISC-V vector support, where the same 
> problem exists. My preferred solution in that case is to export the 
> (potentially very wide) v0 -- v31 registers which hold the raw data and 
> have a fixed width. Then gdb can use the contents of vsew and vl to 
> present this raw data to the user, but broken up into the elements as 
> the hardware treats it currently.

That is one possible way, but not the way SVE is currently implemented.

In my opinion, having a fixed width that contains all the raw data 
simplifies the implementation, but introduces overhead for RSP when you 
have to keep passing unused vector data around.

It also has an overhead in allocated data structures to always hold all 
the raw vector data.

> 
> That feels much simpler than trying to change the architecture on the 
> fly, but I'm far from a gdb expert.

Agreed. But with code to generate the target descriptions on the fly, as 
we have right now for SVE, it is still reasonably simple to pick one 
based on a particular vector length register (vg register for SVE).

Having a more generic solution where targets are free to provide 
whatever target descriptions they need, say, at the thread level, is a 
bit more complicated. But it seems we're leaning towards a simpler 
solution that looks like a middle ground.

> Tim
> 
> On Tue, Jan 28, 2020 at 11:21 AM Luis Machado <luis.machado@linaro.org 
> <mailto:luis.machado@linaro.org>> wrote:
> 
>     On 1/23/20 1:06 PM, Pedro Alves wrote:
>      > On 1/22/20 5:03 PM, Luis Machado wrote:
>      >> On 1/22/20 11:56 AM, Pedro Alves wrote:
>      >>> On 1/6/20 2:08 PM, Luis Machado wrote:
>      >>>> Hi,
>      >>>>
>      >>>> I have a situation at hand and i'm thinking about how to best
>     solve it.
>      >>>>
>      >>>> AArch64 SVE has the capability of changing the vector length
>     mid-execution. This can be done at the thread level.
>      >>>>
>      >>>> Native GDB already supports this by looking at the ptrace
>     data. But doing this for a remote target requires changes to the RSP.
>      >>>>
>      >>>> Instead of changing things just for this particular case, i'm
>     considering having a more general mechanism for updating the
>     architecture data whenever such change is noticed by whoever is
>     controlling the inferior.
>      >>>>
>      >>>> My idea is to get the mechanism started by using the stop
>     reply to send a new notification, say, "arch-changed".
>      >>>>
>      >>>> That should trigger GDB to re-fetch the architecture data and
>     reinitialize it.
>      >>>>
>      >>>> In the particular case of SVE, we only need to fetch the
>     target description again, so we have the proper vector length and
>     data types set.
>      >>>>
>      >>>> Does this sound like a useful feature? Or should i go for the
>     solution with less impact that will only take care of re-fetching
>     the target description?
>      >>>
>      >>> I'm not keep on the idea of potential constant re-fetching of
>     arch data.
>      >>> I'd think that "arch-changed=ARCH" with each arch having its
>     own unique
>      >>> name (can be opaque to GDB) so that GDB can cache the arch
>     description,
>      >>> and avoid refetching it over and over would be better.
>      >>
>      >> I don't like the re-fetching either, so i'm trying to minimize that.
>      >>
>      >> Part of the problem is that the vector length (VL) is
>     per-thread, ...
>      >>
>      >>>
>      >>> Also, I don't think a state transition such a "arch changed" is
>     the best.
>      >>> I'd think making the stop reply say:
>      >>>
>      >>>    "stopped on code running arch foo"
>      >>>
>      >>> is better.
>      >>>
>      >>> See this:
>      >>>
>      >>> https://www.sourceware.org/gdb/papers/multi-arch/real-multi-arch/
>      >>>
>      >>> In which Cagney suggested something very similar:
>      >>>
>      >>>    T00;...;Architecture=<arch>;...
>      >>>        The T packet is used to report the reason the target
>     stopped to GDB. That packet includes information such as the
>     processor and that processors registers. The packet can be extended
>     to include the architecture of the processor that halted.
>      >>>
>      >>
>      >> ... so the above, even though it works nicely for reporting the
>     stop of a single thread, it won't carry information about potential
>     other threads that stopped along with the one the caused the stop
>     reply to be sent, right? We would need to fetch updates from the
>     other threads in case they changed their VL during execution.
>      >
>      > Yes, but we also need to fetch their registers anyway too.  It's just
>      > another piece of information that we need to retrieve.  Maybe we
>     could
>      > include the "current architecture" info in the 'g' packet's reply.
>      >
> 
>     By "current architecture" you mean a reference/id to a particular
>     target
>     description or, more explicitly, a vg value? Like "aarch64-sve-8" for a
>     SVE description with vg==8?
> 
>     If we go the route of using the vg value to create another target
>     description locally (not passed via RSP), then it sounds like we could
>     just use the expedited vg register or vg value passed in via the
>     register fetch/store packets, since that is a static-sized register
>     that
>     we can always parse?
> 
>     By using register fetches/stores we can keep both gdb/gdbserver updated.
> 
>     Whenever gdbserver sees a change in vg, gdb will be informed of that by
>     fetching registers. Similarly, whenever gdb writes to vg, gdbserver
>     will
>     know about it and will update its view of the target description.
> 
>     gdb and the remote stub would need to be kept in sync regarding their
>     views of the vector length.
> 
>     Would this be more acceptable?
> 
>      >>
>      >>>
>      >>> Though for the SVE case, I'm not sure a target description
>     change is the
>      >>> best model, where you end up with a different target
>     description description
>      >>> for each potential different vector length.
>      >>
>      >> Right. A new target description comes along with new sizes for
>     the particular types and aggregates it defines.
>      >>
>      >>>
>      >>> An alternative could be for the target description to always
>     describe the
>      >>> largest possible vector file, or explicitly describe the VLE
>     registers as variable
>      >>> length, and then gdb would handle presenting the usable
>     registers.  GDB should
>      >>> be able to tell the size of the vector registers by looking at
>     the VQ (or was
>      >>> it VL?  Or whatever it is called) register.
>      >>
>      >> The variable length description is technically more correct, but
>     i think we already opted for a different solution with multiple
>     VL-based target descriptions.
>      >
>      > We did?
>      >
> 
>     Well, we already have a working solution with support for changing
>     vector lengths on both gdb and gdbserver. What we are missing is a way
>     to do that via RSP, which would enable us to support changing vector
>     lengths in QEMU, for example.
> 
>     I'd rather avoid having to change some of gdb's type framework at this
>     moment to support a sizeless type.
> 
>      >>
>      >> My idea is to not rely on register values and, instead, focus on
>     sizes of some aggregates the target description defines. That way we
>     are not forced to fetch any registers and can infer the vector
>     length from the sizes on the new target description.
>      >>
>      >> Both native sides (GDB and gdbserver) and QEMU know how to
>     detect VL changes. It is just the communication of that change to
>     GDB that we need to sort out via RSP.
>      >
>      > I don't know about QEMU, but the native side communicates the
>     vector length with
>      > the kernel exactly via a register..
>      >
>      >>
>      >>>
>      >>> In effect, we can see the current native support as a middle
>     ground,
>      >>> where aarch64_linux_nat_target::thread_architecture returns a
>     different
>      >>> gdbarch, there's no target description re-fetching, I believe.
>      >>>
>      >>
>      >> There is no re-fetching in the sense that data doesn't get
>     passed around, but new target descriptions do get created
>     dynamically (aarch64_create_target_description) based on the new VL.
>     The resulting gdbarch then gets cached so we don't need to recreate
>     that particular variation.
>      >
>      > Which is still a lot more work than reporting "N" size, where N
>     is a simple number.
>      > Tdescs are de-dupped by hashing/comparing the whole tdesc xml
>     text, for example.
>      >
>      > And how are you going to force a vector length change from GDB,
>     if you go the
>      > tdesc route?  With the register-like approach, you can just write
>     to a register,
>      > or something like it with a known value.  With the tdesc route,
>     you can't, it's
>      > read-only.  Writes _must_ be considered, for inferior function calls,
>      > for example.  That's one of the features of the current ptrace
>     interface
>      > design, see:
> 
>     I agree inferior function calls would get a bit more complicated to
>     support.
> 
>     If we use vg directly, then it will be simpler, though a little less
>     generic.
> 
>      >
>      >
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/thread.html#478941
>      >
>     http://lists.infradead.org/pipermail/linux-arm-kernel/2017-January/479843.html
>      >
>      >>
>      >> My idea for a RSP-based target description update tries to mimic
>     that as follows...
>      >>
>      >> - Remote end notices a target description / gdbarch change and
>     notifies GDB via a stop reply packet entry.
>      >>
>      >> - GDB fetches the stop reply data and knows it has to query the
>     remote about what particular threads had their target descriptions
>     updated. I think this needs to be a new packet, maybe a qXfer one
>     with a different object. The qXfer packet would handle large lists
>     of threads (thinking about future use cases, GPU's etc).
>      >>
>      >> - Remote sends a list of threads to GDB.
>      >>
>      >> - GDB fetches the list of threads it needs to re-fetch the
>     target descriptions from and proceeds to query the remote about
>     those descriptions. I think we could cache the descriptions here, or
>     have an opaque description that gets passed down to the
>     target-specific code as you suggested.
>      >>
>      >> - GDB finishes the update and caches (as much as possible) the
>     gdbarch per-thread/per-regcache.
>      >>
>      >> When no target description change has taken place we have
>     nothing to do and no RSP overhead, so it wouldn't slow things down.
>      >>
>      >> Does the above sound like an acceptable way forward?
>      > I'm afraid this doesn't consider the whole story.
>      >
>      > Thanks,
>      > Pedro Alves
>      >
>