Re: Remote query for structure layout

Re: Remote query for structure layout

[Tim Newsome]

On Sun, Mar 28, 2021 at 5:00 AM <gdb-request@sourceware.org> wrote:

> I would like to propose a new command for the remote protocol that can be
> used to
> query structure layouts.
> Essentially a `qSymbol` equivalent for the output of `ptype`.
>

I'm really interested in this. I've been working on OpenOCD FreeRTOS
support and as Thomas points out, it's a fragile mess.

There's another use case that this would be great for. FreeRTOS stores
registers on the stack when it switches to another thread. OpenOCD needs to
know what this register layout is, and it also depends on compile time
options. With this mechanism I think it would be possible for FreeRTOS to
define a struct that represents how the registers are laid out on the
stack. E.g.:
struct register_stacking_t {
    uint32_t x1;
    uint32_t x2;
    uint32_t x3;
    uint32_t x4;
...
    uint32_t pc;
};

Am I right that this will end up in the dwarf info even if the struct
itself isn't used anywhere in the FreeRTOS code? This would simplify so
much.

Tim

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-30 4:35 p.m., Tim Newsome wrote:
> On Sun, Mar 28, 2021 at 5:00 AM <gdb-request@sourceware.org> wrote:
> 
>> I would like to propose a new command for the remote protocol that can be
>> used to
>> query structure layouts.
>> Essentially a `qSymbol` equivalent for the output of `ptype`.
>>
> 
> I'm really interested in this. I've been working on OpenOCD FreeRTOS
> support and as Thomas points out, it's a fragile mess.
> 
> There's another use case that this would be great for. FreeRTOS stores
> registers on the stack when it switches to another thread. OpenOCD needs to
> know what this register layout is, and it also depends on compile time
> options. With this mechanism I think it would be possible for FreeRTOS to
> define a struct that represents how the registers are laid out on the
> stack. E.g.:
> struct register_stacking_t {
>     uint32_t x1;
>     uint32_t x2;
>     uint32_t x3;
>     uint32_t x4;
> ...
>     uint32_t pc;
> };
> 
> Am I right that this will end up in the dwarf info even if the struct
> itself isn't used anywhere in the FreeRTOS code? This would simplify so
> much.

It doesn't seem like it:

    $ cat test.cpp
    struct foo
    {
      int a;
    };

    int main() {

    }
    $ g++ test.cpp -g3 -O0
    $ llvm-dwarfdump -F -color a.out
    a.out:  file format elf64-x86-64

    .debug_info contents:
    0x00000000: Compile Unit: length = 0x00000053, format = DWARF32, version = 0x0004, abbr_offset = 0x0000, addr_size = 0x08 (next unit at 0x00000057)

    0x0000000b: DW_TAG_compile_unit
                  DW_AT_producer [DW_FORM_strp]     ("GNU C++14 10.2.0 -mtune=generic -march=x86-64 -g3 -O0")
                  DW_AT_language [DW_FORM_data1]    (DW_LANG_C_plus_plus)
                  DW_AT_name [DW_FORM_strp] ("test.cpp")
                  DW_AT_comp_dir [DW_FORM_strp]     ("/home/simark/build/lttng-ust/doc/examples/easy-ust")
                  DW_AT_low_pc [DW_FORM_addr]       (0x0000000000001119)
                  DW_AT_high_pc [DW_FORM_data8]     (0x000000000000000b)
                  DW_AT_stmt_list [DW_FORM_sec_offset]      (0x00000000)
                  DW_AT_GNU_macros [DW_FORM_sec_offset]     (0x00000000)

    0x00000031:   DW_TAG_base_type
                    DW_AT_byte_size [DW_FORM_data1] (0x04)
                    DW_AT_encoding [DW_FORM_data1]  (DW_ATE_signed)
                    DW_AT_name [DW_FORM_string]     ("int")

    0x00000038:   DW_TAG_subprogram
                    DW_AT_external [DW_FORM_flag_present]   (true)
                    DW_AT_name [DW_FORM_strp]       ("main")
                    DW_AT_decl_file [DW_FORM_data1] ("/home/simark/build/lttng-ust/doc/examples/easy-ust/test.cpp")
                    DW_AT_decl_line [DW_FORM_data1] (6)
                    DW_AT_decl_column [DW_FORM_data1]       (0x05)
                    DW_AT_type [DW_FORM_ref4]       (0x00000031 "int")
                    DW_AT_low_pc [DW_FORM_addr]     (0x0000000000001119)
                    DW_AT_high_pc [DW_FORM_data8]   (0x000000000000000b)
                    DW_AT_frame_base [DW_FORM_exprloc]      (DW_OP_call_frame_cfa)
                    DW_AT_GNU_all_call_sites [DW_FORM_flag_present] (true)

    0x00000056:   NULL

There might be some clever trick to force the compiler in emitting it
though, but I suppose it will always be a bit fragile, dependent on the
particular compiler.

Simon

Re: Remote query for structure layout

[David Blaikie]

On Tue, Mar 30, 2021 at 2:33 PM Simon Marchi via Gdb <gdb@sourceware.org>
wrote:

> On 2021-03-30 4:35 p.m., Tim Newsome wrote:
> > On Sun, Mar 28, 2021 at 5:00 AM <gdb-request@sourceware.org> wrote:
> >
> >> I would like to propose a new command for the remote protocol that can
> be
> >> used to
> >> query structure layouts.
> >> Essentially a `qSymbol` equivalent for the output of `ptype`.
> >>
> >
> > I'm really interested in this. I've been working on OpenOCD FreeRTOS
> > support and as Thomas points out, it's a fragile mess.
> >
> > There's another use case that this would be great for. FreeRTOS stores
> > registers on the stack when it switches to another thread. OpenOCD needs
> to
> > know what this register layout is, and it also depends on compile time
> > options. With this mechanism I think it would be possible for FreeRTOS to
> > define a struct that represents how the registers are laid out on the
> > stack. E.g.:
> > struct register_stacking_t {
> >     uint32_t x1;
> >     uint32_t x2;
> >     uint32_t x3;
> >     uint32_t x4;
> > ...
> >     uint32_t pc;
> > };
> >
> > Am I right that this will end up in the dwarf info even if the struct
> > itself isn't used anywhere in the FreeRTOS code? This would simplify so
> > much.
>
> It doesn't seem like it:
>
>     $ cat test.cpp
>     struct foo
>     {
>       int a;
>     };
>
>     int main() {
>
>     }
>     $ g++ test.cpp -g3 -O0
>     $ llvm-dwarfdump -F -color a.out
>     a.out:  file format elf64-x86-64
>
>     .debug_info contents:
>     0x00000000: Compile Unit: length = 0x00000053, format = DWARF32,
> version = 0x0004, abbr_offset = 0x0000, addr_size = 0x08 (next unit at
> 0x00000057)
>
>     0x0000000b: DW_TAG_compile_unit
>                   DW_AT_producer [DW_FORM_strp]     ("GNU C++14 10.2.0
> -mtune=generic -march=x86-64 -g3 -O0")
>                   DW_AT_language [DW_FORM_data1]    (DW_LANG_C_plus_plus)
>                   DW_AT_name [DW_FORM_strp] ("test.cpp")
>                   DW_AT_comp_dir [DW_FORM_strp]
>  ("/home/simark/build/lttng-ust/doc/examples/easy-ust")
>                   DW_AT_low_pc [DW_FORM_addr]       (0x0000000000001119)
>                   DW_AT_high_pc [DW_FORM_data8]     (0x000000000000000b)
>                   DW_AT_stmt_list [DW_FORM_sec_offset]      (0x00000000)
>                   DW_AT_GNU_macros [DW_FORM_sec_offset]     (0x00000000)
>
>     0x00000031:   DW_TAG_base_type
>                     DW_AT_byte_size [DW_FORM_data1] (0x04)
>                     DW_AT_encoding [DW_FORM_data1]  (DW_ATE_signed)
>                     DW_AT_name [DW_FORM_string]     ("int")
>
>     0x00000038:   DW_TAG_subprogram
>                     DW_AT_external [DW_FORM_flag_present]   (true)
>                     DW_AT_name [DW_FORM_strp]       ("main")
>                     DW_AT_decl_file [DW_FORM_data1]
> ("/home/simark/build/lttng-ust/doc/examples/easy-ust/test.cpp")
>                     DW_AT_decl_line [DW_FORM_data1] (6)
>                     DW_AT_decl_column [DW_FORM_data1]       (0x05)
>                     DW_AT_type [DW_FORM_ref4]       (0x00000031 "int")
>                     DW_AT_low_pc [DW_FORM_addr]     (0x0000000000001119)
>                     DW_AT_high_pc [DW_FORM_data8]   (0x000000000000000b)
>                     DW_AT_frame_base [DW_FORM_exprloc]
> (DW_OP_call_frame_cfa)
>                     DW_AT_GNU_all_call_sites [DW_FORM_flag_present] (true)
>
>     0x00000056:   NULL
>
> There might be some clever trick to force the compiler in emitting it
> though, but I suppose it will always be a bit fragile, dependent on the
> particular compiler.
>

Is the register layout in this struct part of the ABI? If so the debugger
could assume it/hardcode the knowledge.

If not, then it seems to me the right thing might be for the compiler to
have builtin support for emitting this struct under a reserved name into
every CU, perhaps? Not relying on source quirks to encourage it (as you
say, would leave you at the whims of the compiler implementation), but
actually make it explicitly part of compiler support for this architecture.

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-30 5:41 p.m., David Blaikie wrote:> 
> Is the register layout in this struct part of the ABI? If so the debugger could assume it/hardcode the knowledge.

I don't think this is part of any ABI, this is just how FreeRTOS stores
the registers of inactive tasks.  So it's like an implementation detail
of FreeRTOS, but one that the debugger needs to know in order to inspect
existing threads.

From what I undersand from Tim's message, the GDB stub inside OpenOCD
already hardcodes that knowledge, but the problem is that the layout can
change based on compilation options (which OpenOCD probably can't easily
detect), and the layout could change between FreeRTOS versions.

> If not, then it seems to me the right thing might be for the compiler to have builtin support for emitting this struct under a reserved name into every CU, perhaps? Not relying on source quirks to encourage it (as you say, would leave you at the whims of the compiler implementation), but actually make it explicitly part of compiler support for this architecture.

Maybe an attribute

struct the_struct {
} __attribute__((always_emit_debug));

?

Simon

Re: Remote query for structure layout

[David Blaikie]

On Tue, Mar 30, 2021 at 2:49 PM Simon Marchi <simon.marchi@polymtl.ca>
wrote:

> On 2021-03-30 5:41 p.m., David Blaikie wrote:>
> > Is the register layout in this struct part of the ABI? If so the
> debugger could assume it/hardcode the knowledge.
>
> I don't think this is part of any ABI, this is just how FreeRTOS stores
> the registers of inactive tasks.  So it's like an implementation detail
> of FreeRTOS, but one that the debugger needs to know in order to inspect
> existing threads.
>
> From what I undersand from Tim's message, the GDB stub inside OpenOCD
> already hardcodes that knowledge, but the problem is that the layout can
> change based on compilation options (which OpenOCD probably can't easily
> detect), and the layout could change between FreeRTOS versions.
>
> > If not, then it seems to me the right thing might be for the compiler to
> have builtin support for emitting this struct under a reserved name into
> every CU, perhaps? Not relying on source quirks to encourage it (as you
> say, would leave you at the whims of the compiler implementation), but
> actually make it explicitly part of compiler support for this architecture.
>
> Maybe an attribute
>
> struct the_struct {
> } __attribute__((always_emit_debug));
>
> ?
>

Perhaps - though I was thinking actually hardcoded in the compiler - since
it's the one that chooses/knows the layout, it's not (presumably) using any
code it reads from a header, etc, in user code to define the layout.

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-30 6:03 p.m., David Blaikie wrote:> 
> Perhaps - though I was thinking actually hardcoded in the compiler - since it's the one that chooses/knows the layout, it's not (presumably) using any code it reads from a header, etc, in user code to define the layout. 

Is it really the compiler that decides this layout?  My understanding is
that this is decided by the FreeRTOS code.

Having used FreeRTOS a few times, my understanding is that it's
basically a big library, the compiler doesn't really know about it.  For
all it knows, you are compiling a bare metal program that happens to
implement a task system.

Simon

Re: Remote query for structure layout

[David Blaikie]

On Tue, Mar 30, 2021 at 3:38 PM Simon Marchi <simon.marchi@polymtl.ca>
wrote:

> On 2021-03-30 6:03 p.m., David Blaikie wrote:>
> > Perhaps - though I was thinking actually hardcoded in the compiler -
> since it's the one that chooses/knows the layout, it's not (presumably)
> using any code it reads from a header, etc, in user code to define the
> layout.
>
> Is it really the compiler that decides this layout?  My understanding is
> that this is decided by the FreeRTOS code.
>
> Having used FreeRTOS a few times, my understanding is that it's
> basically a big library, the compiler doesn't really know about it.  For
> all it knows, you are compiling a bare metal program that happens to
> implement a task system.
>

Ah, sorry, my misunderstanding then - knew nothing about it. Sorry about
that.

If it's "just" some user-code, is there a variable of the desired type
being declared around the function call?

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-30 6:44 p.m., David Blaikie wrote:
> If it's "just" some user-code, is there a variable of the desired type being declared around the function call? 

AFAIK, this type wouldn't be used by the FreeRTOS code at all, so no.
Again, here's my understanding, hopefully it's close enough to the
reality.

When a trap occurs, FreeRTOS saves the current task's register values on
the task's stack, using some arch-specific assembly code.  For example,
for RISC-V:

    https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/534eba66ce4a5bda45d5edeeb81ac5a3cf6d0df8/portable/GCC/RISC-V/portASM.S#L121

The way these registers are pushed is an implementation detail of
FreeRTOS.  And we can imagine that it can vary depending on the
compile-time FreeRTOS configuration, which OpenOCD doesn't know about.

To get register values of scheduled out tasks, OpenOCD needs to
interpret these register values from the tasks' stacks.

So Tim's suggestion is: have FreeRTOS declare a structure that has the
exact layout as the saved registers on the stack:

struct freertos_saved_regs {
  int x1;
  int x5;
  int x6;
  ...
};

Consumers could read that structure's layout from the DWARF info, and
read the register values based on that.  That would be a lot more robust
than hard-coding in the consumers how FreeRTOS stores things.

However, since that struct would never actually be used by FreeRTOS'
code, the compiler won't emit it.  Hence the need to find a way to force
the compiler to include it in the DWARF.

Does that clarify the situation?

Simon

Re: Remote query for structure layout

[David Blaikie]

(let me know if I'm just being an unhelpful bystander here - I clearly
don't have a lot of context)

On Tue, Mar 30, 2021 at 5:16 PM Simon Marchi <simon.marchi@polymtl.ca> wrote:
>
> On 2021-03-30 6:44 p.m., David Blaikie wrote:
> > If it's "just" some user-code, is there a variable of the desired type being declared around the function call?
>
> AFAIK, this type wouldn't be used by the FreeRTOS code at all, so no.
> Again, here's my understanding, hopefully it's close enough to the
> reality.
>
> When a trap occurs, FreeRTOS saves the current task's register values on
> the task's stack, using some arch-specific assembly code.  For example,
> for RISC-V:
>
>     https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/534eba66ce4a5bda45d5edeeb81ac5a3cf6d0df8/portable/GCC/RISC-V/portASM.S#L121

Ah, OK. So this is part of signal handling - so it's not part of the
code that's compiled into the user's program, for instance... not even
in some system library linked in, necessarily, I guess?

> The way these registers are pushed is an implementation detail of
> FreeRTOS.  And we can imagine that it can vary depending on the
> compile-time FreeRTOS configuration,

I guess in the worst case it could be totally dynamic - it could pick
a different layout each time.

(guess a side question: How's this different from other systems? I
don't know how other/more common systems handle registers during
signals)

> which OpenOCD doesn't know about.
>
> To get register values of scheduled out tasks, OpenOCD needs to
> interpret these register values from the tasks' stacks.
>
> So Tim's suggestion is: have FreeRTOS declare a structure that has the
> exact layout as the saved registers on the stack:
>
> struct freertos_saved_regs {
>   int x1;
>   int x5;
>   int x6;
>   ...
> };
>
> Consumers could read that structure's layout from the DWARF info, and
> read the register values based on that.  That would be a lot more robust
> than hard-coding in the consumers how FreeRTOS stores things.

If practical experience has shown the hard-coding is not
stable/reliable (than FreeRTOS does change its strategy from time to
time - but it's always constant for any given build of the FreeRTOS) I
guess.

But I'm not sure where FreeRTOS would expose this structure to user
code - it's not like there's a system library header that user code
must include...

> However, since that struct would never actually be used by FreeRTOS'
> code, the compiler won't emit it.  Hence the need to find a way to force
> the compiler to include it in the DWARF.
>
> Does that clarify the situation?

Somewhat - any lack of understanding is just my ignorance in this
field/area in general, to be clear. (I'm also not a core gdb
developer, so I'm not the sort of person you have to convince of
anything - just a curious bystander trying to understand/maybe offer
some insightful suggestions (I predominantly work on LLVM's debug info
emission, so that's my background/connection))

- Dave

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-30 11:27 p.m., David Blaikie wrote:
> (let me know if I'm just being an unhelpful bystander here - I clearly
> don't have a lot of context)

I think it is interesting.  If people are not interested they can just
skip reading this thread :).

> On Tue, Mar 30, 2021 at 5:16 PM Simon Marchi <simon.marchi@polymtl.ca> wrote:
>>
>> On 2021-03-30 6:44 p.m., David Blaikie wrote:
>>> If it's "just" some user-code, is there a variable of the desired type being declared around the function call?
>>
>> AFAIK, this type wouldn't be used by the FreeRTOS code at all, so no.
>> Again, here's my understanding, hopefully it's close enough to the
>> reality.
>>
>> When a trap occurs, FreeRTOS saves the current task's register values on
>> the task's stack, using some arch-specific assembly code.  For example,
>> for RISC-V:
>>
>>     https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/534eba66ce4a5bda45d5edeeb81ac5a3cf6d0df8/portable/GCC/RISC-V/portASM.S#L121
> 
> Ah, OK. So this is part of signal handling - so it's not part of the
> code that's compiled into the user's program, for instance... not even
> in some system library linked in, necessarily, I guess?

It is part of "user code".  Wih "big" OSes (e.g. Linux), the OS is
compiled on its own and the user programs are compiled separately.  With
embedded RTOSes (at least those I worked with), it's just one big
program.  Your user code includes <FreeRTOS.h> and calls FreeRTOS
functions to spawn to tasks.

>> The way these registers are pushed is an implementation detail of
>> FreeRTOS.  And we can imagine that it can vary depending on the
>> compile-time FreeRTOS configuration,
> 
> I guess in the worst case it could be totally dynamic - it could pick
> a different layout each time.

Yes, if it wanted to be really annoying :).

> (guess a side question: How's this different from other systems? I
> don't know how other/more common systems handle registers during
> signals)

I think Linux does the same, pushing the registers (and some information
about the signal) on the thread's stack before calling a signal handler.
The layout is defined by the ucontext structure, exposed in the Linux
ABI:

    https://github.com/torvalds/linux/blob/master/arch/riscv/include/uapi/asm/ucontext.h

GDB hardcodes that layout here:

    https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/riscv-linux-tdep.c;h=ca97a60128ffe681766e7e1600d3f24048bf1f68;hb=HEAD#l103

Being exposed by the Linux kernel and part of this ABI, this structure
is not likely to change.  That makes it reasonable to hardcode it.

>> which OpenOCD doesn't know about.
>>
>> To get register values of scheduled out tasks, OpenOCD needs to
>> interpret these register values from the tasks' stacks.
>>
>> So Tim's suggestion is: have FreeRTOS declare a structure that has the
>> exact layout as the saved registers on the stack:
>>
>> struct freertos_saved_regs {
>>   int x1;
>>   int x5;
>>   int x6;
>>   ...
>> };
>>
>> Consumers could read that structure's layout from the DWARF info, and
>> read the register values based on that.  That would be a lot more robust
>> than hard-coding in the consumers how FreeRTOS stores things.
> 
> If practical experience has shown the hard-coding is not
> stable/reliable (than FreeRTOS does change its strategy from time to
> time - but it's always constant for any given build of the FreeRTOS) I
> guess.

My guess is that FreeRTOS does not change it that often, there's not
reason to.  But it can vary from build to build due to compile-time
configuration of FreeRTOS (i.e. include or exclude support for using
some particular register).  An external observer such as OpenOCD does
not have access to the compile-time configuration that was used to build
FreeRTOS, so it's difficult to know what the layout used by this
particular build is.

> But I'm not sure where FreeRTOS would expose this structure to user
> code - it's not like there's a system library header that user code
> must include...

There is, any code using FreeRTOS will include FreeRTOS.h.  You can see
FreeRTOS more like a library than an OS.  FreeRTOS is not involved at
all until the user program calls functions like xTaskCreate to create
some tasks and calls vTaskStartScheduler to hand over control to
FreeRTOS.

So if the structure was declared somewhere in the port's include file,
it would necessarily be seen by some CUs (those that invoke FreeRTOS
functions).

>> However, since that struct would never actually be used by FreeRTOS'
>> code, the compiler won't emit it.  Hence the need to find a way to force
>> the compiler to include it in the DWARF.
>>
>> Does that clarify the situation?
> 
> Somewhat - any lack of understanding is just my ignorance in this
> field/area in general, to be clear. (I'm also not a core gdb
> developer, so I'm not the sort of person you have to convince of
> anything - just a curious bystander trying to understand/maybe offer
> some insightful suggestions (I predominantly work on LLVM's debug info
> emission, so that's my background/connection))

Well, since it's related to debug info emission, it could land on your
plate at some point, it's good that you have some context.

Simon

Re: Remote query for structure layout

[Tim Newsome]

On Tue, Mar 30, 2021 at 8:27 PM David Blaikie <dblaikie@gmail.com> wrote:

> (guess a side question: How's this different from other systems? I
> don't know how other/more common systems handle registers during
> signals)


Other systems have this same problem.
https://sourceforge.net/p/openocd/code/ci/master/tree/src/rtos/FreeRTOS.c
around line 440 has some heuristics.
For ARM, it assumes there are FPU registers on the stack "if (cpacr &
0x00F00000)".
For Cortex-M, it checks whether there are FPU registers on the stack by
reading something else off the stack.

The problem is worse on RISC-V because the architecture is intended to be
extensible. E.g. for pulpino additional registers are saved on the stack,
defined in
https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/95433d02848af0d274648ae4cdd846f6eff76dde/portable/GCC/RISC-V/chip_specific_extensions/Pulpino_Vega_RV32M1RM/freertos_risc_v_chip_specific_extensions.h#L76

Tim

Re: Remote query for structure layout

[David Blaikie]

On Wed, Mar 31, 2021 at 6:00 AM Simon Marchi <simon.marchi@polymtl.ca> wrote:
>
> On 2021-03-30 11:27 p.m., David Blaikie wrote:
> > (let me know if I'm just being an unhelpful bystander here - I clearly
> > don't have a lot of context)
>
> I think it is interesting.  If people are not interested they can just
> skip reading this thread :).
>
> > On Tue, Mar 30, 2021 at 5:16 PM Simon Marchi <simon.marchi@polymtl.ca> wrote:
> >>
> >> On 2021-03-30 6:44 p.m., David Blaikie wrote:
> >>> If it's "just" some user-code, is there a variable of the desired type being declared around the function call?
> >>
> >> AFAIK, this type wouldn't be used by the FreeRTOS code at all, so no.
> >> Again, here's my understanding, hopefully it's close enough to the
> >> reality.
> >>
> >> When a trap occurs, FreeRTOS saves the current task's register values on
> >> the task's stack, using some arch-specific assembly code.  For example,
> >> for RISC-V:
> >>
> >>     https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/534eba66ce4a5bda45d5edeeb81ac5a3cf6d0df8/portable/GCC/RISC-V/portASM.S#L121
> >
> > Ah, OK. So this is part of signal handling - so it's not part of the
> > code that's compiled into the user's program, for instance... not even
> > in some system library linked in, necessarily, I guess?
>
> It is part of "user code".  Wih "big" OSes (e.g. Linux), the OS is
> compiled on its own and the user programs are compiled separately.  With
> embedded RTOSes (at least those I worked with), it's just one big
> program.  Your user code includes <FreeRTOS.h> and calls FreeRTOS
> functions to spawn to tasks.
>
> >> The way these registers are pushed is an implementation detail of
> >> FreeRTOS.  And we can imagine that it can vary depending on the
> >> compile-time FreeRTOS configuration,
> >
> > I guess in the worst case it could be totally dynamic - it could pick
> > a different layout each time.
>
> Yes, if it wanted to be really annoying :).
>
> > (guess a side question: How's this different from other systems? I
> > don't know how other/more common systems handle registers during
> > signals)
>
> I think Linux does the same, pushing the registers (and some information
> about the signal) on the thread's stack before calling a signal handler.
> The layout is defined by the ucontext structure, exposed in the Linux
> ABI:
>
>     https://github.com/torvalds/linux/blob/master/arch/riscv/include/uapi/asm/ucontext.h
>
> GDB hardcodes that layout here:
>
>     https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/riscv-linux-tdep.c;h=ca97a60128ffe681766e7e1600d3f24048bf1f68;hb=HEAD#l103
>
> Being exposed by the Linux kernel and part of this ABI, this structure
> is not likely to change.  That makes it reasonable to hardcode it.

Ah, fair enough - good to understand and compare/contrast.

> >> which OpenOCD doesn't know about.
> >>
> >> To get register values of scheduled out tasks, OpenOCD needs to
> >> interpret these register values from the tasks' stacks.
> >>
> >> So Tim's suggestion is: have FreeRTOS declare a structure that has the
> >> exact layout as the saved registers on the stack:
> >>
> >> struct freertos_saved_regs {
> >>   int x1;
> >>   int x5;
> >>   int x6;
> >>   ...
> >> };
> >>
> >> Consumers could read that structure's layout from the DWARF info, and
> >> read the register values based on that.  That would be a lot more robust
> >> than hard-coding in the consumers how FreeRTOS stores things.
> >
> > If practical experience has shown the hard-coding is not
> > stable/reliable (than FreeRTOS does change its strategy from time to
> > time - but it's always constant for any given build of the FreeRTOS) I
> > guess.
>
> My guess is that FreeRTOS does not change it that often, there's not
> reason to.  But it can vary from build to build due to compile-time
> configuration of FreeRTOS (i.e. include or exclude support for using
> some particular register).  An external observer such as OpenOCD does
> not have access to the compile-time configuration that was used to build
> FreeRTOS, so it's difficult to know what the layout used by this
> particular build is.

Fair enough.

> > But I'm not sure where FreeRTOS would expose this structure to user
> > code - it's not like there's a system library header that user code
> > must include...
>
> There is, any code using FreeRTOS will include FreeRTOS.h.

Oh, that helps a lot, then.

> You can see
> FreeRTOS more like a library than an OS.  FreeRTOS is not involved at
> all until the user program calls functions like xTaskCreate to create
> some tasks and calls vTaskStartScheduler to hand over control to
> FreeRTOS.

OK

> So if the structure was declared somewhere in the port's include file,
> it would necessarily be seen by some CUs (those that invoke FreeRTOS
> functions).

Great!

One possibly "cheap" way would be for FreeRTOS to use the structure in
C code to do the register stashing - rather than or in addition to/in
some kind of hybrid manner along with the assembly. If that's not
possible/would make the code harder to understand, then finding some
other way to pin the data structure into the DWARF would be needed.

I'd be inclined to avoid adding a new attribute or other feature
(though I wouldn't be entirely opposed to it - I can certainly think
of other places where it may be useful to have a "whenever the
compiler sees this type, ensure it makes it into the resulting DWARF
no matter what" attribute) if reasonably possible - instead finding
some lowest-common-denominator/highly reliable signal that compilers
use to emit type information.

The simplest such signal is a global variable of the type (not a
pointer to it, but of the type itself) - though probably has to be
annotated in some way that ensures the compiler won't optimize the
variable away even under LTO, etc. I'm not sure sure off-hand if I
have a great way to do that portably (non-portably I guess there's
various "exported" attributes that ensure the entity is visible even
beyond the shared library/executable scope, and thus can't be
optimized away - maybe such an attribute would be sufficiently
portable for FreeRTOS's needs (ie: no less portable than the rest of
the code already)).
If such a variable would be problematic (taking up space in the image,
etc) - maybe there's a way to ensure it through an extra nonce
parameter (if it's C++ code, a parameter with a default argument - so
the caller never has to think about it) to some core FreeRTOS function
that won't be optimized away/removed as unused/etc. Again, has a
runtime impact, though - maybe if it was zero-size (like a class
template specialization with no members) & with the type as a template
parameter /maybe/ that would suffice...

Happy to play around with mechanisms like that if it's of interest.

> >> However, since that struct would never actually be used by FreeRTOS'
> >> code, the compiler won't emit it.  Hence the need to find a way to force
> >> the compiler to include it in the DWARF.
> >>
> >> Does that clarify the situation?
> >
> > Somewhat - any lack of understanding is just my ignorance in this
> > field/area in general, to be clear. (I'm also not a core gdb
> > developer, so I'm not the sort of person you have to convince of
> > anything - just a curious bystander trying to understand/maybe offer
> > some insightful suggestions (I predominantly work on LLVM's debug info
> > emission, so that's my background/connection))
>
> Well, since it's related to debug info emission, it could land on your
> plate at some point, it's good that you have some context.

Re: Remote query for structure layout

[Simon Marchi]

On 2021-04-02 6:05 p.m., David Blaikie wrote:
> One possibly "cheap" way would be for FreeRTOS to use the structure in
> C code to do the register stashing - rather than or in addition to/in
> some kind of hybrid manner along with the assembly. If that's not
> possible/would make the code harder to understand, then finding some
> other way to pin the data structure into the DWARF would be needed.
> 
> I'd be inclined to avoid adding a new attribute or other feature
> (though I wouldn't be entirely opposed to it - I can certainly think
> of other places where it may be useful to have a "whenever the
> compiler sees this type, ensure it makes it into the resulting DWARF
> no matter what" attribute) if reasonably possible - instead finding
> some lowest-common-denominator/highly reliable signal that compilers
> use to emit type information.
> 
> The simplest such signal is a global variable of the type (not a
> pointer to it, but of the type itself) - though probably has to be
> annotated in some way that ensures the compiler won't optimize the
> variable away even under LTO, etc. I'm not sure sure off-hand if I
> have a great way to do that portably (non-portably I guess there's
> various "exported" attributes that ensure the entity is visible even
> beyond the shared library/executable scope, and thus can't be
> optimized away - maybe such an attribute would be sufficiently
> portable for FreeRTOS's needs (ie: no less portable than the rest of
> the code already)).
> If such a variable would be problematic (taking up space in the image,
> etc) - maybe there's a way to ensure it through an extra nonce
> parameter (if it's C++ code, a parameter with a default argument - so
> the caller never has to think about it) to some core FreeRTOS function
> that won't be optimized away/removed as unused/etc. Again, has a
> runtime impact, though - maybe if it was zero-size (like a class
> template specialization with no members) & with the type as a template
> parameter /maybe/ that would suffice...
> 
> Happy to play around with mechanisms like that if it's of interest.

Ok, good to know that the intent is now clear.  I have no real opinion
on how this is achieved, since I am neither a FreeRTOS, OpenOCD or compiler
developer :).

Tim, Thomas: as for the GDB implementation part, feel free to send a
patch on gdb-patches, even if it's just a prototype.

Simon

Re: Remote query for structure layout

[Philippe Waroquiers]

On Mon, 2021-03-29 at 16:10 -0400, Simon Marchi wrote:
> It's the other way around.  In the problem at hand, GDB has all the
> information and it's the target who queries GDB for symbol and types.

Thanks for the correction ...
A dwarf reader in the target is likely not a good/lightweight idea.

Philippe

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-29 4:02 p.m., Philippe Waroquiers wrote:
> On Mon, 2021-03-29 at 15:42 -0400, Simon Marchi via Gdb wrote:
>> So I would lean towards a home-grown format, but it's more work to
>> ensure it is extensible if we want to include more information in the
>> replies in the future.
> 
> Wouldn't it be possible to have the target sending back some (maybe limited
> and/or partial) 'standard debug info'  (such as dwarf) for these types ?
> 
> I have a very limited knowledge of dwarf, so no idea if it is easy
> to have 'small scale' debug info being returned by the target
> (e.g. by having the needed dwarf entries linked as binary data
> inside the target, or generated by the target).
> 
> This might allow to reuse the existing GDB dwarf parser and likely
> give a flexible enough description 'language'.

It's the other way around.  In the problem at hand, GDB has all the
information and it's the target who queries GDB for symbol and types.

Simon

Re: Remote query for structure layout

[Philippe Waroquiers]

On Mon, 2021-03-29 at 15:42 -0400, Simon Marchi via Gdb wrote:
> So I would lean towards a home-grown format, but it's more work to
> ensure it is extensible if we want to include more information in the
> replies in the future.

Wouldn't it be possible to have the target sending back some (maybe limited
and/or partial) 'standard debug info'  (such as dwarf) for these types ?

I have a very limited knowledge of dwarf, so no idea if it is easy
to have 'small scale' debug info being returned by the target
(e.g. by having the needed dwarf entries linked as binary data
inside the target, or generated by the target).

This might allow to reuse the existing GDB dwarf parser and likely
give a flexible enough description 'language'.


Philippe

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-29 1:33 p.m., Thomas Weißschuh wrote:
> Hi!
> 
> On Mo, 2021-03-29T12:05-0400, Simon Marchi wrote:
>> On 2021-03-28 7:06 a.m., Thomas Weißschuh wrote:> Hi everybody,
>>> I would like to propose a new command for the remote protocol that can be used to
>>> query structure layouts.
>>> Essentially a `qSymbol` equivalent for the output of `ptype`.
>>>
>> I think that would make sense.  We already help the stub look up some
>> things in the debugged process using qSymbol, so it would be a bit silly
>> to say "now you're on your own to interpret it!".
> 
> Good to know, I'll take a shot at it.
> 
>> If we go there, we could also help the stub make the link between the
>> symbol and its type, so that you don't have to hardcode that symbol
>> "foo" is of type "foo_t".  For example, if you could also pass a symbol
>> name (like pxCurrentTCB) to qType, you wouldn't have to hardcode its
>> type name in the openocd source code, it's one less thing that can go
>> wrong.  Or it could be another packet that does symbol name to type
>> name, which you then pass to qType.
> 
> Would it not make more sense to report the type of a symbol as part of qSymbol?
> Gated behind a qSupported flag "qSymbol:type+" or so?

If it can easily be made in a backwards-compatible way (considering all
permutations of old gdb / new gdb, old stub / new stub), sure.

> What do you think about the dataformat of the qType response?
> Should it be a homegrown textformat or XML which would be much easier to
> extend.

The problem with XML is that it would need to be parsed on the target
side.  We try to keep things simple on the target, because that may be
on a constrained device, you don't want to have to include an
XML-parsing library.  XML the other way (from stub to GDB) is OK,
because it's easy to produce some basic XML using printf, and GDB can
link with an XML parsing library.

So I would lean towards a home-grown format, but it's more work to
ensure it is extensible if we want to include more information in the
replies in the future.

I don't know all the remote packets by heart, maybe there's one that
already uses a scheme that could be re-used here.  Otherwise, XML can
certainly be used to start prototyping.  With the appropriate
abstractions in place, it should be relatively easy to swap the
encoding for another one.

Simon

Re: Remote query for structure layout

[Thomas Weißschuh]

Hi!

On Mo, 2021-03-29T12:05-0400, Simon Marchi wrote:
> On 2021-03-28 7:06 a.m., Thomas Weißschuh wrote:> Hi everybody,
> > I would like to propose a new command for the remote protocol that can be used to
> > query structure layouts.
> > Essentially a `qSymbol` equivalent for the output of `ptype`.
> > 
> I think that would make sense.  We already help the stub look up some
> things in the debugged process using qSymbol, so it would be a bit silly
> to say "now you're on your own to interpret it!".

Good to know, I'll take a shot at it.

> If we go there, we could also help the stub make the link between the
> symbol and its type, so that you don't have to hardcode that symbol
> "foo" is of type "foo_t".  For example, if you could also pass a symbol
> name (like pxCurrentTCB) to qType, you wouldn't have to hardcode its
> type name in the openocd source code, it's one less thing that can go
> wrong.  Or it could be another packet that does symbol name to type
> name, which you then pass to qType.

Would it not make more sense to report the type of a symbol as part of qSymbol?
Gated behind a qSupported flag "qSymbol:type+" or so?

What do you think about the dataformat of the qType response?
Should it be a homegrown textformat or XML which would be much easier to
extend.

Regards,
Thomas

Re: Remote query for structure layout

[Simon Marchi]

On 2021-03-28 7:06 a.m., Thomas Weißschuh wrote:> Hi everybody,
> 
> I would like to propose a new command for the remote protocol that can be used to
> query structure layouts.
> Essentially a `qSymbol` equivalent for the output of `ptype`.
> 
> Background:
> 
> The RTOS-support in OpenOCD has to read in-memory datastructures of the debug
> target to reconstruct the threading information for GDB.
> For example for FreeRTOS this is done by first looking up the scheduler
> datastructure via `qSymbol` and then doing hardcoded pointer arithmetic based on
> the retrieved symbol addresses.
> (https://repo.or.cz/openocd.git/blob/HEAD:/src/rtos/FreeRTOS.c#l60)
> 
> Unfortunately the specific offsets inside the structures can change based on
> compilation options of FreeRTOS.
> 
> If GDB could report the information it already has from the debug information
> (as shown in `ptype /o`) via its remote protocol then the logic in OpenOCD
> could be more robust.
> 
> Ideas:
> 
> While a simple mapping of (structure name, member name) -> (offset) mapping
> would be enough for my usecase it would probably be better to report more data.
> 
> * type of a typename (struct, union)
> * total size of the type
> * all members of the type including their own type and offsets
> 
> Example:
> 
> # request info for type "foo_t"
> IN  qType:666f6f5f74
> # "foo_t" is a struct of size 32 and members:
> # * bar_t bar at offset 0
> # * baz_t baz at offset 16
> OUT qType:666f6f5f74:struct:32:0;6261725f74;626172,16;62617a5f74;62617a
> 
> (Or some XML equivalent)
> 
> Is this something that would fit into gdb?
> 
> Regards,
> Thomas

Hi,

I think that would make sense.  We already help the stub look up some
things in the debugged process using qSymbol, so it would be a bit silly
to say "now you're on your own to interpret it!".

If we go there, we could also help the stub make the link between the
symbol and its type, so that you don't have to hardcode that symbol
"foo" is of type "foo_t".  For example, if you could also pass a symbol
name (like pxCurrentTCB) to qType, you wouldn't have to hardcode its
type name in the openocd source code, it's one less thing that can go
wrong.  Or it could be another packet that does symbol name to type
name, which you then pass to qType.

Simon

Remote query for structure layout

[Thomas Weißschuh]

Hi everybody,

I would like to propose a new command for the remote protocol that can be used to
query structure layouts.
Essentially a `qSymbol` equivalent for the output of `ptype`.

Background:

The RTOS-support in OpenOCD has to read in-memory datastructures of the debug
target to reconstruct the threading information for GDB.
For example for FreeRTOS this is done by first looking up the scheduler
datastructure via `qSymbol` and then doing hardcoded pointer arithmetic based on
the retrieved symbol addresses.
(https://repo.or.cz/openocd.git/blob/HEAD:/src/rtos/FreeRTOS.c#l60)

Unfortunately the specific offsets inside the structures can change based on
compilation options of FreeRTOS.

If GDB could report the information it already has from the debug information
(as shown in `ptype /o`) via its remote protocol then the logic in OpenOCD
could be more robust.

Ideas:

While a simple mapping of (structure name, member name) -> (offset) mapping
would be enough for my usecase it would probably be better to report more data.

* type of a typename (struct, union)
* total size of the type
* all members of the type including their own type and offsets

Example:

# request info for type "foo_t"
IN  qType:666f6f5f74
# "foo_t" is a struct of size 32 and members:
# * bar_t bar at offset 0
# * baz_t baz at offset 16
OUT qType:666f6f5f74:struct:32:0;6261725f74;626172,16;62617a5f74;62617a

(Or some XML equivalent)

Is this something that would fit into gdb?

Regards,
Thomas