Re: [PATCH,RFC,V3 0/8] Definition and Implementation of CTF Frame format

[Indu Bhagat <indu.bhagat@oracle.com>]

On 6/14/22 12:06 AM, Indu Bhagat wrote:
> Hello,
> 
> This is version 3 of the previously sent RFC series.
> 
> Initial (V1) posting contains necessary introductions:
> https://sourceware.org/pipermail/binutils/2022-May/120731.html
> V2 posting with some improvements:
> https://sourceware.org/pipermail/binutils/2022-May/120899.html
> 

In addition to the above, here are few little notes which can hopefully 
help with the review and feedback on the CTF Frame format.

This is a quick high-level snapshot of what CTF Frame section looks like 
and how to try it out with this patch series.

$ cat a.c
static int cnt;
int foo (int a) { return a++; }
int bar (int b) { return foo (b * 5); }
int main() { cnt++; return bar (cnt); }

Pass --gctf-frame option to the assembler to emit a .ctf_frame section. 
  GAS uses the .cfi_* directives embedded by the compiler to generate 
the .ctf_frame section.

The support for gas option --gctf-frame and readelf option 
--ctf-frame=.ctf_frame is provided in this patch series.

$ gcc -c -Wa,--gctf-frame a.c

Lets dump .eh_frame just for sake of understanding the whole picture.

$ readelf --debug=frames-interp a.o
00000000 0000000000000014 00000000 CIE "zR" cf=1 df=-8 ra=16
    LOC           CFA      ra
0000000000000000 rsp+8    c-8

00000018 000000000000001c 0000001c FDE cie=00000000 
pc=0000000000000000..0000000000000012
    LOC           CFA      rbp   ra
0000000000000000 rsp+8    u     c-8
0000000000000001 rsp+16   c-16  c-8
0000000000000004 rbp+16   c-16  c-8
0000000000000011 rsp+8    c-16  c-8

00000038 000000000000001c 0000003c FDE cie=00000000 
pc=0000000000000012..0000000000000030
    LOC           CFA      rbp   ra
0000000000000012 rsp+8    u     c-8
0000000000000013 rsp+16   c-16  c-8
0000000000000016 rbp+16   c-16  c-8
000000000000002f rsp+8    c-16  c-8

00000058 000000000000001c 0000005c FDE cie=00000000 
pc=0000000000000030..0000000000000052
    LOC           CFA      rbp   ra
0000000000000030 rsp+8    u     c-8
0000000000000031 rsp+16   c-16  c-8
0000000000000034 rbp+16   c-16  c-8
0000000000000051 rsp+8    c-16  c-8

$readelf --ctf-frame=.ctf_frame a.o output is as follows (with 
hand-marked annotations with @@ for illustration only @@) :
...
     Version: CTF_FRAME_VERSION_1
     Flags: NONE
     Num FDEs: 3
     Num FREs: 12

   Function Index :

     func idx [0]: pc = 0x0, size = 18 bytes     @@ FDE 1 @@
     STARTPC           CFA       FP        RA
     0000000000000000  sp+8      u         u         @@ FRE 1 @@
     0000000000000001  sp+16     c-16      u         @@ FRE 2 @@
     0000000000000004  fp+16     c-16      u         @@ FRE 3 @@
     0000000000000011  sp+8      c-16      u         @@ FRE 4 @@

     func idx [1]: pc = 0x0, size = 30 bytes         @@ FDE 1 @@
     STARTPC           CFA       FP        RA
     0000000000000000  sp+8      u         u         @@ FRE 5 @@
     0000000000000001  sp+16     c-16      u         @@ FRE 6 @@
     0000000000000004  fp+16     c-16      u         @@ FRE 7 @@
     000000000000001d  sp+8      c-16      u         @@ FRE 8 @@

     func idx [2]: pc = 0x0, size = 34 bytes         @@ FDE 1 @@
     STARTPC           CFA       FP        RA
     0000000000000000  sp+8      u         u         @@ FRE 9 @@
     0000000000000001  sp+16     c-16      u         @@ FRE 10 @@
     0000000000000004  fp+16     c-16      u         @@ FRE 11 @@
     0000000000000021  sp+8      c-16      u         @@ FRE 12 @@

.ctf_frame section in an object is relocated at link-time as usual.

$ readelf --relocs a.o
...
Relocation section '.rela.ctf_frame' at offset 0x388 contains 3 entries:
   Offset          Info           Type           Sym. Value    Sym. Name 
+ Addend
00000000001b  000200000002 R_X86_64_PC32     0000000000000000 .text + 0
00000000002c  000200000002 R_X86_64_PC32     0000000000000000 .text + 12
00000000003d  000200000002 R_X86_64_PC32     0000000000000000 .text + 30

Notes:
1. .ctf_frame information is for tracking CFA, FP, RA only. Although not 
apparent in the dump of .eh_frame here, but .eh_frame tracks the 
applicable set of callee-saved registers as well per function.
2. FDE = Function Descriptor entry, one per function. FRE = Frame Row Entry.
3. An FRE is a self-sufficient entity that can be used to infer the 
unwind information for a group of contiguous PCs.
4. In the final linked artifact, the output .ctf_frame section has FDEs 
in sorted order of their start PC.  This helps in using binary search 
for looking up the FDE.  For each FDE, the FREs are sorted in order of 
the PC of instructions.

The CTF Frame header file will also be a helpful resource for 
understanding the format.

> The commit log of each patch in the current patch series specifies the changes
> from the version V2 of the respective patch.  Briefly, following are the main
> changes in the current version V3 as compared to V2:
> 
> 1. Testsuites for gas, ld, libctfframe.
> 2. Some code cleanup, improvements and bugfixes.
> 
> This is a work in progress. In the subsequent iterations of the series,  I plan
> to focus on the bloat caused by the CTF Frame unwind information for the plt*
> entries on x86_64 and enhance the testsuite for CTF Frame.
> 
> Important note: libctfframe/configure has NOT been included in the patch
> series, as it causes the size of some patches to go beyond the allowed limit.
> Please regenerate the libctfframe/configure after applying the series.
> 
> Please comment and provide feedback, it will help shape the format.  At this
> time, I really need some inputs on:
> 
> 1. What is a good place for an unwinder based on CTF Frame format ? Currently
> to facilitate discussion, it is presented in a library of its own:
> libctfbacktrace which, in turn, uses the libctfframe library for decoding the
> .ctf_frame section for unwinding.  We brainstormed a bit about the possible
> candidates being libbacktace, libgcc or libunwind ? Are there any
> recommendations ?
>   
> Thanks,
> 
> Indu Bhagat (6):
>    ctf-frame.h: Add CTF Frame format definition
>    gas: add new command line option --gctf-frame
>    gas: generate .ctf_frame
>    bfd: linker: merge .ctf_frame sections
>    readelf/objdump: support for CTF Frame section
>    gdb: sim: buildsystem changes to accommodate libctfframe
> 
> Weimin Pan (2):
>    libctfframe: add the CTF Frame library
>    unwinder: generate backtrace using CTF Frame format