Re: [PATCH,RFC,V4 0/8] Definition and Implementation of CTF Frame format

[Indu Bhagat <indu.bhagat@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 5471 bytes --]

On 6/27/22 4:47 PM, Indu Bhagat wrote:
> Hi,
> 
> This is version 4 of the previously sent RFC series.
> 
> Initial (V1) posting contains necessary introductions:
> https://sourceware.org/pipermail/binutils/2022-May/120731.html
> V2 posting with some improvements:
> https://sourceware.org/pipermail/binutils/2022-May/120899.html
> V3 posting with further improvements:
> https://sourceware.org/pipermail/binutils/2022-June/121245.html
> 
> The commit log of each patch in the current patch series specifies the changes
> from the version V3 of the respective patch.  Briefly, following are the main
> changes in the current version V4 as compared to V3:
> 
> 1. CTF Frame format can now compactly represent unwind information for regular,
>    repetitive instruction patterns like the plt entries.  This has helped make
>    the size of .ctf_frame more attractive.  [As .ctf_frame encodes a subset of
>    information in .eh_frame, the sizes of .eh_frame will be justifiably higher.]
> 
>    ratio = (.ctf_frame size) / (.eh_frame size + .eh_frame_hdr size)
> 
>    Program   | ratio
>    ----------------
>    addr2line | .76
>    as        | .81
>    elfedit   | .76
>    gdb       | .78
>    gdbserver | .78
>    gprof     | .80
>    ld        | .81
>    nm        | .78
>    objdump   | .81
>    readelf   | .82
>    strings   | .76
>    ar        | .78
>    c++filt   | .75
>    gprofng   | .59
>    ld.bfd    | .81
>    objcopy   | .81
>    ranlib    | .78
>    size      | .76
>    strip     | .81
> 
>    In summary, earlier the CTF Frame unwind information for plt entires looked
>    like follows (in V3).
> 
>    $ readelf --ctf-frame=.ctf_frame testbinary
>    ...
>      func idx [0]: pc = 0x401020, size = 64 bytes
>      STARTPC           CFA       FP        RA
>      0000000000401020  sp+16     u         u
>      0000000000401026  sp+24     u         u
>      0000000000401030  sp+8      u         u
>      000000000040103b  sp+16     u         u
>      0000000000401040  sp+8      u         u
>      000000000040104b  sp+16     u         u
>      0000000000401050  sp+8      u         u
>      000000000040105b  sp+16     u         u
> 
>    Now with the V4 version of the patch series:
> 
>      func idx [0]: pc = 0x401020, size = 16 bytes
>      STARTPC         CFA       FP        RA
>      0000000000401020  sp+16     u         u
>      0000000000401026  sp+24     u         u
> 
>      func idx [1]: pc = 0x401030, size = 48 bytes
>      STARTPC[m]      CFA       FP        RA
>      0000000000000000  sp+8      u         u
>      000000000000000b  sp+16     u         u
> 
> 2. More testcases in libctfframe, further improvements and bugfixes.
> 
> This is a work in progress. In the subsequent iterations of the series,  I plan
> to focus on further testing CTF Frame unwind information for correctness and
> completeness.  We currently have a suite of unwinder tests which are used to
> validate the CTF Frame unwind information in various scenarios on x86_64 and
> aarch64 by unwinding through a variety of call stacks.  We will now work on
> increasing the coverage of those tests and incorporating the testsuite (perhaps
> with libctfbacktrace) in the next patch series.
> 
> That said, CTF Frame support V4 patch series has been regression tested on a
> host of targets with no regressions.
> 
> Important note: libctfframe/configure has NOT been included in the patch
> series, as it causes the size of some patches to go beyond the allowed limit.
> Please regenerate the libctfframe/configure after applying the series.
> 
> Please comment and provide feedback, it will help shape the format.  At this
> time, I really need some inputs on:
> 
> 1. What is a good place for an unwinder based on CTF Frame format ? Currently
> to facilitate discussion, it is presented in a library of its own:
> libctfbacktrace which, in turn, uses the libctfframe library for decoding the
> .ctf_frame section for unwinding.  We brainstormed a bit about the possible
> candidates being libbacktace, libgcc or libunwind ? Are there any
> recommendations ?
>   
> Thanks,
> 
> Indu Bhagat (6):
>    ctf-frame.h: Add CTF Frame format definition
>    gas: add new command line option --gctf-frame
>    gas: generate .ctf_frame
>    bfd: linker: merge .ctf_frame sections
>    readelf/objdump: support for CTF Frame section
>    gdb: sim: buildsystem changes to accommodate libctfframe
> 
> Weimin Pan (2):
>    libctfframe: add the CTF Frame library
>    unwinder: generate backtrace using CTF Frame format
> 

Hello,

Here are two Poke pickles for CTF Frame format (as defined in the V4 
posting of the patch series).

I am adding the two pickles to the libctfframe/ dir for now. I will send 
these to GNU poke for inclusion, meanwhile, sending them here in hope 
they help folks understand the ins and outs of the CTF Frame section:

libctfframe/ctf-frame.pk
libctfframe/ctf-frame-dump.pk

If you want to use these pickles in GNU poke, following is one way to 
use these pickles.

$ objcopy --dump-section .ctf_frame=output_ctf_frame binary
$ poke output_ctf_frame

In the GNU poke program:
(poke) .load pickles/ctf-frame-dump.pk
## for nicely formatted dump in tree mode
(poke) .set omode tree
(poke) var ctf_frame = CTF_Frame_section @ 0#B
(poke) ctf_frame
(poke) ctf_frame_dump_fres (ctf_frame)
(poke) ctf_frame_dump_fdes (ctf_frame)

Then you can use the functions in the ctf-frame-dump.pk pickle or 
inspect the ctf_frame as you please.

Thanks
Indu

[-- Attachment #2: 0001-libctfframe-add-GNU-poke-pickles-for-CTF-Frame.patch --]
[-- Type: text/x-patch, Size: 9735 bytes --]

From bcbe5bc132b2a012012abf02a11d206d91446d33 Mon Sep 17 00:00:00 2001
From: Indu Bhagat <indu.bhagat@oracle.com>
Date: Wed, 29 Jun 2022 11:55:51 -0700
Subject: [PATCH] libctfframe: add GNU poke pickles for CTF Frame

If you want to use these pickles in GNU poke, following is one way to
use the pickles.

$ objcopy --dump-section .ctf_frame=output_ctf_frame binary
$ poke output_ctf_frame

In the GNU poke program:
(poke) .load pickles/ctf-frame-dump.pk
(poke) .set omode tree
(poke) var ctf_frame = CTF_Frame_section @ 0#B

ChangeLog:

	* libctfframe/ctf-frame-dump.pk: New file.
	* libctfframe/ctf-frame.pk: Likewise.
---
 libctfframe/ctf-frame-dump.pk |  68 ++++++++++++
 libctfframe/ctf-frame.pk      | 200 ++++++++++++++++++++++++++++++++++
 2 files changed, 268 insertions(+)
 create mode 100644 libctfframe/ctf-frame-dump.pk
 create mode 100644 libctfframe/ctf-frame.pk

diff --git a/libctfframe/ctf-frame-dump.pk b/libctfframe/ctf-frame-dump.pk
new file mode 100644
index 00000000000..ad90d423e61
--- /dev/null
+++ b/libctfframe/ctf-frame-dump.pk
@@ -0,0 +1,68 @@
+/* ctf-frame-dump.pk - Utilities for dumping CTF Frame information.  */
+
+/* Copyright (C) 2022 Free Software Foundation.  */
+
+/* This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* This file contains routines for dumping out CTF Frame information.  */
+
+load "ctf-frame.pk";
+
+/* Dump each FDE in the CTF Frame section.  */
+
+fun ctf_frame_dump_fdes = (CTF_Frame_section ctf_frame_arg) void:
+{
+  var i = 0;
+  for (index in ctf_frame_arg.funcidx)
+    printf ("%u32d:   %v \n", i++, index);
+}
+
+/* Dump CTF Frame FREs.  */
+
+fun ctf_frame_dump_fres = (CTF_Frame_section ctf_frame_arg) void:
+{
+  var i = 0;
+
+  for (func in ctf_frame_arg.funcidx)
+    {
+      /* Get the FRE type.  */
+      var fre_type = func.func_info.fre_type;
+      var num_fres = func.func_num_fres;
+      var fre_off = ctf_frame_arg.header'size + ctf_frame_arg.header.cth_freoff;
+
+      if (fre_type == CTF_FRAME_ROW_ENTRY_TYPE_ADDR1)
+	{
+	  var fres = func.get_fres_addr1s (fre_off);
+	  /* print fres.  */
+	  for (fre in fres)
+	    printf ("%u32d:   %v \n", i++, fre);
+	}
+      else if (fre_type == CTF_FRAME_ROW_ENTRY_TYPE_ADDR2)
+	{
+	  var fres = func.get_fres_addr2s (fre_off);
+	  /* print fres.  */
+	  for (fre in fres)
+	    printf ("%u32d:   %v \n", i++, fre);
+	}
+      else if (fre_type == CTF_FRAME_ROW_ENTRY_TYPE_ADDR4)
+	{
+	  var fres = func.get_fres_addr4s (fre_off);
+	  /* print fres.  */
+	  for (fre in fres)
+	    printf ("%u32d:   %v \n", i++, fre);
+	}
+
+    }
+}
diff --git a/libctfframe/ctf-frame.pk b/libctfframe/ctf-frame.pk
new file mode 100644
index 00000000000..059c20c72cf
--- /dev/null
+++ b/libctfframe/ctf-frame.pk
@@ -0,0 +1,200 @@
+/* ctf-frame.pk - CTF Frame implementation for GNU poke.  */
+
+/* Copyright (C) 2022 Free Software Foundation.  */
+
+/* This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+load elf;
+
+/* Constants for CTF Preamble.  */
+var CTF_FRAME_VERSION_1 = 1;
+var CTF_FRAME_MAGIC = 0xdee2;
+/* Various flags for CTF Frame section.  */
+var CTF_FRAME_F_FDE_SORTED = 0x1;
+var CTF_FRAME_F_FRAME_POINTER = 0x2;
+
+type CTF_Frame_Preamble =
+  struct
+  {
+    uint<16> ctfp_magic : ((ctfp_magic == CTF_FRAME_MAGIC)
+			   || (ctfp_magic == 0xe2de && set_endian (!get_endian)))
+		          = CTF_FRAME_MAGIC;
+
+    byte ctfp_version = CTF_FRAME_VERSION_1;
+    byte ctfp_flags;
+  };
+
+var CTF_FRAME_ROW_ENTRY_TYPE_ADDR1 = 1;
+var CTF_FRAME_ROW_ENTRY_TYPE_ADDR2 = 2;
+var CTF_FRAME_ROW_ENTRY_TYPE_ADDR4 = 3;
+
+/* CTF Frame FDE types.
+   The CTF Frame format has two possible representations for functions.  The
+   choice of which type to use is made according to the instruction patterns
+   in the relevant program stub.  */
+
+/* Unwinders perform a (PC >= FRE_START_ADDR) to look up a matching FRE.  */
+var CTF_FRAME_FUNC_DESC_ENTRY_TYPE_PCINC = 0;
+/* Unwinders perform a (PC & FRE_START_ADDR_AS_MASK >= FRE_START_ADDR_AS_MASK)
+   to look up a matching FRE.  */
+var CTF_FRAME_FUNC_DESC_ENTRY_TYPE_PCMASK = 1;
+
+/* CTF Frame FDE Function Info.  */
+type CTF_Frame_Func_Info =
+  struct
+{
+  uint<3> unused;
+  uint<1> fde_type : fde_type in [CTF_FRAME_FUNC_DESC_ENTRY_TYPE_PCINC,
+				  CTF_FRAME_FUNC_DESC_ENTRY_TYPE_PCMASK];
+  uint<4> fre_type : fre_type in [CTF_FRAME_ROW_ENTRY_TYPE_ADDR1,
+				  CTF_FRAME_ROW_ENTRY_TYPE_ADDR2,
+				  CTF_FRAME_ROW_ENTRY_TYPE_ADDR4];
+};
+
+/* Supported ABIs/Arch.  */
+var CTF_FRAME_ABI_AARCH64_ENDIAN_BIG = 1; /* AARCH64 little endian.  */
+var CTF_FRAME_ABI_AARCH64_ENDIAN_LITTLE = 2; /* AARCH64 big endian.  */
+var CTF_FRAME_ABI_AMD64_ENDIAN_LITTLE = 3; /* AMD64 little endian.  */
+
+var CTF_FRAME_FRE_TYPE_BASIC = 0x0;
+
+type CTF_Frame_Header =
+  struct
+  {
+    CTF_Frame_Preamble cth_frame_preamble;
+    byte cth_frame_abi_arch : cth_frame_abi_arch in [CTF_FRAME_ABI_AARCH64_ENDIAN_BIG,
+						    CTF_FRAME_ABI_AARCH64_ENDIAN_LITTLE,
+					            CTF_FRAME_ABI_AMD64_ENDIAN_LITTLE];
+    int<8> cth_cfa_fixed_bp_offset;
+    int<8> cth_cfa_fixed_ra_offset;
+    uint<32> cth_num_fdes;
+    uint<32> cth_num_fres;
+    offset<uint32,B> cth_frelen;
+
+    offset<uint32,B> cth_fdeoff;
+    offset<uint32,B> cth_freoff;
+  };
+
+var CTF_FRAME_FRE_OFFSET_1B = 0;
+var CTF_FRAME_FRE_OFFSET_2B = 1;
+var CTF_FRAME_FRE_OFFSET_4B = 2;
+/* At this time, a max of 3 distinct offsets are supported.  */
+var CTF_FRAME_FRE_OFFSET_NUM = 3;
+
+/* A CTF FRE can be SP or BP based.  */
+var CTF_FRAME_BASE_REG_BP = 0;
+var CTF_FRAME_BASE_REG_SP = 1;
+
+type CTF_Frame_FRE_Info =
+  struct byte
+  {
+    uint<1> unused;
+    uint<2> offset_size : offset_size in [CTF_FRAME_FRE_OFFSET_1B,
+					  CTF_FRAME_FRE_OFFSET_2B,
+					  CTF_FRAME_FRE_OFFSET_4B];
+    uint<4> offset_num : offset_num <= CTF_FRAME_FRE_OFFSET_NUM;
+    uint<1> base_reg_id : base_reg_id in [CTF_FRAME_BASE_REG_BP,
+					  CTF_FRAME_BASE_REG_SP];
+  };
+
+/* CTF FRE.  */
+type CTF_FRE_addr1 =
+  struct
+  {
+    uint<8> fre_start_address;
+    CTF_Frame_FRE_Info fre_info;
+    union
+    {
+      int<8>[fre_info.offset_num] offsets_1B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_1B;
+      int<16>[fre_info.offset_num] offsets_2B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_2B;
+      int<32>[fre_info.offset_num] offsets_4B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_4B;
+    } offsets;
+  };
+
+type CTF_FRE_addr2 =
+  struct
+  {
+    uint<16> fre_start_address;
+    CTF_Frame_FRE_Info fre_info;
+    union
+    {
+      int<8>[fre_info.offset_num] offsets_1B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_1B;
+      int<16>[fre_info.offset_num] offsets_2B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_2B;
+      int<32>[fre_info.offset_num] offsets_4B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_4B;
+    } offsets;
+  };
+
+type CTF_FRE_addr4=
+  struct
+  {
+    uint<32> fre_start_address;
+    CTF_Frame_FRE_Info fre_info;
+    union
+    {
+      int<8>[fre_info.offset_num] offsets_1B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_1B;
+      int<16>[fre_info.offset_num] offsets_2B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_2B;
+      int<32>[fre_info.offset_num] offsets_4B
+         : fre_info.offset_size == CTF_FRAME_FRE_OFFSET_4B;
+    } offsets;
+  };
+
+
+type CTF_Frame_Func_Desc_Entry =
+  struct
+  {
+    uint<32> func_start_address;
+    uint<32> func_size;
+    offset<uint32,B> func_freoff;
+    uint<32> func_num_fres;
+    CTF_Frame_Func_Info func_info;
+
+    method get_fres_addr1s = (offset<uint32,B> hdr_freoff) CTF_FRE_addr1[]:
+      {
+	var off = hdr_freoff + func_freoff;
+	return CTF_FRE_addr1[func_num_fres] @ off;
+      }
+    method get_fres_addr2s = (offset<uint32,B> hdr_freoff) CTF_FRE_addr2[]:
+      {
+	var off = hdr_freoff + func_freoff;
+	return CTF_FRE_addr2[func_num_fres] @ off;
+      }
+    method get_fres_addr4s = (offset<uint32,B> hdr_freoff) CTF_FRE_addr4[]:
+      {
+	var off = hdr_freoff + func_freoff;
+	return CTF_FRE_addr4[func_num_fres] @ off;
+      }
+  };
+
+type CTF_Frame_section =
+  struct
+  {
+    CTF_Frame_Header header;
+
+    var func_index_off = OFFSET + header.cth_fdeoff;
+    var func_index_size = header.cth_freoff - header.cth_fdeoff;
+    var fre_off = OFFSET + header.cth_freoff;
+    var fre_size = header.cth_frelen;
+
+    CTF_Frame_Func_Desc_Entry[func_index_size] funcidx @ func_index_off;
+    byte[fre_size] fres_data_bytes @ fre_off;
+  };
-- 
2.31.1