Re: [PATCH, RFC 0/7] Definition and Implementation of CTF Frame format

[Indu Bhagat <indu.bhagat@oracle.com>]

Hi Michael,

On 5/13/22 6:08 AM, Michael Matz wrote:
> Hello,
> 
> On Fri, 6 May 2022, Indu Bhagat via Binutils wrote:
> 
>> PT_GNU_CTF_FRAME) in ELF linked binaries. The CTF Frame format specifies the
>> minimal necessary unwind information, i.e., information needed to recover only
>> the CFA and the return address (RA) for all insns of a program.  This
>> information is a subset of what .eh_frame can convey: .eh_frame specifies how
>> to resurrect all callee-saved registers, if need be.
> 
> So, it's designed to be less complete than .eh_frame.  That in itself is
> not a problem, but ...
> 
>> ratio = (.ctf_frame / (.eh_frame+.eh_frame_hdr))
>>
>> ---------------------------------------------------
>> program    |  [x86_64] ratio  |  [aarch64] ratio
>> ---------------------------------------------------
>> addr2line  |       1.13       |    0.67
>> ar         |       1.03       |    0.70
>> as         |       1.00       |    0.73
>> c++filt    |       1.08       |    0.64
>> elfedit    |       1.03       |    0.66
>> gprof      |       1.06       |    0.71
>> ld         |       1.02       |    0.73
>> nm         |       1.07       |    0.69
>> objcopy    |       1.08       |    0.72
>> objdump    |       1.08       |    0.73
>> size       |       1.10       |    0.67
>> strings    |       1.09       |    0.67
> 
> ... this is.  You need more space on x86-64 to store severely less
> information, and even on aarch64 the savings are meager when compared to
> the information loss.
> 
> So, hmm, who would want to ever enable .ctf_frame instead of .eh_frame?
> 

Those applications which struggle with the following two complaints will 
want to consider .ctf_frame:
- eh_frame-based unwinders are complex and slow to manage with
- eh_frame-based unwinders are large as they need to deal with DWARF 
opcodes via a stack machine implementation

We know of two large applications of interest which have adopted ad-hoc 
solutions (based on post processing of binaries etc.) _and_ devised 
*their own* unwind formats to address the above issues.  The issue is 
that these ad-hoc solutions are themselves reaching their limits or are 
borderline unmaintainable because of the need to support more 
architectures over time, the need to regularly fix issues in different 
components involved as the generated code evolves.

Also see 
https://sourceware.org/pipermail/binutils/2021-December/118880.html.

So, the main premise of CTF Frame format is to address these issues, and 
more importantly make itself available via the toolchain because 
compiler/as/ld has the visibility needed to correctly generate unwind 
information. Post-processing binaries is not the right choice (IMO).

Highlighting two of the most important differences between .ctf_frame 
and .eh_frame in this context:

-- Simple unwinding with no stack machine.
    .eh_frame based unwinding needs a stack machine based implementation 
to work through the DWARF opcodes.
    .ctf_frame based unwinder is very simple to write and need no stack 
machine.

-- Self-sufficient FREs
    To achieve compactness, .eh_frame stores the DWARF opcodes in a 
sequence such that all the applicable instructions need to be executed 
until the desired PC is reached.
    .ctf_frame unwinding, on the other hand, needs to locate what is 
called the CTF Frame Row Entry (CTF FRE) in the .ctf_frame section, for 
the given PC.  Each CTF FRE is a _self-sufficient_ entry which contains 
information to unwind from the given PC.

This should make unwinding faster with .ctf_frame.

That said, the sizes of .ctf_frame sections on x86_64 will improve soon. 
  The current issue of having to encode a distinct CTF FRE for each PC 
in the .plt* section, is causing some bloat on x86_64. I expect some 
reduction (~10-15% ?), but we will see :)

Thanks
Indu

> 
> Ciao,
> Michael.
>