public inbox for binutils@sourceware.org
 help / color / mirror / Atom feed
* [PATCH 00/19] libctf, and CTF support for objdump and readelf
@ 2019-04-30 22:57 Nick Alcock
  2019-04-30 22:57 ` [PATCH 04/19] libctf: low-level list manipulation and helper utilities Nick Alcock
                   ` (22 more replies)
  0 siblings, 23 replies; 52+ messages in thread
From: Nick Alcock @ 2019-04-30 22:57 UTC (permalink / raw)
  To: binutils

This submission is the first part of multiple patch series which together add
support for the Compact ANSI-C Type Format to the GNU toolchain.

Compact C Type Format (CTF) is a reduced form of debugging information whose
main purpose is to describe the type of C entities such as structures, unions,
typedefs and function arguments.  CTF format is optimized for compactness: it
was originally designed for use-cases like dynamic tracing and online
in-application debugging, where debugging information is meant to be present
even in stripped binaries.

CTF gains compactness over DWARF in four ways:

  - a more compact encoding, at the cost of irregularity.  Rather than a regular
    scheme of tags and attributes within those tags, the structures are
    customized for each kind of C type described, which allows significant space
    savings.  IDs are omitted and implied wherever possible to save even more
    space (the ID of each type in the type table is implied by its position,
    trading space for the need to scan the table at load time).

  - reuse of strings from the ELF file: CTF files have one string table built
    into the CTF format itself and one "external" table which is usually the
    table in the ELF file.  Further improvements are possible here without
    format changes: we are looking into this.

  - a very compact association between the ELF symbol table and CTF.  No symbol
    table indexes are recorded: all are implied: the data-object section
    is as compact as possible, containing nothing but a stream of type IDs
    describing the type of data symbols in symbol table order.

  - aggressive link-time deduplication will be added in the next patch
    series and will be the default behavior, resulting in further space
    savings.

Types in CTF can be looked up by traversal from other types via a numeric type
ID, by traversal of all types in the file, by ELF symbol table ID or by name
(though not all types need be named). As in C, there are separate namesapces for
top-level types, enums, structs and unions.  There is no analogue of block
scope: types within functions must either be promoted to the top level, stored
in another CTF container (perhaps using CTF's single-level parent/child
relationship), or not represented at all. Since the principal use case of CTF is
to look up types given symbol table entries, and symbol tables are also a single
flat namespace, this is not expected to be a serious limitation.

All types CTF describes have a fixed size at CTF generation time, and there is
nothing like the DWARF exprloc interpreter to compute the size of variably-sized
entities. This is due to the adopted 'top-level model' and, consequently, VLAs
are not supported.

For an overview of the CTF format, see the documentation in the email I'll post
as a followup to this patch series (we have yet to figure out where to put it).

Most of this patch series implements a library that reads and writes this
format, and a header file describing it.  The linker, debugger, and
objdump/objcopy are expected to use the library, while GCC will not.


This first submission consists of the core library in libctf/, and CTF support
in objdump and readelf: this leverages the debugging dump support in libctf, so
the objdump and readelf support is a few dozen lines each on top of that.


This patch series is an RFC. We are still implementing some additional features
in order to capture the full potential of the format.  Enhancements planned
include:

A new section implementing a compact version of DW_AT_call_site et al, allowing
efficient backtracing even when the debuginfo is missing, including recovery of
the value of parameters in at least 99.9% of the cases handled by DWARF 5 in
existing code.

Enhancements to the core data structures (particularly ctf_stype_t and
ctf_member_t) reducing space spent on unused type bits when type IDs are < 2^16.

Nick Alcock (19):
  include: new header ctf.h: file format description
  include: new header ctf-api.h
  libctf: lowest-level memory allocation and debug-dumping wrappers
  libctf: low-level list manipulation and helper utilities
  libctf: error handling
  libctf: hashing
  libctf: implementation definitions related to file creation
  libctf: creation functions
  libctf: opening
  libctf: ELF file opening
  libctf: core type lookup
  libctf: lookups by name and symbol
  libctf: type copying
  libctf: library version enforcement
  libctf: mmappable archives
  libctf: labels
  libctf: debug dumping
  libctf: build system
  binutils: CTF support for objdump and readelf

 Makefile.def                    |    5 +
 Makefile.in                     |  984 ++++-
 binutils/Makefile.am            |   10 +-
 binutils/Makefile.in            |   18 +-
 binutils/aclocal.m4             |   10 +-
 binutils/doc/Makefile.in        |    9 +-
 binutils/doc/binutils.texi      |   12 +
 binutils/doc/ctf.options.texi   |   19 +
 binutils/objdump.c              |  156 +-
 binutils/readelf.c              |  206 +
 configure                       |    2 +-
 configure.ac                    |    2 +-
 include/ctf-api.h               |  354 ++
 include/ctf.h                   |  427 ++
 libctf/Makefile.am              |   31 +
 libctf/Makefile.in              |  767 ++++
 {binutils => libctf}/aclocal.m4 |   99 +-
 libctf/config.h.in              |   98 +
 libctf/configure                | 7120 +++++++++++++++++++++++++++++++
 libctf/configure.ac             |   59 +
 libctf/ctf-archive.c            |  491 +++
 libctf/ctf-create.c             | 1937 +++++++++
 libctf/ctf-decl.c               |  195 +
 libctf/ctf-dump.c               |  595 +++
 libctf/ctf-error.c              |   93 +
 libctf/ctf-hash.c               |  277 ++
 libctf/ctf-impl.h               |  404 ++
 libctf/ctf-labels.c             |  138 +
 libctf/ctf-lib.c                |  506 +++
 libctf/ctf-lookup.c             |  427 ++
 libctf/ctf-open.c               | 1359 ++++++
 libctf/ctf-subr.c               |   74 +
 libctf/ctf-types.c              | 1019 +++++
 libctf/ctf-util.c               |  176 +
 34 files changed, 18008 insertions(+), 71 deletions(-)
 create mode 100644 binutils/doc/ctf.options.texi
 create mode 100644 include/ctf-api.h
 create mode 100644 include/ctf.h
 create mode 100644 libctf/Makefile.am
 create mode 100644 libctf/Makefile.in
 copy {binutils => libctf}/aclocal.m4 (95%)
 create mode 100644 libctf/config.h.in
 create mode 100755 libctf/configure
 create mode 100644 libctf/configure.ac
 create mode 100644 libctf/ctf-archive.c
 create mode 100644 libctf/ctf-create.c
 create mode 100644 libctf/ctf-decl.c
 create mode 100644 libctf/ctf-dump.c
 create mode 100644 libctf/ctf-error.c
 create mode 100644 libctf/ctf-hash.c
 create mode 100644 libctf/ctf-impl.h
 create mode 100644 libctf/ctf-labels.c
 create mode 100644 libctf/ctf-lib.c
 create mode 100644 libctf/ctf-lookup.c
 create mode 100644 libctf/ctf-open.c
 create mode 100644 libctf/ctf-subr.c
 create mode 100644 libctf/ctf-types.c
 create mode 100644 libctf/ctf-util.c

-- 
2.21.0.237.gd0cfaa883d

^ permalink raw reply	[flat|nested] 52+ messages in thread

end of thread, other threads:[~2019-05-24 20:09 UTC | newest]

Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-30 22:57 [PATCH 00/19] libctf, and CTF support for objdump and readelf Nick Alcock
2019-04-30 22:57 ` [PATCH 04/19] libctf: low-level list manipulation and helper utilities Nick Alcock
2019-05-02 16:04   ` Nick Clifton
2019-05-03 19:25     ` Nick Alcock
2019-04-30 22:57 ` [PATCH 11/19] libctf: core type lookup Nick Alcock
2019-04-30 22:57 ` [PATCH 03/19] libctf: lowest-level memory allocation and debug-dumping wrappers Nick Alcock
2019-05-02 15:29   ` Nick Clifton
2019-05-03 19:12     ` Nick Alcock
2019-04-30 22:57 ` [PATCH 12/19] libctf: lookups by name and symbol Nick Alcock
2019-04-30 22:57 ` [PATCH 14/19] libctf: library version enforcement Nick Alcock
2019-04-30 22:57 ` [PATCH 10/19] libctf: ELF file opening Nick Alcock
2019-04-30 22:57 ` [PATCH 06/19] libctf: hashing Nick Alcock
2019-05-02 16:16   ` Nick Clifton
2019-05-03 19:33     ` Nick Alcock
2019-04-30 22:57 ` [PATCH 09/19] libctf: opening Nick Alcock
2019-04-30 22:57 ` [PATCH 13/19] libctf: type copying Nick Alcock
2019-04-30 22:57 ` [PATCH 05/19] libctf: error handling Nick Alcock
2019-05-02 16:10   ` Nick Clifton
2019-05-03 19:31     ` Nick Alcock
2019-04-30 22:57 ` [PATCH 01/19] include: new header ctf.h: file format description Nick Alcock
2019-05-01 16:57   ` Nick Clifton
2019-05-01 21:29     ` Jim Wilson
2019-05-03 11:15       ` Nick Alcock
2019-04-30 22:57 ` [PATCH 15/19] libctf: mmappable archives Nick Alcock
2019-04-30 22:57 ` [PATCH 02/19] include: new header ctf-api.h Nick Alcock
2019-05-02 15:07   ` Nick Clifton
2019-05-03 11:23     ` Nick Alcock
2019-04-30 22:57 ` [PATCH 08/19] libctf: creation functions Nick Alcock
2019-04-30 22:57 ` CTF format overview Nick Alcock
2019-04-30 22:57 ` [PATCH 19/19] binutils: CTF support for objdump and readelf Nick Alcock
2019-04-30 22:58 ` [PATCH 18/19] libctf: build system Nick Alcock
2019-05-01  0:13 ` [PATCH 17/19] libctf: debug dumping Nick Alcock
2019-05-01  0:19 ` [PATCH 16/19] libctf: labels Nick Alcock
2019-05-01  1:57 ` [PATCH 07/19] libctf: implementation definitions related to file creation Nick Alcock
2019-05-01 16:02 ` [PATCH 00/19] libctf, and CTF support for objdump and readelf Nick Clifton
2019-05-01 16:16   ` Jose E. Marchesi
2019-05-03 10:47     ` Nick Alcock
2019-05-02 15:22 ` Joseph Myers
2019-05-03 12:33   ` Nick Clifton
2019-05-06 16:40     ` Nick Alcock
2019-05-08 14:34     ` Michael Matz
2019-05-08 16:01       ` Nick Clifton
2019-05-08 16:20         ` Nick Alcock
2019-05-03 14:23   ` Nick Alcock
     [not found]     ` <alpine.DEB.2.21.1905072117440.19308@digraph.polyomino.org.uk>
2019-05-08 11:39       ` Nick Alcock
2019-05-24  8:57     ` Pedro Alves
2019-05-24 16:05       ` Nick Alcock
2019-05-24 16:19         ` Pedro Alves
2019-05-24 20:09           ` Nick Alcock
2019-05-03 16:19 ` Florian Weimer
2019-05-03 19:45   ` Nick Alcock
2019-05-06 12:07     ` Florian Weimer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).