[PATCH v4 0/6] btf: refactor and add pruning option

[David Faust <david.faust@oracle.com>]

[v3: https://gcc.gnu.org/pipermail/gcc-patches/2024-May/653165.html
 Changes from v3:
 - Address typos, comment fixes and other minor nits pointed out by
   Indu in patches 1-3 and 5.
 - Rename option added in patch 4 from -fprune-btf to -gprune-btf.
 - Reword commit message in patch 4 to better describe behavior of
   -gprune-btf between LTO and non-LTO builds.
 - Improve the option parsing in patch 6 using Indu's suggested
   approach.  Add a few tests for -gdwarf, -gbtf, -gctf interaction. ]

 Review status:
 - Patches 1-3 have been approved, not yet pushed.
 - The CTF/BTF portion of patch 5 has been approved; the remaining
   portion needs BPF backend reviewer approval.
 - Patches 4 and 6 need review.  Both have portions which I think
   need global maintainer approval. ]

This patch series signficantly refactors the BTF generation in gcc,
making it simpler and easier to understand, extend and maintain.

It also introduces an optional algorithm to "prune" BTF information
before emission.  This pruning is meant to be used for BPF programs, to
alleviate the massive bloating of BTF information caused by including
Linux kernel internal headers.  The pruning is designed to be compatible
with the unconditional pruning performed  by the LLVM BPF backend when
generating BTF information.

While the changes are fairly significant, there is no actual change in
emitted BTF information (unless pruning is enabled), other than bug
fixes and small additions to the assembler debug comments.

Patch 1 restructures the emission of CTF and BTF information, with the
result that CTF is always completely generated and emitted before any
BTF-related procedures are run.  BTF emission is moved to late finish
for all targets, except when building with -flto.

Patch 2 changes the data structures shared by CTF and BTF to use
pointers rather than type IDs for all inter-type references.  This
change is completely transparent to both CTF and BTF.

Patch 3 heavily refactors btfout.cc to take advantage of the prior
changes and significantly simplify the BTF implementation.  The changes
are nearly transparent, however some small but important improvements
are also made possible by the refactor, such as fixing PR113566 for
non-LTO builds.

Patch 4 adds a new option to perform pruning of the BTF information
before emission.  This is intended to be used for BPF programs which
often include kernel headers, and in many cases reduces the size of
the resulting BTF information by a factor of 10.

Patch 5 makes BTF pruning work with BPF CO-RE, and enables the pruning
by default in the BPF backend.

Patch 6 takes advantage of the prior changes, and removes the
restriction on generating both CTF and BTF in the same compiler run,
allowing for any combinaion of -gdwarf, -gctf and -gbtf.

Tested on x86_64-linux-gnu, and on x86_64-linux-gnu host for
bpf-unknown-none target.

Also tested by compiling and runninng Linux kernel BPF selftests.
No known regressions.

David Faust (6):
  ctf, btf: restructure CTF/BTF emission
  ctf: use pointers instead of IDs internally
  btf: refactor and simplify implementation
  btf: add -gprune-btf option
  bpf,btf: enable BTF pruning by default for BPF
  opts: allow any combination of DWARF, CTF, BTF

 gcc/btfout.cc                                 | 1615 +++++++++--------
 gcc/common.opt                                |    4 +
 gcc/config/bpf/bpf.cc                         |    5 +
 gcc/config/bpf/btfext-out.cc                  |   14 +-
 gcc/config/bpf/core-builtins.cc               |   74 +-
 gcc/ctfc.cc                                   |  153 +-
 gcc/ctfc.h                                    |  113 +-
 gcc/ctfout.cc                                 |   22 +-
 gcc/doc/invoke.texi                           |   23 +
 gcc/dwarf2ctf.cc                              |  324 ++--
 gcc/dwarf2ctf.h                               |    2 +-
 gcc/dwarf2out.cc                              |    4 +-
 gcc/opts.cc                                   |   20 +-
 gcc/testsuite/gcc.dg/debug/btf/btf-3.c        |    8 +
 gcc/testsuite/gcc.dg/debug/btf/btf-4.c        |    8 +
 gcc/testsuite/gcc.dg/debug/btf/btf-5.c        |    9 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |   25 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |   33 +
 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |   35 +
 .../gcc.dg/debug/btf/btf-prune-maps.c         |   20 +
 .../gcc.dg/debug/btf/btf-variables-5.c        |    6 +-
 include/btf.h                                 |    5 +
 22 files changed, 1457 insertions(+), 1065 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-4.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-5.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
 create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c

-- 
2.43.0