public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
From: Richard Biener <richard.guenther@gmail.com>
To: David Faust <david.faust@oracle.com>
Cc: gcc-patches@gcc.gnu.org, indu.bhagat@oracle.com,
	jose.marchesi@oracle.com, cupertino.miranda@oracle.com
Subject: Re: [PATCH v3 4/6] btf: add -fprune-btf option
Date: Fri, 31 May 2024 18:24:38 +0200	[thread overview]
Message-ID: <0567A10A-BF86-4275-9E70-F4B2793389BB@gmail.com> (raw)
In-Reply-To: <38fe6199-5bdc-41b9-9f06-0359aea003a4@oracle.com>



> Am 31.05.2024 um 17:58 schrieb David Faust <david.faust@oracle.com>:
> 
> 
> 
>> On 5/31/24 00:07, Richard Biener wrote:
>>> On Thu, May 30, 2024 at 11:34 PM David Faust <david.faust@oracle.com> wrote:
>>> 
>>> This patch adds a new option, -fprune-btf, to control BTF debug info
>>> generation.
>> 
>> Can you name it -gprune-btf instead?
> 
> Yes, sure.
> 
> I think I followed -feliminate-unused-debug-types,

Unfortunately GCC is full of historical mistakes…

> but I see the large
> majority of options controlling e.g. DWARF use -g instead.
> 
> Happy to change it to -gprune-btf.

Please.

Richard 

>> 
>>> As the name implies, this option enables a kind of "pruning" of the BTF
>>> information before it is emitted.  When enabled, rather than emitting
>>> all type information translated from DWARF, only information for types
>>> directly used in the source program is emitted.
>>> 
>>> The primary purpose of this pruning is to reduce the amount of
>>> unnecessary BTF information emitted, especially for BPF programs.  It is
>>> very common for BPF programs to incldue Linux kernel internal headers in
>>> order to have access to kernel data structures.  However, doing so often
>>> has the side effect of also adding type definitions for a large number
>>> of types which are not actually used by nor relevant to the program.
>>> In these cases, -fprune-btf commonly reduces the size of the resulting
>>> BTF information by 10x or more, as seen on average when compiling Linux
>>> kernel BPF selftests.  This both slims down the size of the resulting
>>> object and reduces the time required by the BPF loader to verify the
>>> program and its BTF information.
>>> 
>>> Note that the pruning implemented in this patch follows the same rules
>>> as the BTF pruning performed unconditionally by LLVM's BPF backend when
>>> generating BTF.  In particular, the main sources of pruning are:
>>> 
>>>  1) Only generate BTF for types used by variables and functions at
>>>     the file scope.  Note that with or without pruning, BTF_KIND_VAR
>>>     entries are only generated for variables present in the final
>>>     object - unused static variables or variables completely optimized
>>>     away must not have VAR entries in BTF.
>>> 
>>>  2) Avoid emitting full BTF for struct and union types which are only
>>>     pointed-to by members of other struct/union types.  In these cases,
>>>     the full BTF_KIND_STRUCT or BTF_KIND_UNION which would normally
>>>     be emitted is replaced with a BTF_KIND_FWD, as though the
>>>     underlying type was a forward-declared struct or union type.
>>> 
>>> gcc/
>>>        * btfout.cc (btf_used_types): New hash set.
>>>        (struct btf_fixup): New.
>>>        (fixups, forwards): New vecs.
>>>        (btf_output): Calculate num_types depending on flag_prune_btf.
>>>        (btf_early_finsih): New initialization for flag_prune_btf.
>>>        (btf_add_used_type): New function.
>>>        (btf_used_type_list_cb): Likewise.
>>>        (btf_late_collect_pruned_types): Likewise.
>>>        (btf_late_add_vars): Handle special case for variables in ".maps"
>>>        section when generating BTF for BPF CO-RE target.
>>>        (btf_late_finish): Use btf_late_collect_pruned_types when
>>>        flag_prune_btf in effect.  Move some initialization to btf_early_finish.
>>>        (btf_finalize): Additional deallocation for flag_prune_btf.
>>>        * common.opt (fprune-btf): New flag.
>>>        * ctfc.cc (init_ctf_strtable): Make non-static.
>>>        * ctfc.h (struct ctf_dtdef): Add visited_children_p boolean flag.
>>>        (init_ctf_strtable, ctfc_delete_strtab): Make extern.
>>>        * doc/invoke.texi (Debugging Options): Document -fprune-btf.
>>> 
>>> gcc/testsuite/
>>>        * gcc.dg/debug/btf/btf-prune-1.c: New test.
>>>        * gcc.dg/debug/btf/btf-prune-2.c: Likewise.
>>>        * gcc.dg/debug/btf/btf-prune-3.c: Likewise.
>>>        * gcc.dg/debug/btf/btf-prune-maps.c: Likewise.
>>> ---
>>> gcc/btfout.cc                                 | 359 +++++++++++++++++-
>>> gcc/common.opt                                |   4 +
>>> gcc/ctfc.cc                                   |   2 +-
>>> gcc/ctfc.h                                    |   3 +
>>> gcc/doc/invoke.texi                           |  20 +
>>> gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c  |  25 ++
>>> gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c  |  33 ++
>>> gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c  |  35 ++
>>> .../gcc.dg/debug/btf/btf-prune-maps.c         |  20 +
>>> 9 files changed, 494 insertions(+), 7 deletions(-)
>>> create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
>>> create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
>>> create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
>>> create mode 100644 gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c
>>> 
>>> diff --git a/gcc/btfout.cc b/gcc/btfout.cc
>>> index 32fda14f704b..a7da164f6b31 100644
>>> --- a/gcc/btfout.cc
>>> +++ b/gcc/btfout.cc
>>> @@ -828,7 +828,10 @@ output_btf_types (ctf_container_ref ctfc)
>>> {
>>>   size_t i;
>>>   size_t num_types;
>>> -  num_types = ctfc->ctfc_types->elements ();
>>> +  if (flag_prune_btf)
>>> +    num_types = max_translated_id;
>>> +  else
>>> +    num_types = ctfc->ctfc_types->elements ();
>>> 
>>>   if (num_types)
>>>     {
>>> @@ -957,6 +960,212 @@ btf_early_add_func_records (ctf_container_ref ctfc)
>>>     }
>>> }
>>> 
>>> +/* The set of types used directly in the source program, and any types manually
>>> +   marked as used.  This is the set of types which will be emitted when
>>> +   flag_prune_btf is set.  */
>>> +static GTY (()) hash_set<ctf_dtdef_ref> *btf_used_types;
>>> +
>>> +/* Fixup used to avoid unnecessary pointer chasing for types.  A fixup is
>>> +   created when a structure or union member is a pointer to another struct
>>> +   or union type.  In such cases, avoid emitting full type information for
>>> +   the pointee struct or union type (which may be quite large), unless that
>>> +   type is used directly elsewhere.  */
>>> +struct btf_fixup
>>> +{
>>> +  ctf_dtdef_ref pointer_dtd; /* Type node to which the fixup is applied.  */
>>> +  ctf_dtdef_ref pointee_dtd; /* Original type node referred to by pointer_dtd.
>>> +                               If this concrete type is not otherwise used,
>>> +                               then a forward is created.  */
>>> +};
>>> +
>>> +/* Stores fixups while processing types.  */
>>> +static vec<struct btf_fixup> fixups;
>>> +
>>> +/* For fixups where the underlying type is not used in the end, a BTF_KIND_FWD
>>> +   is created and emitted.  This vector stores them.  */
>>> +static GTY (()) vec<ctf_dtdef_ref, va_gc> *forwards;
>>> +
>>> +/* Recursively add type DTD and any types it references to the used set.
>>> +   Return a type that should be used for references to DTD - usually DTD itself,
>>> +   but may be NULL if DTD corresponds to a type which will not be emitted.
>>> +   CHECK_PTR is true if one of the predecessors in recursive calls is a struct
>>> +   or union member.  SEEN_PTR is true if CHECK_PTR is true AND one of the
>>> +   predecessors was a pointer type.  These two flags are used to avoid chasing
>>> +   pointers to struct/union only used from pointer members.  For such types, we
>>> +   will emit a forward instead of the full type information, unless
>>> +   CREATE_FIXUPS is false.  */
>>> +
>>> +static ctf_dtdef_ref
>>> +btf_add_used_type (ctf_container_ref ctfc, ctf_dtdef_ref dtd,
>>> +                  bool check_ptr, bool seen_ptr, bool create_fixups)
>>> +{
>>> +  if (dtd == NULL)
>>> +    return NULL;
>>> +
>>> +  uint32_t ctf_kind = CTF_V2_INFO_KIND (dtd->dtd_data.ctti_info);
>>> +  uint32_t kind = get_btf_kind (ctf_kind);
>>> +
>>> +  /* Check whether the type has already been added.  */
>>> +  if (btf_used_types->contains (dtd))
>>> +    {
>>> +      /* It's possible the type was already added as a fixup, but that we now
>>> +        have a concrete use of it.  */
>>> +      switch (kind)
>>> +       {
>>> +       case BTF_KIND_PTR:
>>> +       case BTF_KIND_TYPEDEF:
>>> +       case BTF_KIND_CONST:
>>> +       case BTF_KIND_VOLATILE:
>>> +       case BTF_KIND_RESTRICT:
>>> +         if (check_ptr)
>>> +           /* Type was previously added as a fixup, and that's OK.  */
>>> +           return dtd;
>>> +         else
>>> +           {
>>> +             /* The type was previously added as a fixup, but now we have
>>> +                a concrete use of it.  Remove the fixup.  */
>>> +             for (size_t i = 0; i < fixups.length (); i++)
>>> +               if (fixups[i].pointer_dtd == dtd)
>>> +                 fixups.unordered_remove (i);
>>> +
>>> +             /* Add the concrete base type.  */
>>> +             dtd->ref_type = btf_add_used_type (ctfc, dtd->ref_type, check_ptr,
>>> +                                                seen_ptr, create_fixups);
>>> +             return dtd;
>>> +           }
>>> +       default:
>>> +         return dtd;
>>> +       }
>>> +    }
>>> +
>>> +  if (ctf_kind == CTF_K_SLICE)
>>> +    {
>>> +      /* Bitfield.  Add the underlying type to the used set, but leave
>>> +        the reference to the bitfield.  The slice type won't be emitted,
>>> +        but we need the information in it when writing out the bitfield
>>> +        encoding.  */
>>> +      btf_add_used_type (ctfc, dtd->dtd_u.dtu_slice.cts_type,
>>> +                        check_ptr, seen_ptr, create_fixups);
>>> +      return dtd;
>>> +    }
>>> +
>>> +  /* Skip redundant definitions of void and types with no BTF encoding.  */
>>> +  if ((kind == BTF_KIND_INT && dtd->dtd_data.ctti_size == 0)
>>> +      || (kind == BTF_KIND_UNKN))
>>> +    return NULL;
>>> +
>>> +  /* Add the type itself, and assign its id.
>>> +     Do this before recursing to handle things like linked list structures.  */
>>> +  gcc_assert (ctfc->ctfc_nextid <= BTF_MAX_TYPE);
>>> +  dtd->dtd_type = ctfc->ctfc_nextid++;
>>> +  btf_used_types->add (dtd);
>>> +  ctf_add_string (ctfc, dtd->dtd_name, &(dtd->dtd_data.ctti_name), CTF_STRTAB);
>>> +  ctfc->ctfc_num_types++;
>>> +  ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (dtd);
>>> +
>>> +  /* Recursively add types referenced by this type.  */
>>> +  switch (kind)
>>> +    {
>>> +    case BTF_KIND_INT:
>>> +    case BTF_KIND_FLOAT:
>>> +    case BTF_KIND_FWD:
>>> +      /* Leaf kinds which do not refer to any other types.  */
>>> +      break;
>>> +
>>> +    case BTF_KIND_FUNC:
>>> +    case BTF_KIND_VAR:
>>> +      /* Root kinds; no type we are visiting may refer to these.  */
>>> +      gcc_unreachable ();
>>> +
>>> +    case BTF_KIND_PTR:
>>> +    case BTF_KIND_TYPEDEF:
>>> +    case BTF_KIND_CONST:
>>> +    case BTF_KIND_VOLATILE:
>>> +    case BTF_KIND_RESTRICT:
>>> +      {
>>> +       /* These type kinds refer to exactly one other type.  */
>>> +       if (check_ptr && !seen_ptr)
>>> +         seen_ptr = (kind == BTF_KIND_PTR);
>>> +
>>> +       /* Try to avoid chasing pointers to struct/union types if the
>>> +          underlying type isn't used.  */
>>> +       if (check_ptr && seen_ptr && create_fixups)
>>> +         {
>>> +           ctf_dtdef_ref ref = dtd->ref_type;
>>> +           uint32_t ref_kind = btf_dtd_kind (ref);
>>> +
>>> +           if ((ref_kind == BTF_KIND_STRUCT || ref_kind == BTF_KIND_UNION)
>>> +               && !btf_used_types->contains (ref))
>>> +             {
>>> +               struct btf_fixup fixup;
>>> +               fixup.pointer_dtd = dtd;
>>> +               fixup.pointee_dtd = ref;
>>> +               fixups.safe_push (fixup);
>>> +               break;
>>> +             }
>>> +         }
>>> +
>>> +       /* Add the type to which this type refers.  */
>>> +       dtd->ref_type = btf_add_used_type (ctfc, dtd->ref_type, check_ptr,
>>> +                                          seen_ptr, create_fixups);
>>> +       break;
>>> +      }
>>> +    case BTF_KIND_ARRAY:
>>> +      {
>>> +       /* Add element and index types.  */
>>> +       ctf_arinfo_t *arr = &(dtd->dtd_u.dtu_arr);
>>> +       arr->ctr_contents = btf_add_used_type (ctfc, arr->ctr_contents, false,
>>> +                                              false, create_fixups);
>>> +       arr->ctr_index = btf_add_used_type (ctfc, arr->ctr_index, false, false,
>>> +                                           create_fixups);
>>> +       break;
>>> +      }
>>> +    case BTF_KIND_STRUCT:
>>> +    case BTF_KIND_UNION:
>>> +    case BTF_KIND_ENUM:
>>> +    case BTF_KIND_ENUM64:
>>> +      {
>>> +       /* Add members.  */
>>> +       ctf_dmdef_t *dmd;
>>> +       for (dmd = dtd->dtd_u.dtu_members;
>>> +            dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd))
>>> +         {
>>> +           /* Add member type for struct/union members.  For enums, only the
>>> +              enumerator names are needed.  */
>>> +           if (kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION)
>>> +             dmd->dmd_type = btf_add_used_type (ctfc, dmd->dmd_type, true,
>>> +                                                false, create_fixups);
>>> +           ctf_add_string (ctfc, dmd->dmd_name, &(dmd->dmd_name_offset),
>>> +                           CTF_STRTAB);
>>> +         }
>>> +       break;
>>> +      }
>>> +    case BTF_KIND_FUNC_PROTO:
>>> +      {
>>> +       /* Add return type.  */
>>> +       dtd->ref_type = btf_add_used_type (ctfc, dtd->ref_type, false, false,
>>> +                                          create_fixups);
>>> +
>>> +       /* Add arg types.  */
>>> +       ctf_func_arg_t * farg;
>>> +       for (farg = dtd->dtd_u.dtu_argv;
>>> +            farg != NULL; farg = (ctf_func_arg_t *) ctf_farg_list_next (farg))
>>> +         {
>>> +           farg->farg_type = btf_add_used_type (ctfc, farg->farg_type, false,
>>> +                                                false, create_fixups);
>>> +           /* Note: argument names are stored in the auxilliary string table,
>>> +              since CTF does not include arg names.  That table has not been
>>> +              cleared, so no need to re-add argument names here.  */
>>> +         }
>>> +       break;
>>> +      }
>>> +    default:
>>> +      return NULL;
>>> +    }
>>> +
>>> +  return dtd;
>>> +}
>>> +
>>> /* Initial entry point of BTF generation, called at early_finish () after
>>>    CTF information has possibly been output.  Translate all CTF information
>>>    to BTF, and do any processing that must be done early, such as creating
>>> @@ -972,6 +1181,27 @@ btf_early_finish (void)
>>> 
>>>   btf_early_add_const_void (tu_ctfc);
>>>   btf_early_add_func_records (tu_ctfc);
>>> +
>>> +  /* Note: from here on, destructive changes are made to the TU CTFC to
>>> +     translate CTF to BTF.  These fields are reset to count BTF types etc.  */
>>> +  tu_ctfc->ctfc_num_types = 0;
>>> +  tu_ctfc->ctfc_num_vlen_bytes = 0;
>>> +  tu_ctfc->ctfc_vars_list_count = 0;
>>> +
>>> +  if (flag_prune_btf)
>>> +    {
>>> +      btf_used_types
>>> +       = hash_set<ctf_dtdef_ref>::create_ggc (tu_ctfc->ctfc_types->elements ());
>>> +      tu_ctfc->ctfc_nextid = 1;
>>> +      fixups.create (1);
>>> +
>>> +      /* Empty the string table, which was already populated with strings for
>>> +        all types translated from DWARF.  We may only need a very small subset
>>> +        of these strings; those will be re-added below.  */
>>> +      ctfc_delete_strtab (&tu_ctfc->ctfc_strtable);
>>> +      init_ctf_strtable (&tu_ctfc->ctfc_strtable);
>>> +      tu_ctfc->ctfc_strlen++;
>>> +    }
>>> }
>>> 
>>> /* Push a BTF datasec entry ENTRY into the datasec named SECNAME,
>>> @@ -1154,6 +1384,25 @@ btf_late_add_vars (ctf_container_ref ctfc)
>>> 
>>>       /* Add a BTF_KIND_DATASEC entry for the variable.  */
>>>       btf_datasec_add_var (ctfc, var, dvd);
>>> +
>>> +      const char *section = var->get_section ();
>>> +      if (section && (strcmp (section, ".maps") == 0) && flag_prune_btf)
>>> +       {
>>> +         /* The .maps section has special meaning in BTF: it is used for BPF
>>> +            map definitions.  These definitions should be structs.  We must
>>> +            collect pointee types used in map members as though they are used
>>> +            directly, effectively ignoring (from the pruning perspective) that
>>> +            they are struct members.  */
>>> +         ctf_dtdef_ref dtd = dvd->dvd_type;
>>> +         uint32_t kind = btf_dtd_kind (dvd->dvd_type);
>>> +         if (kind == BTF_KIND_STRUCT)
>>> +           {
>>> +             ctf_dmdef_t *dmd;
>>> +             for (dmd = dtd->dtd_u.dtu_members;
>>> +                  dmd != NULL; dmd = (ctf_dmdef_t *) ctf_dmd_list_next (dmd))
>>> +               btf_add_used_type (ctfc, dmd->dmd_type, false, false, true);
>>> +           }
>>> +       }
>>>     }
>>> }
>>> 
>>> @@ -1252,6 +1501,86 @@ btf_late_assign_datasec_ids (ctf_container_ref ctfc)
>>>     }
>>> }
>>> 
>>> +/* Callback used for assembling the only-used-types list.  Note that this is
>>> +   the same as btf_type_list_cb above, but the hash_set traverse requires a
>>> +   different function signature.  */
>>> +
>>> +static bool
>>> +btf_used_type_list_cb (const ctf_dtdef_ref& dtd, ctf_container_ref ctfc)
>>> +{
>>> +  ctfc->ctfc_types_list[dtd->dtd_type] = dtd;
>>> +  return true;
>>> +}
>>> +
>>> +/* Collect the set of types reachable from global variables and functions.
>>> +   This is the minimal set of types, used when generating pruned BTF.  */
>>> +
>>> +static void
>>> +btf_late_collect_pruned_types (ctf_container_ref ctfc)
>>> +{
>>> +  vec_alloc (forwards, 1);
>>> +
>>> +  /* Add types used from functions.  */
>>> +  ctf_dtdef_ref dtd;
>>> +  size_t i;
>>> +  FOR_EACH_VEC_ELT (*funcs, i, dtd)
>>> +    {
>>> +      btf_add_used_type (ctfc, dtd->ref_type, false, false, true);
>>> +      ctf_add_string (ctfc, dtd->dtd_name, &(dtd->dtd_data.ctti_name),
>>> +                     CTF_STRTAB);
>>> +    }
>>> +
>>> +  /* Add types used from global variables.  */
>>> +  for (i = 0; i < ctfc->ctfc_vars_list_count; i++)
>>> +    {
>>> +      ctf_dvdef_ref dvd = ctfc->ctfc_vars_list[i];
>>> +      btf_add_used_type (ctfc, dvd->dvd_type, false, false, true);
>>> +      ctf_add_string (ctfc, dvd->dvd_name, &(dvd->dvd_name_offset), CTF_STRTAB);
>>> +    }
>>> +
>>> +  /* Process fixups.  If the base type was never added, create a forward for it
>>> +     and adjust the reference to point to that.  If it was added, then nothing
>>> +     needs to change.  */
>>> +  for (i = 0; i < fixups.length (); i++)
>>> +    {
>>> +      struct btf_fixup *fx = &fixups[i];
>>> +      if (!btf_used_types->contains (fx->pointee_dtd))
>>> +       {
>>> +         /* The underlying type is not used.  Create a forward.  */
>>> +         ctf_dtdef_ref fwd = ggc_cleared_alloc<ctf_dtdef_t> ();
>>> +         ctf_id_t id = ctfc->ctfc_nextid++;
>>> +         gcc_assert (id <= BTF_MAX_TYPE);
>>> +
>>> +         bool union_p = (btf_dtd_kind (fx->pointee_dtd) == BTF_KIND_UNION);
>>> +
>>> +         fwd->dtd_name = fx->pointee_dtd->dtd_name;
>>> +         fwd->dtd_data.ctti_info = CTF_TYPE_INFO (CTF_K_FORWARD, union_p, 0);
>>> +         fwd->dtd_type = id;
>>> +         ctfc->ctfc_num_types++;
>>> +         ctfc->ctfc_num_vlen_bytes += btf_calc_num_vbytes (fwd);
>>> +         ctf_add_string (ctfc, fwd->dtd_name, &(fwd->dtd_data.ctti_name),
>>> +                         CTF_STRTAB);
>>> +
>>> +         /* Update the pointer to point to the forward.  */
>>> +         fx->pointer_dtd->ref_type = fwd;
>>> +         vec_safe_push (forwards, fwd);
>>> +       }
>>> +    }
>>> +
>>> +  /* Construct the resulting pruned type list.  */
>>> +  ctfc->ctfc_types_list
>>> +    = ggc_vec_alloc<ctf_dtdef_ref> (btf_used_types->elements () + 1
>>> +                                   + vec_safe_length (forwards));
>>> +
>>> +  btf_used_types->traverse<ctf_container_ref, btf_used_type_list_cb> (ctfc);
>>> +
>>> +  /* Insert the newly created forwards into the regular types list too.  */
>>> +  FOR_EACH_VEC_ELT (*forwards, i, dtd)
>>> +    ctfc->ctfc_types_list[dtd->dtd_type] = dtd;
>>> +
>>> +  max_translated_id = btf_used_types->elements () + vec_safe_length (forwards);
>>> +}
>>> +
>>> /* Late entry point for BTF generation, called from dwarf2out_finish ().
>>>    Complete and emit BTF information.  */
>>> 
>>> @@ -1263,13 +1592,22 @@ btf_finish (void)
>>> 
>>>   datasecs.create (0);
>>> 
>>> -  tu_ctfc->ctfc_num_types = 0;
>>> -  tu_ctfc->ctfc_num_vlen_bytes = 0;
>>> -  tu_ctfc->ctfc_vars_list_count = 0;
>>> -
>>>   btf_late_add_vars (tu_ctfc);
>>> -  btf_late_collect_translated_types (tu_ctfc);
>>> +  if (flag_prune_btf)
>>> +    {
>>> +      /* Collect pruned set of BTF types and prepare for emission.
>>> +        This includes only types directly used in file-scope variables and
>>> +        function return/argument types.  */
>>> +      btf_late_collect_pruned_types (tu_ctfc);
>>> +    }
>>> +  else
>>> +    {
>>> +      /* Collect all BTF types and prepare for emission.
>>> +        This includes all types translated from DWARF.  */
>>> +      btf_late_collect_translated_types (tu_ctfc);
>>> +    }
>>>   btf_late_add_func_datasec_entries (tu_ctfc);
>>> +
>>>   btf_late_assign_var_ids (tu_ctfc);
>>>   btf_late_assign_func_ids (tu_ctfc);
>>>   btf_late_assign_datasec_ids (tu_ctfc);
>>> @@ -1302,6 +1640,15 @@ btf_finalize (void)
>>>   func_map->empty ();
>>>   func_map = NULL;
>>> 
>>> +  if (flag_prune_btf)
>>> +    {
>>> +      btf_used_types->empty ();
>>> +      btf_used_types = NULL;
>>> +
>>> +      fixups.release ();
>>> +      forwards = NULL;
>>> +    }
>>> +
>>>   ctf_container_ref tu_ctfc = ctf_get_tu_ctfc ();
>>>   ctfc_delete_container (tu_ctfc);
>>>   tu_ctfc = NULL;
>>> diff --git a/gcc/common.opt b/gcc/common.opt
>>> index 2c078fdd1f86..3d4a55d02c26 100644
>>> --- a/gcc/common.opt
>>> +++ b/gcc/common.opt
>>> @@ -2588,6 +2588,10 @@ fpatchable-function-entry=
>>> Common Var(flag_patchable_function_entry) Joined Optimization
>>> Insert NOP instructions at each function entry.
>>> 
>>> +fprune-btf
>>> +Common Var(flag_prune_btf) Init(0)
>>> +Generate minimal BTF debug info.
>>> +
>>> frandom-seed
>>> Common Var(common_deferred_options) Defer
>>> 
>>> diff --git a/gcc/ctfc.cc b/gcc/ctfc.cc
>>> index 678ca2a072cf..27fc3a6d5851 100644
>>> --- a/gcc/ctfc.cc
>>> +++ b/gcc/ctfc.cc
>>> @@ -913,7 +913,7 @@ ctfc_get_dvd_srcloc (ctf_dvdef_ref dvd, ctf_srcloc_ref loc)
>>> /* Initialize the CTF string table.
>>>    The first entry in the CTF string table (empty string) is added.  */
>>> 
>>> -static void
>>> +void
>>> init_ctf_strtable (ctf_strtable_t * strtab)
>>> {
>>>   strtab->ctstab_head = NULL;
>>> diff --git a/gcc/ctfc.h b/gcc/ctfc.h
>>> index d0b724817a7f..29267dc036d1 100644
>>> --- a/gcc/ctfc.h
>>> +++ b/gcc/ctfc.h
>>> @@ -369,6 +369,9 @@ extern unsigned int ctfc_get_num_ctf_vars (ctf_container_ref);
>>> 
>>> extern ctf_strtable_t * ctfc_get_strtab (ctf_container_ref, int);
>>> 
>>> +extern void init_ctf_strtable (ctf_strtable_t *);
>>> +extern void ctfc_delete_strtab (ctf_strtable_t *);
>>> +
>>> /* Get the length of the specified string table in the CTF container.  */
>>> 
>>> extern size_t ctfc_get_strtab_len (ctf_container_ref, int);
>>> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
>>> index 517a782987d9..6dac00d14381 100644
>>> --- a/gcc/doc/invoke.texi
>>> +++ b/gcc/doc/invoke.texi
>>> @@ -534,6 +534,7 @@ Objective-C and Objective-C++ Dialects}.
>>> -gvms -gz@r{[}=@var{type}@r{]}
>>> -gsplit-dwarf  -gdescribe-dies  -gno-describe-dies
>>> -fdebug-prefix-map=@var{old}=@var{new}  -fdebug-types-section
>>> +-fprune-btf -fno-prune-btf
>>> -fno-eliminate-unused-debug-types
>>> -femit-struct-debug-baseonly  -femit-struct-debug-reduced
>>> -femit-struct-debug-detailed@r{[}=@var{spec-list}@r{]}
>>> @@ -12292,6 +12293,25 @@ compressed debug sections, the option is rejected.  Otherwise, if the
>>> assembler does not support them, @option{-gz} is silently ignored when
>>> producing object files.
>>> 
>>> +@opindex fprune-btf
>>> +@opindex fno-prune-btf
>>> +@item -fprune-btf
>>> +@itemx -fno-prune-btf
>>> +Prune BTF information before emission.  When pruning, only type
>>> +information for types used by global variables and file-scope functions
>>> +will be emitted.  If compiling for the BPF target with BPF CO-RE
>>> +enabled, type information will also be emitted for types used in BPF
>>> +CO-RE relocations.  In addition, struct and union types which are only
>>> +referred to via pointers from members of other struct or union types
>>> +shall be pruned and replaced with BTF_KIND_FWD, as though those types
>>> +were only present in the input as forward declarations.
>>> +
>>> +This option substantially reduces the size of produced BTF information,
>>> +but at significant loss in the amount of detailed type information.
>>> +It is primarily useful when compiling for the BPF target, to minimize
>>> +the size of the resulting object, and to eliminate BTF information
>>> +which is not immediately relevant to the BPF program loading process.
>>> +
>>> @opindex femit-struct-debug-baseonly
>>> @item -femit-struct-debug-baseonly
>>> Emit debug information for struct-like types
>>> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
>>> new file mode 100644
>>> index 000000000000..3c9b59a07ecf
>>> --- /dev/null
>>> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-1.c
>>> @@ -0,0 +1,25 @@
>>> +/* Simple test of -fprune-btf option operation.
>>> +   Since 'struct foo' is not used, no BTF shall be emitted for it.  */
>>> +
>>> +/* { dg-do compile } */
>>> +/* { dg-options "-gbtf -fprune-btf -dA" } */
>>> +
>>> +/* No BTF info for 'struct foo' nor types used only by it.  */
>>> +/* { dg-final { scan-assembler-not "BTF_KIND_STRUCT 'foo'" } } */
>>> +/* { dg-final { scan-assembler-not "BTF_KIND_INT 'char'" } } */
>>> +
>>> +/* We should get BTF info for 'struct bar' since it is used.  */
>>> +/* { dg-final { scan-assembler "BTF_KIND_STRUCT 'bar'"} } */
>>> +
>>> +struct foo {
>>> +  int a;
>>> +  char c;
>>> +};
>>> +
>>> +struct bar {
>>> +  int x;
>>> +  long z[4];
>>> +};
>>> +
>>> +struct bar a_bar;
>>> +
>>> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
>>> new file mode 100644
>>> index 000000000000..20183dffcc7f
>>> --- /dev/null
>>> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-2.c
>>> @@ -0,0 +1,33 @@
>>> +/* Test that -fprune-btf does not chase pointer-to-struct members.  */
>>> +
>>> +/* { dg-do compile } */
>>> +/* { dg-options "-gbtf -fprune-btf -dA" } */
>>> +
>>> +/* Only use of B is via a pointer member of C.
>>> +   Full BTF for B is replaced with a forward.  */
>>> +/* { dg-final { scan-assembler-not "BTF_KIND_STRUCT 'B'" } } */
>>> +/* { dg-final { scan-assembler-times "TYPE \[0-9\]+ BTF_KIND_FWD 'B'" 1 } } */
>>> +
>>> +/* Detailed info for B is omitted, and A is otherwise unused.  */
>>> +/* { dg-final { scan-assembler-not "BTF_KIND_\[A-Z\]+ 'A'" } } */
>>> +
>>> +/* { dg-final { scan-assembler "BTF_KIND_STRUCT 'C'" } } */
>>> +
>>> +struct A;
>>> +
>>> +struct B {
>>> +  int x;
>>> +  int (*do_A_thing) (int, int);
>>> +  struct A *other;
>>> +};
>>> +
>>> +struct C {
>>> +  unsigned int x;
>>> +  struct B * a;
>>> +};
>>> +
>>> +int
>>> +foo (struct C *c)
>>> +{
>>> +  return c->x;
>>> +}
>>> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
>>> new file mode 100644
>>> index 000000000000..57a079cf0b4d
>>> --- /dev/null
>>> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-3.c
>>> @@ -0,0 +1,35 @@
>>> +/* Test that -fprune-btf */
>>> +
>>> +/* { dg-do compile } */
>>> +/* { dg-options "-gbtf -fprune-btf -dA" } */
>>> +
>>> +/* We expect full BTF information each struct.  */
>>> +/* { dg-final { scan-assembler "TYPE \[0-9\]+ BTF_KIND_FWD 'file'" } } */
>>> +/* { dg-final { scan-assembler "TYPE \[0-9\]+ BTF_KIND_STRUCT 'A'" } } */
>>> +/* { dg-final { scan-assembler "TYPE \[0-9\]+ BTF_KIND_STRUCT 'B'" } } */
>>> +/* { dg-final { scan-assembler "TYPE \[0-9\]+ BTF_KIND_STRUCT 'C'" } } */
>>> +
>>> +struct file;
>>> +
>>> +struct A {
>>> +  void *private;
>>> +  long (*read)(struct file *, char *, unsigned long);
>>> +  long (*write)(struct file *, const char *, unsigned long);
>>> +};
>>> +
>>> +struct B {
>>> +  unsigned int x;
>>> +  struct A **as;
>>> +};
>>> +
>>> +struct C {
>>> +  struct A *arr_a[4];
>>> +  struct A *lone_a;
>>> +  unsigned int z;
>>> +};
>>> +
>>> +unsigned int
>>> +foo (struct B *b, struct C *c)
>>> +{
>>> +  return b->x + c->z;
>>> +}
>>> diff --git a/gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c
>>> new file mode 100644
>>> index 000000000000..bf3a25e984ff
>>> --- /dev/null
>>> +++ b/gcc/testsuite/gcc.dg/debug/btf/btf-prune-maps.c
>>> @@ -0,0 +1,20 @@
>>> +/* Test special meaning of .maps section for BTF when pruning.  For global
>>> +   variables of struct type placed in this section, we must treat members as
>>> +   though they are used directly, always collecting pointee types.
>>> +   Therefore, full type information for struct keep_me should be emitted.  */
>>> +
>>> +/* { dg-do compile } */
>>> +/* { dg-options "-gbtf -fprune-btf -dA" } */
>>> +
>>> +/* { dg-final { scan-assembler-not "BTF_KIND_FWD 'keep_me'" } } */
>>> +/* { dg-final { scan-assembler "BTF_KIND_STRUCT 'keep_me'" } } */
>>> +
>>> +struct keep_me {
>>> +  int a;
>>> +  char c;
>>> +};
>>> +
>>> +struct {
>>> +  int *key;
>>> +  struct keep_me *value;
>>> +} my_map __attribute__((section (".maps")));
>>> --
>>> 2.43.0
>>> 

  reply	other threads:[~2024-05-31 16:24 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-05-30 21:32 [PATCH v3 0/6] btf: refactor and add pruning option David Faust
2024-05-30 21:32 ` [PATCH v3 1/6] ctf, btf: restructure CTF/BTF emission David Faust
2024-06-05  7:27   ` Indu Bhagat
2024-05-30 21:32 ` [PATCH v3 2/6] ctf: use pointers instead of IDs internally David Faust
2024-06-05  7:31   ` Indu Bhagat
2024-05-30 21:32 ` [PATCH v3 3/6] btf: refactor and simplify implementation David Faust
2024-06-05  7:41   ` Indu Bhagat
2024-05-30 21:32 ` [PATCH v3 4/6] btf: add -fprune-btf option David Faust
2024-05-31  7:07   ` Richard Biener
2024-05-31 15:57     ` David Faust
2024-05-31 16:24       ` Richard Biener [this message]
2024-06-05  7:56   ` Indu Bhagat
2024-05-30 21:32 ` [PATCH v3 5/6] bpf,btf: enable BTF pruning by default for BPF David Faust
2024-06-05 20:49   ` Indu Bhagat
2024-05-30 21:32 ` [PATCH v3 6/6] opts: allow any combination of DWARF, CTF, BTF David Faust
2024-06-05 22:16   ` Indu Bhagat

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0567A10A-BF86-4275-9E70-F4B2793389BB@gmail.com \
    --to=richard.guenther@gmail.com \
    --cc=cupertino.miranda@oracle.com \
    --cc=david.faust@oracle.com \
    --cc=gcc-patches@gcc.gnu.org \
    --cc=indu.bhagat@oracle.com \
    --cc=jose.marchesi@oracle.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).