Re: [PATCH, V2 3/3] dwarf2out: Emit BTF in dwarf2out_finish for BPF CO-RE usecase

[David Faust <david.faust@oracle.com>]

On 8/10/21 5:00 AM, Richard Biener via Gcc-patches wrote:
> On Thu, Aug 5, 2021 at 2:53 AM Indu Bhagat via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
>>
>> DWARF generation is split between early and late phases when LTO is in effect.
>> This poses challenges for CTF/BTF generation especially if late debug info
>> generation is desirable, as turns out to be the case for BPF CO-RE.
>>
>> In case of BPF CO-RE, the BPF backend adds information about CO-RE relocations
>> to the CTF container. This information is what needs to be emitted as a
>> separate .BTF.ext section when -more is in effect. Further, each CO-RE
>> relocation record holds an offset to a string specifying the access to the
>> structure's field. This means that .BTF string table needs to be modified
>> "late" in the compilation process. In other words, this implies that the BTF
>> sections cannot be finalized in dwarf2out_early_finish when -mcore for the BPF
>> backend is in effect.
> 
> OK, it didn't really get any clearer as to why the late annotation
> cannot be done
> after the early info was output.  Something to do with the BTF string table,
> but all structure field names must be already present there so I must be missing
> something.
> 
> ISTR the CO-RE info is fully contained in a new section .BTF.ext and the
> "early" .BTF section is not altered?

This is unfortunately not quite the case. The CO-RE relocation records 
themselves are placed in the .BTF.ext section. But a key component of 
each record is an "access string," a string which encodes the accessed 
field via a sequence of field or array indices.

The .BTF.ext section does not have a string table of its own, so these 
"access strings" are placed in the .BTF section string table. The CO-RE 
relocation records refer to them by offset into the .BTF string table.

As a result, the CO-RE information spills into the regular .BTF section,
and we need to delay writing out the .BTF section until after these 
strings can be added.

Personally I don't think this way of splitting the CO-RE information is 
ideal, but hopefully now the issue is more clear?

Thank you for your reviews!


Example of access string encoding:

struct S {
   int a;
   union {
     int _unused;
     int b;
     char c;
   } u[4];
};

struct S *foo = ...;
int x  = foo->a;          /* encoded as "0:0"     */
int y  = foo[4]->u[2].b   /* encoded as "4:1:2:1" */
char z = foo->u[3].c      /* encoded as "0:1:3:2" */


> 
>> Now, the emission of CTF/BTF debug info cannot be moved unconditionally to
>> dwarf2out_finish because dwarf2out_finish is not invoked at all for the LTO
>> compile phase for slim LTO objects, thus breaking CTF/BTF generation for other
>> targets when used with LTO.
>>
>> The approach taken here in this patch is that -
>>
>> 1. LTO is disabled for BPF CO-RE
>> The reason to disable LTO for BPF CO-RE is that if LTO is in effect, BPF CO-RE
>> relocations need to be generated in the LTO link phase _after_ the optimizations
>> are done. This means we need to devise way to combine early and late BTF. At
>> this time, in absence of linker support for BTF sections, it makes sense to
>> steer clear of LTO for BPF CO-RE and bypass the issue.
>>
>> 2. Use a target hook to allow BPF backend to cleanly convey the case when late
>> finalization of the CTF container is desirable.
>>
>> So, in other words,
>>
>> dwarf2out_early_finish
>>    - Always emit CTF here.
>>    - if (BTF && ctfc_debuginfo_early_finish_p), emit BTF now.
>>
>> dwarf2out_finish
>>    - if (BTF && !ctfc_debuginfo_early_finish_p && !in_lto_p) emit BTF now.
>>    - Use of in_lto_p to make sure LTO link phase does not affect BTF sections
>> for other targets.
>>
>> gcc/ChangeLog:
>>
>>          * dwarf2ctf.c (ctf_debug_finalize): Make it static.
>>          (ctf_debug_early_finish): New definition.
>>          (ctf_debug_finish): Likewise.
>>          * dwarf2ctf.h (ctf_debug_finalize): Remove declaration.
>>          (ctf_debug_early_finish): New declaration.
>>          (ctf_debug_finish): Likewise.
>>          * dwarf2out.c (dwarf2out_finish): Invoke ctf_debug_finish.
>>          (dwarf2out_early_finish): Invoke ctf_debug_early_finish.
>> ---
>>   gcc/dwarf2ctf.c | 55 +++++++++++++++++++++++++++++++++++++++++++------------
>>   gcc/dwarf2ctf.h |  4 +++-
>>   gcc/dwarf2out.c |  9 +++++++--
>>   3 files changed, 53 insertions(+), 15 deletions(-)
>>
>> diff --git a/gcc/dwarf2ctf.c b/gcc/dwarf2ctf.c
>> index 5e8a725..0fa429c 100644
>> --- a/gcc/dwarf2ctf.c
>> +++ b/gcc/dwarf2ctf.c
>> @@ -917,6 +917,27 @@ gen_ctf_type (ctf_container_ref ctfc, dw_die_ref die)
>>     return type_id;
>>   }
>>
>> +/* Prepare for output and write out the CTF debug information.  */
>> +
>> +static void
>> +ctf_debug_finalize (const char *filename, bool btf)
>> +{
>> +  if (btf)
>> +    {
>> +      btf_output (filename);
>> +      btf_finalize ();
>> +    }
>> +
>> +  else
>> +    {
>> +      /* Emit the collected CTF information.  */
>> +      ctf_output (filename);
>> +
>> +      /* Reset the CTF state.  */
>> +      ctf_finalize ();
>> +    }
>> +}
>> +
>>   bool
>>   ctf_do_die (dw_die_ref die)
>>   {
>> @@ -966,25 +987,35 @@ ctf_debug_init_postprocess (bool btf)
>>       btf_init_postprocess ();
>>   }
>>
>> -/* Prepare for output and write out the CTF debug information.  */
>> +/* Early finish CTF/BTF debug info.  */
>>
>>   void
>> -ctf_debug_finalize (const char *filename, bool btf)
>> +ctf_debug_early_finish (const char * filename)
>>   {
>> -  if (btf)
>> +  /* Emit CTF debug info early always.  */
>> +  if (ctf_debug_info_level > CTFINFO_LEVEL_NONE
>> +      /* Emit BTF debug info early if the target does not require late
>> +        emission.  */
>> +       || (btf_debuginfo_p ()
>> +          && targetm.ctfc_debuginfo_early_finish_p ()))
>>       {
>> -      btf_output (filename);
>> -      btf_finalize ();
>> +      /* Emit CTF/BTF debug info.  */
>> +      ctf_debug_finalize (filename, btf_debuginfo_p ());
>>       }
>> +}
>>
>> -  else
>> -    {
>> -      /* Emit the collected CTF information.  */
>> -      ctf_output (filename);
>> +/* Finish CTF/BTF debug info emission.  */
>>
>> -      /* Reset the CTF state.  */
>> -      ctf_finalize ();
>> -    }
>> +void
>> +ctf_debug_finish (const char * filename)
>> +{
>> +  /* Emit BTF debug info here when the target needs to update the CTF container
>> +     (ctfc) in the backend.  An example of this, at this time is the BPF CO-RE
>> +     usecase.  */
>> +  if (btf_debuginfo_p ()
>> +      && (!in_lto_p && !targetm.ctfc_debuginfo_early_finish_p ()))
>> +    /* Emit BTF debug info.  */
>> +    ctf_debug_finalize (filename, btf_debuginfo_p ());
>>   }
>>
>>   #include "gt-dwarf2ctf.h"
>> diff --git a/gcc/dwarf2ctf.h b/gcc/dwarf2ctf.h
>> index a3cf567..9edbde0 100644
>> --- a/gcc/dwarf2ctf.h
>> +++ b/gcc/dwarf2ctf.h
>> @@ -24,13 +24,15 @@ along with GCC; see the file COPYING3.  If not see
>>   #define GCC_DWARF2CTF_H 1
>>
>>   #include "dwarf2out.h"
>> +#include "flags.h"
>>
>>   /* Debug Format Interface.  Used in dwarf2out.c.  */
>>
>>   extern void ctf_debug_init (void);
>>   extern void ctf_debug_init_postprocess (bool);
>>   extern bool ctf_do_die (dw_die_ref);
>> -extern void ctf_debug_finalize (const char *, bool);
>> +extern void ctf_debug_early_finish (const char *);
>> +extern void ctf_debug_finish (const char *);
>>
>>   /* Wrappers for CTF/BTF to fetch information from GCC DWARF DIE.  Used in
>>      ctfc.c.
>> diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
>> index b91a9b5..708cd1f 100644
>> --- a/gcc/dwarf2out.c
>> +++ b/gcc/dwarf2out.c
>> @@ -31895,6 +31895,11 @@ dwarf2out_finish (const char *filename)
>>     unsigned char checksum[16];
>>     char dl_section_ref[MAX_ARTIFICIAL_LABEL_BYTES];
>>
>> +  /* Generate CTF/BTF debug info.  */
>> +  if ((ctf_debug_info_level > CTFINFO_LEVEL_NONE
>> +       || btf_debuginfo_p ()) && lang_GNU_C ())
>> +    ctf_debug_finish (filename);
>> +
>>     /* Skip emitting DWARF if not required.  */
>>     if (!dwarf_debuginfo_p ())
>>       return;
>> @@ -32799,8 +32804,8 @@ dwarf2out_early_finish (const char *filename)
>>          ctf_debug_do_cu (node->die);
>>         /* Post process the debug data in the CTF container if necessary.  */
>>         ctf_debug_init_postprocess (btf_debuginfo_p ());
>> -      /* Emit CTF/BTF debug info.  */
>> -      ctf_debug_finalize (filename, btf_debuginfo_p ());
>> +
>> +      ctf_debug_early_finish (filename);
>>       }
>>
>>     /* Do not generate DWARF assembler now when not producing LTO bytecode.  */
>> --
>> 1.8.3.1
>>