Re: [PATCH 0/9] Add debug_annotate attributes

[Yonghong Song <yhs@meta.com>]

Hi, Jose and David,

Any progress on implement debug_annotate attribute in gcc?

Thanks,

Yonghong


On 6/15/22 3:56 PM, Yonghong Song wrote:
> 
> 
> On 6/15/22 1:57 PM, David Faust wrote:
>>
>>
>> On 6/14/22 22:53, Yonghong Song wrote:
>>>
>>>
>>> On 6/7/22 2:43 PM, David Faust wrote:
>>>> Hello,
>>>>
>>>> This patch series adds support for:
>>>>
>>>> - Two new C-language-level attributes that allow to associate (to 
>>>> "annotate" or
>>>>     to "tag") particular declarations and types with arbitrary 
>>>> strings. As
>>>>     explained below, this is intended to be used to, for example, 
>>>> characterize
>>>>     certain pointer types.
>>>>
>>>> - The conveyance of that information in the DWARF output in the form 
>>>> of a new
>>>>     DIE: DW_TAG_GNU_annotation.
>>>>
>>>> - The conveyance of that information in the BTF output in the form 
>>>> of two new
>>>>     kinds of BTF objects: BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG.
>>>>
>>>> All of these facilities are being added to the eBPF ecosystem, and 
>>>> support for
>>>> them exists in some form in LLVM.
>>>>
>>>> Purpose
>>>> =======
>>>>
>>>> 1)  Addition of C-family language constructs (attributes) to specify 
>>>> free-text
>>>>       tags on certain language elements, such as struct fields.
>>>>
>>>>       The purpose of these annotations is to provide additional 
>>>> information about
>>>>       types, variables, and function parameters of interest to the 
>>>> kernel. A
>>>>       driving use case is to tag pointer types within the linux 
>>>> kernel and eBPF
>>>>       programs with additional semantic information, such as 
>>>> '__user' or '__rcu'.
>>>>
>>>>       For example, consider the linux kernel function do_execve with 
>>>> the
>>>>       following declaration:
>>>>
>>>>         static int do_execve(struct filename *filename,
>>>>            const char __user *const __user *__argv,
>>>>            const char __user *const __user *__envp);
>>>>
>>>>       Here, __user could be defined with these annotations to record 
>>>> semantic
>>>>       information about the pointer parameters (e.g., they are 
>>>> user-provided) in
>>>>       DWARF and BTF information. Other kernel facilites such as the 
>>>> eBPF verifier
>>>>       can read the tags and make use of the information.
>>>>
>>>> 2)  Conveying the tags in the generated DWARF debug info.
>>>>
>>>>       The main motivation for emitting the tags in DWARF is that the 
>>>> Linux kernel
>>>>       generates its BTF information via pahole, using DWARF as a 
>>>> source:
>>>>
>>>>           +--------+  BTF                  BTF   +----------+
>>>>           | pahole |-------> vmlinux.btf ------->| verifier |
>>>>           +--------+                             +----------+
>>>>               ^                                        ^
>>>>               |                                        |
>>>>         DWARF |                                    BTF |
>>>>               |                                        |
>>>>            vmlinux                              +-------------+
>>>>            module1.ko                           | BPF program |
>>>>            module2.ko                           +-------------+
>>>>              ...
>>>>
>>>>       This is because:
>>>>
>>>>       a)  Unlike GCC, LLVM will only generate BTF for BPF programs.
>>>>
>>>>       b)  GCC can generate BTF for whatever target with -gbtf, but 
>>>> there is no
>>>>           support for linking/deduplicating BTF in the linker.
>>>>
>>>>       In the scenario above, the verifier needs access to the 
>>>> pointer tags of
>>>>       both the kernel types/declarations (conveyed in the DWARF and 
>>>> translated
>>>>       to BTF by pahole) and those of the BPF program (available 
>>>> directly in BTF).
>>>>
>>>>       Another motivation for having the tag information in DWARF, 
>>>> unrelated to
>>>>       BPF and BTF, is that the drgn project (another DWARF consumer) 
>>>> also wants
>>>>       to benefit from these tags in order to differentiate between 
>>>> different
>>>>       kinds of pointers in the kernel.
>>>>
>>>> 3)  Conveying the tags in the generated BTF debug info.
>>>>
>>>>       This is easy: the main purpose of having this info in BTF is 
>>>> for the
>>>>       compiled eBPF programs. The kernel verifier can then access 
>>>> the tags
>>>>       of pointers used by the eBPF programs.
>>>>
>>>>
>>>> For more information about these tags and the motivation behind 
>>>> them, please
>>>> refer to the following linux kernel discussions:
>>>>
>>>>     https://lore.kernel.org/bpf/20210914223004.244411-1-yhs@fb.com/
>>>>     https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com/
>>>>     https://lore.kernel.org/bpf/20211112012604.1504583-1-yhs@fb.com/
>>>>
>>>>
>>>> Implementation Overview
>>>> =======================
>>>>
>>>> To enable these annotations, two new C language attributes are added:
>>>> __attribute__((debug_annotate_decl("foo"))) and
>>>> __attribute__((debug_annotate_type("bar"))). Both attributes accept 
>>>> a single
>>>> arbitrary string constant argument, which will be recorded in the 
>>>> generated
>>>> DWARF and/or BTF debug information. They have no effect on code 
>>>> generation.
>>>>
>>>> Note that we are not using the same attribute names as LLVM 
>>>> (btf_decl_tag and
>>>> btf_type_tag, respectively). While these attributes are functionally 
>>>> very
>>>> similar, they have grown beyond purely BTF-specific uses, so 
>>>> inclusion of "btf"
>>>> in the attribute name seems misleading.
>>>>
>>>> DWARF support is enabled via a new DW_TAG_GNU_annotation. When 
>>>> generating DWARF,
>>>> declarations and types will be checked for the corresponding 
>>>> attributes. If
>>>> present, a DW_TAG_GNU_annotation DIE will be created as a child of 
>>>> the DIE for
>>>> the annotated type or declaration, one for each tag. These DIEs link 
>>>> the
>>>> arbitrary tag value to the item they annotate.
>>>>
>>>> For example, the following variable declaration:
>>>>
>>>>     #define __typetag1 __attribute__((debug_annotate_type 
>>>> ("typetag1")))
>>>>
>>>>     #define __decltag1 __attribute__((debug_annotate_decl 
>>>> ("decltag1")))
>>>>     #define __decltag2 __attribute__((debug_annotate_decl 
>>>> ("decltag2")))
>>>>
>>>>     int * __typetag1 x __decltag1 __decltag2;
>>>
>>> Based on the above example
>>>           static int do_execve(struct filename *filename,
>>>             const char __user *const __user *__argv,
>>>             const char __user *const __user *__envp);
>>>
>>> Should the above example should be the below?
>>>       int __typetag1 * x __decltag1 __decltag2
>>>
>>
>> This example is not related to the one above. It is just meant to
>> show the behavior of both attributes. My apologies for not making
>> that clear.
> 
> Okay, it should be fine if the dwarf debug_info is shown.
> 
>>
>>>>
>>>> Produces the following DWARF information:
>>>>
>>>>    <1><1e>: Abbrev Number: 3 (DW_TAG_variable)
>>>>       <1f>   DW_AT_name        : x
>>>>       <21>   DW_AT_decl_file   : 1
>>>>       <22>   DW_AT_decl_line   : 7
>>>>       <23>   DW_AT_decl_column : 18
>>>>       <24>   DW_AT_type        : <0x49>
>>>>       <28>   DW_AT_external    : 1
>>>>       <28>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0     
>>>> (DW_OP_addr: 0)
>>>>       <32>   DW_AT_sibling     : <0x49>
>>>>    <2><36>: Abbrev Number: 1 (User TAG value: 0x6000)
>>>>       <37>   DW_AT_name        : (indirect string, offset: 0xd6): 
>>>> debug_annotate_decl
>>>>       <3b>   DW_AT_const_value : (indirect string, offset: 0xcd): 
>>>> decltag2
>>>>    <2><3f>: Abbrev Number: 1 (User TAG value: 0x6000)
>>>>       <40>   DW_AT_name        : (indirect string, offset: 0xd6): 
>>>> debug_annotate_decl
>>>>       <44>   DW_AT_const_value : (indirect string, offset: 0x0): 
>>>> decltag1
>>>>    <2><48>: Abbrev Number: 0
>>>>    <1><49>: Abbrev Number: 4 (DW_TAG_pointer_type)
>>>>       <4a>   DW_AT_byte_size   : 8
>>>>       <4b>   DW_AT_type        : <0x5d>
>>>>       <4f>   DW_AT_sibling     : <0x5d>
>>>>    <2><53>: Abbrev Number: 1 (User TAG value: 0x6000)
>>>>       <54>   DW_AT_name        : (indirect string, offset: 0x9): 
>>>> debug_annotate_type
>>>>       <58>   DW_AT_const_value : (indirect string, offset: 0x1d): 
>>>> typetag1
>>>>    <2><5c>: Abbrev Number: 0
>>>>    <1><5d>: Abbrev Number: 5 (DW_TAG_base_type)
>>>>       <5e>   DW_AT_byte_size   : 4
>>>>       <5f>   DW_AT_encoding    : 5    (signed)
>>>>       <60>   DW_AT_name        : int
>>>>    <1><64>: Abbrev Number: 0
> 
> This shows the info in .debug_abbrev. What I mean is to
> show the related info in .debug_info section which seems more useful to
> understand the relationships between different tags. Maybe this is due 
> to that I am not fully understanding what <1>/<2> means in <1><49> and 
> <2><53> etc.
> 
>>>
>>> Maybe you can also show what dwarf debug_info looks like
>> I am not sure what you mean. This is the .debug_info section as output
>> by readelf -w. I did trim some information not relevant to the discussion
>> such as the DW_TAG_compile_unit DIE, for brevity.
>>
>>>
>>>>
>>>> In the case of BTF, the annotations are recorded in two type kinds 
>>>> recently
>>>> added to the BTF specification: BTF_KIND_DECL_TAG and 
>>>> BTF_KIND_TYPE_TAG.
>>>> The above example declaration prodcues the following BTF information:
>>>>
>>>> [1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
>>>> [2] PTR '(anon)' type_id=3
>>>> [3] TYPE_TAG 'typetag1' type_id=1
>>>> [4] DECL_TAG 'decltag1' type_id=6 component_idx=-1
>>>> [5] DECL_TAG 'decltag2' type_id=6 component_idx=-1
>>>> [6] VAR 'x' type_id=2, linkage=global
>>>> [7] DATASEC '.bss' size=0 vlen=1
>>>>     type_id=6 offset=0 size=8 (VAR 'x')
>>>>
>>>>
>>> [...]