multi debug files and artificial module

multi debug files and artificial module

[Sasha Da Rocha Pinheiro]

Hi,
we are currently dealing with multiple separate debug files, the normal stripped ones put in .debug/ folder and now the ones generated by DWZ and put into .dwz/ folder.
When loading a normal stripped debug files who has a dwz file, I saw the same DIE (same id) twice with different data. Would it be a bug in DWZ or a correct dwarf state?
Also is "<artificial>" the name of the following compilation unit? Or is it a bug in eu-redealf/libdw?

Sasha
Thanks

Compilation unit at offset 946:
 Version: 4, Abbreviation section offset: 0, Address size: 8, Offset size: 4
 [   3bd]  compile_unit         abbrev: 63
           producer             (strp) "GNU GIMPLE 10.2.1 20200723 (Red Hat 10.2.1-1) -m64 -mtune=generic -march=x86-64 -g -g -O2 -O2 -fno-openmp -fno-openacc -fPIC -fstack-protector-strong -fltrans -fplugin=ann
obin"
           language             (data1) C99 (12)
           name                 (GNU_strp_alt) "<artificial>"
           comp_dir             (GNU_strp_alt) "/usr/src/debug/libiscsi-1.19.0-2.fc33.x86_64/lib"
           low_pc               (addr) +0x0000000000008030 <iscsi_free_discovery_addresses>
           high_pc              (udata) 51811 (+0x0000000000014a93 <.annobin_iscsi_extended_copy_task.end>)
           stmt_list            (sec_offset) 0

Re: multi debug files and artificial module

[Mark Wielaard]

Hi Sasha,

On Tue, 2020-11-03 at 21:37 +0000, Sasha Da Rocha Pinheiro via
Elfutils-devel wrote:
> we are currently dealing with multiple separate debug files, the
> normal stripped ones put in .debug/ folder and now the ones generated
> by DWZ and put into .dwz/ folder.
> When loading a normal stripped debug files who has a dwz file, I saw
> the same DIE (same id) twice with different data. Would it be a bug
> in DWZ or a correct dwarf state?
> Also is "<artificial>" the name of the following compilation unit? Or
> is it a bug in eu-redealf/libdw?

Looking at what you posted you are actually looking at 3 different
types of CU DIEs. The "normal" separate .debug DIEs. The supplemental
(dwz alt file) DIEs and LTO (gcc -flto generated) DIEs. For the last
ones (which have GNU GIMPLE as producer, the internal GCC
representation of the program) it is correct to have them marked
"artificial", these CUs contain common code/types from the objects
combined by LTO (Link Time Optimization). If by "same id" you mean
"offset" (the hex value in square brackets) then yes, DIE offsets in
separate files (Dwarf objects) can be the same. The DIEs from the
.debug file and the DIEs from the .multi (supplemental) file are
represented by different Dwarf objects and DIEs with the same offset in
separate Dwarf objects are different DIEs.

Cheers,

Mark

> Compilation unit at offset 946:
>  Version: 4, Abbreviation section offset: 0, Address size: 8, Offset
> size: 4
>  [   3bd]  compile_unit         abbrev: 63
>            producer             (strp) "GNU GIMPLE 10.2.1 20200723
> (Red Hat 10.2.1-1) -m64 -mtune=generic -march=x86-64 -g -g -O2 -O2
> -fno-openmp -fno-openacc -fPIC -fstack-protector-strong -fltrans
> -fplugin=ann
> obin"
>            language             (data1) C99 (12)
>            name                 (GNU_strp_alt) "<artificial>"
>            comp_dir             (GNU_strp_alt)
> "/usr/src/debug/libiscsi-1.19.0-2.fc33.x86_64/lib"
>            low_pc               (addr) +0x0000000000008030
> <iscsi_free_discovery_addresses>
>            high_pc              (udata) 51811 (+0x0000000000014a93
> <.annobin_iscsi_extended_copy_task.end>)
>            stmt_list            (sec_offset) 0

Re: multi debug files and artificial module

[Sasha Da Rocha Pinheiro]

Hi Mark,
Thanks for your response.

In libdw.h it says: /* The offset can be computed from the address.  */
How do I get the CU DIE offset from the address? Only saving the first CU and subtracting it on the others to get the offset?

When we go through the .debug_info using dwarf_nextcu, we are getting partial units too. How should we deal with them? If they're not actually CU, should they not be returned and only used internally by libdw in order to 'complete' the other CUs? 

It seems that libdw is automatically searching and loading the supplemental dwz file. Given the following:

** file .debug **
<1><f0d>: Abbrev Number: 37 (DW_TAG_subprogram)
    <f0e>   DW_AT_abstract_origin: <alt 0x5112>
    <f12>   DW_AT_low_pc      : 0x8650
    <f1a>   DW_AT_high_pc     : 229
    <f1c>   DW_AT_frame_base  : 1 byte block: 9c        (DW_OP_call_frame_cfa)
    <f1e>   DW_AT_GNU_all_call_sites: 1
    <f1e>   DW_AT_sibling     : <0x107b>
...
 <4><5112>: Abbrev Number: 0

** file dwz **
<1><5112>: Abbrev Number: 81 (DW_TAG_subprogram)
    <5113>   DW_AT_external    : 1
    <5113>   DW_AT_name        : (indirect string, offset: 0x9852): iscsi_tcp_free_pdu
    <5117>   DW_AT_decl_file   : 51
    <5118>   DW_AT_decl_line   : 142
    <5119>   DW_AT_decl_column : 1
    <511a>   DW_AT_prototyped  : 1
    <511a>   DW_AT_sibling     : <0x5135>

When I'm parsing f0d, libdw automatically fetches the abstract origin on the dwz file. But when we try to parse the abstract origin on our own it gives us the one in the same file, which is 0. Is it because we should look into the form of the DW_AT_abstract_origin? In this case seems to be the GNU_ref_alt, correct?

Regards,
Sasha



From: Mark Wielaard <mark@klomp.org>
Sent: Wednesday, November 4, 2020 7:35 AM
To: Sasha Da Rocha Pinheiro <darochapinhe@wisc.edu>; elfutils-devel@sourceware.org <elfutils-devel@sourceware.org>
Cc: Tim Haines <thaines@astro.wisc.edu>; bolo@cs.wisc.edu <bolo@cs.wisc.edu>
Subject: Re: multi debug files and artificial module 
 
Hi Sasha,

On Tue, 2020-11-03 at 21:37 +0000, Sasha Da Rocha Pinheiro via
Elfutils-devel wrote:
> we are currently dealing with multiple separate debug files, the
> normal stripped ones put in .debug/ folder and now the ones generated
> by DWZ and put into .dwz/ folder.
> When loading a normal stripped debug files who has a dwz file, I saw
> the same DIE (same id) twice with different data. Would it be a bug
> in DWZ or a correct dwarf state?
> Also is "<artificial>" the name of the following compilation unit? Or
> is it a bug in eu-redealf/libdw?

Looking at what you posted you are actually looking at 3 different
types of CU DIEs. The "normal" separate .debug DIEs. The supplemental
(dwz alt file) DIEs and LTO (gcc -flto generated) DIEs. For the last
ones (which have GNU GIMPLE as producer, the internal GCC
representation of the program) it is correct to have them marked
"artificial", these CUs contain common code/types from the objects
combined by LTO (Link Time Optimization). If by "same id" you mean
"offset" (the hex value in square brackets) then yes, DIE offsets in
separate files (Dwarf objects) can be the same. The DIEs from the
.debug file and the DIEs from the .multi (supplemental) file are
represented by different Dwarf objects and DIEs with the same offset in
separate Dwarf objects are different DIEs.

Cheers,

Mark

> Compilation unit at offset 946:
>  Version: 4, Abbreviation section offset: 0, Address size: 8, Offset
> size: 4
>  [   3bd]  compile_unit         abbrev: 63
>            producer             (strp) "GNU GIMPLE 10.2.1 20200723
> (Red Hat 10.2.1-1) -m64 -mtune=generic -march=x86-64 -g -g -O2 -O2
> -fno-openmp -fno-openacc -fPIC -fstack-protector-strong -fltrans
> -fplugin=ann
> obin"
>            language             (data1) C99 (12)
>            name                 (GNU_strp_alt) "<artificial>"
>            comp_dir             (GNU_strp_alt)
> "/usr/src/debug/libiscsi-1.19.0-2.fc33.x86_64/lib"
>            low_pc               (addr) +0x0000000000008030
> <iscsi_free_discovery_addresses>
>            high_pc              (udata) 51811 (+0x0000000000014a93
> <.annobin_iscsi_extended_copy_task.end>)
>            stmt_list            (sec_offset) 0

Re: multi debug files and artificial module

[Mark Wielaard]

Hi Sasha,

On Thu, 2020-11-05 at 22:15 +0000, Sasha Da Rocha Pinheiro via
Elfutils-devel wrote:
> In libdw.h it says: /* The offset can be computed from the address.
>  */
> How do I get the CU DIE offset from the address? Only saving the
> first CU and subtracting it on the others to get the offset?

Yeah, that comment is correct, but not really that helpful. There are
two offsets that might matter. The start of Dwarf (.debug_info) offset
and the start of the Dwarf_CU offset. But getting either by hand is
some work since it depends on the size of the CU header, which can be
different based on the version and type of CU. You can get the
header_size when iterating through the CUs with dwarf_nextcu or
dwarf_next_unit but it is not easily available otherwise.

But given a Dwarf_Die you can get the offsets through dwarf_dieoffset
and dwarf_cuoffset.

Also note that since 0.178 there is dwarf_die_addr_die, which simply
takes the (main) Dwarf object and a Dwarf_Die addr.

/* Given a Dwarf_Die addr returns a (reconstructed) Dwarf_Die, or NULL
   if the given addr didn't come from a valid Dwarf_Die.  In particular
   it will make sure that the correct Dwarf_CU pointer is set for the
   Dwarf_Die, the Dwarf_Abbrev pointer will not be set up yet (it will
   only be once the Dwarf_Die is used to read attributes, children or
   siblings).  This functions can be used to keep a reference to a
   Dwarf_Die which you want to refer to later.  The addr, and the result
   of this function, is only valid while the associated Dwarf is valid.  */

Although the comment doesn't explicitly say so, this also works for
DIEs from the associated alt or split Dwarf objects associated with the
given Dwarf.

> When we go through the .debug_info using dwarf_nextcu, we are getting
> partial units too. How should we deal with them? If they're not
> actually CU, should they not be returned and only used internally by
> libdw in order to 'complete' the other CUs?

You should probably simply skip them because they will ultimately be
imported into a "real" compile unit. Note that there also is
dwarf_get_units () which will directly give you the unit_type
(DW_UT_compile, DW_UT_type or DW_UT_partial) so you can easily skipe
any you are not interested in.

> It seems that libdw is automatically searching and loading the
> supplemental dwz file. Given the following:
> 
> ** file .debug **
> <1><f0d>: Abbrev Number: 37 (DW_TAG_subprogram)
>     <f0e>   DW_AT_abstract_origin: <alt 0x5112>
>     <f12>   DW_AT_low_pc      : 0x8650
>     <f1a>   DW_AT_high_pc     : 229
>     <f1c>   DW_AT_frame_base  : 1 byte block: 9c      
>  (DW_OP_call_frame_cfa)
>     <f1e>   DW_AT_GNU_all_call_sites: 1
>     <f1e>   DW_AT_sibling     : <0x107b>
> ...
>  <4><5112>: Abbrev Number: 0
> 
> ** file dwz **
> <1><5112>: Abbrev Number: 81 (DW_TAG_subprogram)
>     <5113>   DW_AT_external    : 1
>     <5113>   DW_AT_name        : (indirect string, offset: 0x9852):
> iscsi_tcp_free_pdu
>     <5117>   DW_AT_decl_file   : 51
>     <5118>   DW_AT_decl_line   : 142
>     <5119>   DW_AT_decl_column : 1
>     <511a>   DW_AT_prototyped  : 1
>     <511a>   DW_AT_sibling     : <0x5135>
> 
> When I'm parsing f0d, libdw automatically fetches the abstract origin
> on the dwz file. But when we try to parse the abstract origin on our
> own it gives us the one in the same file, which is 0. Is it because
> we should look into the form of the DW_AT_abstract_origin? In this
> case seems to be the GNU_ref_alt, correct?

Yes, if the form is DW_FORM_GNU_ref_alt (or DW_FORM_ref_sup4 or
DW_FORM_ref_sup8 in DWARF5) then the "raw offset" is against the alt
file (which you can get with dwarf_getalt).

But I would recommend you simply use dwarf_formref_die to get the
Dwarf_Die since that has all the logic already to resolve the DIE
whether it is a "global" offset, a CU-relative offset, an alt-reference 
or a type signature.

Cheers,

Mark