Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[Sharma, Alok Kumar]

Hi all,

I am planning to contribute for supporting split DWARF.

Please let me know if it is already being worked on?

Regards,
Alok

Re: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[Mark Wielaard]

Hi Alok,

On Mon, Feb 15, 2021 at 05:26:29PM +0000, Sharma, Alok Kumar via Dwz wrote:
> I am planning to contribute for supporting split DWARF.
> 
> Please let me know if it is already being worked on?

That is great. Nobody is currently working on that as far as I
know. One issue to resolve is knowing how to "unsplit" the DWARF
first. Since it isn't clear (to me) whether it makes sense to do split
DWARF .dwo (sections/files) together with Supplementary DWARF Object
references. One idea might be to handle DWP processed files?

There are a couple of bugs that might be prerequirements for handling
split DWARF, you probably need "split" variants of these bugs:
https://sourceware.org/bugzilla/show_bug.cgi?id=27376
https://sourceware.org/bugzilla/show_bug.cgi?id=27375
https://sourceware.org/bugzilla/show_bug.cgi?id=27373

See also this comment:
https://sourceware.org/bugzilla/show_bug.cgi?id=24726#c5
Which describes some other issues, like DW_FORM_strx, .debug_str_offset, 
DW_FORM_rnglistx and various _base attributes that need supporting.
(This should probabably have their own bugs filed.)

There is already one explicit -gsplit-dwarf variant bug:
https://sourceware.org/bugzilla/show_bug.cgi?id=24756 But that is only
for part of the problem (and might be simply resolved by bug #27375).

Cheers,

Mark

RE: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[E, Nagajyothi]

[AMD Official Use Only - Internal Distribution Only]

Hi,

I am working on adding support for the dwarf5 .debug_str_offsets, .debug_addr sections and the related forms DW_FORM_strx[1234], DW_FORM_addrx[1234].
Please let me know if anyone has already started work on these.

Thanks,
Nagajyothi

-----Original Message-----
From: Mark Wielaard <mark@klomp.org> 
Sent: Tuesday, February 16, 2021 12:24 AM
To: Sharma, Alok Kumar <AlokKumar.Sharma@amd.com>
Cc: dwz@sourceware.org; George, Jini Susan <JiniSusan.George@amd.com>; E, Nagajyothi <Nagajyothi.E@amd.com>; Achra, Nitika <Nitika.Achra@amd.com>
Subject: Re: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[CAUTION: External Email]

Hi Alok,

On Mon, Feb 15, 2021 at 05:26:29PM +0000, Sharma, Alok Kumar via Dwz wrote:
> I am planning to contribute for supporting split DWARF.
>
> Please let me know if it is already being worked on?

That is great. Nobody is currently working on that as far as I know. One issue to resolve is knowing how to "unsplit" the DWARF first. Since it isn't clear (to me) whether it makes sense to do split DWARF .dwo (sections/files) together with Supplementary DWARF Object references. One idea might be to handle DWP processed files?

There are a couple of bugs that might be prerequirements for handling split DWARF, you probably need "split" variants of these bugs:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D27376&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7Cedeb271285e94b3a6d6b08d8d1e3447a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637490121349829241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=HiyzxPs6SJnUE%2BMEKAsUE8HL2dvneJD1V7Yfcb9ZDcQ%3D&amp;reserved=0
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D27375&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7Cedeb271285e94b3a6d6b08d8d1e3447a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637490121349829241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=QpjPErW8i8JG75ke528vxPry5pvkwfIq6QO3yxdxURo%3D&amp;reserved=0
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D27373&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7Cedeb271285e94b3a6d6b08d8d1e3447a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637490121349829241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h4WU7ejc7KeSBEluEAgnM5%2FRMRTOVMjRXuzZVbdBlec%3D&amp;reserved=0

See also this comment:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D24726%23c5&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7Cedeb271285e94b3a6d6b08d8d1e3447a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637490121349829241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=uLu90OuqGGBKIFwqItGHPNomPHb2HJNG8TkXKJ%2FUggM%3D&amp;reserved=0
Which describes some other issues, like DW_FORM_strx, .debug_str_offset, DW_FORM_rnglistx and various _base attributes that need supporting.
(This should probabably have their own bugs filed.)

There is already one explicit -gsplit-dwarf variant bug:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D24756&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7Cedeb271285e94b3a6d6b08d8d1e3447a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637490121349829241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2BoaV%2Fqfl7feS7siRNIsze0aZui7S9qN17vBj%2FYOkjdA%3D&amp;reserved=0 But that is only for part of the problem (and might be simply resolved by bug #27375).

Cheers,

Mark

Re: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[Mark Wielaard]

Hi Nagajyothi,

On Wed, 2021-02-17 at 05:53 +0000, E, Nagajyothi via Dwz wrote:
> I am working on adding support for the dwarf5 .debug_str_offsets,
> .debug_addr sections and the related forms DW_FORM_strx[1234], DW_FORM_addrx[1234].
> Please let me know if anyone has already started work on these.

Thanks, I don't believe anybody is working on this right now.

There is already a bug for .debug_addr/DW_FORM_addrx[1234] support:
https://sourceware.org/bugzilla/show_bug.cgi?id=27375
I think this shouldn't be too hard. Any DIE that references an address
cannot be moved to an alt file, so simply mark it with
die_no_multifile. We do try to optimize DW_AT_low/high_pc attribute
pairs, so double check that doesn't interfere with DW_FORM_addrx[1234]
processing.

Also note that there are two new location operations DW_OP_addrx and
DW_OP_constx that can reference the .debug_addr section index.

As a followup you could try optimizing things, but then you need to
keep track of all the addresses used and see if there are duplicates
that could be turned into a (smaller) DW_FORM_addrx[1234] index. But I
would keep that separate from the initial implementation.

For .debug_str_offset/DW_FORM_strx[1234] I opened a new bug.
https://sourceware.org/bugzilla/show_bug.cgi?id=27434

This is a little trickier than DW_FORM_addrx[1234] since such DIEs
using DW_FORM_strx may be moved to a multifile. You also probably want
to make them equal to DW_FORM_strp entries pointing (indirectly) to the
same string.

You might want to add a note to the the bugs that you are working on
them.

Thanks,

Mark

RE: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[E, Nagajyothi]

[AMD Official Use Only - Internal Distribution Only]

Thanks Mark, for your valuable pointers.

-----Original Message-----
From: Mark Wielaard <mark@klomp.org> 
Sent: Thursday, February 18, 2021 8:15 PM
To: E, Nagajyothi <Nagajyothi.E@amd.com>; dwz@sourceware.org
Cc: George, Jini Susan <JiniSusan.George@amd.com>; Achra, Nitika <Nitika.Achra@amd.com>; Sharma, Alok Kumar <AlokKumar.Sharma@amd.com>
Subject: Re: Plan to contribute for supporting split dwarf (-gsplit-dwarf) in dwz

[CAUTION: External Email]

Hi Nagajyothi,

On Wed, 2021-02-17 at 05:53 +0000, E, Nagajyothi via Dwz wrote:
> I am working on adding support for the dwarf5 .debug_str_offsets, 
> .debug_addr sections and the related forms DW_FORM_strx[1234], DW_FORM_addrx[1234].
> Please let me know if anyone has already started work on these.

Thanks, I don't believe anybody is working on this right now.

There is already a bug for .debug_addr/DW_FORM_addrx[1234] support:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D27375&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7C37c0fb4399464264b01808d8d41bcb9c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637492563147320988%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=7AjB%2BXKo9J5gTlW1CeEhHXZykYKB2vFZ%2BTnBQvW4%2BIA%3D&amp;reserved=0
I think this shouldn't be too hard. Any DIE that references an address cannot be moved to an alt file, so simply mark it with die_no_multifile. We do try to optimize DW_AT_low/high_pc attribute pairs, so double check that doesn't interfere with DW_FORM_addrx[1234] processing.

Also note that there are two new location operations DW_OP_addrx and DW_OP_constx that can reference the .debug_addr section index.

As a followup you could try optimizing things, but then you need to keep track of all the addresses used and see if there are duplicates that could be turned into a (smaller) DW_FORM_addrx[1234] index. But I would keep that separate from the initial implementation.

For .debug_str_offset/DW_FORM_strx[1234] I opened a new bug.
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceware.org%2Fbugzilla%2Fshow_bug.cgi%3Fid%3D27434&amp;data=04%7C01%7CNagajyothi.E%40amd.com%7C37c0fb4399464264b01808d8d41bcb9c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637492563147330979%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=AqQRLjgKh9h7YHPIr4wDiqP1rRTXz2kG6IZlAT0SaSQ%3D&amp;reserved=0

This is a little trickier than DW_FORM_addrx[1234] since such DIEs using DW_FORM_strx may be moved to a multifile. You also probably want to make them equal to DW_FORM_strp entries pointing (indirectly) to the same string.

You might want to add a note to the the bugs that you are working on them.

Thanks,

Mark