DW_TAG_imported_unit support

DW_TAG_imported_unit support

[Roland McGrath]

When my DWARF size reduction plan comes along on the producer side,
there will be some new format elements and/or extensions that GDB
will want to grok so it can use the packaged debuginfo directly in
its most compact form.  This remains nebulous since we're still a
few months away from having any tool to produce any new-style data.
But I want to let you plan to be ready for what I think we'll have.

I have various ideas for low-level variations/extensions of the format that
would be used for the most thoroughly compact way to package debuginfo.
Actuals details of those remain quite vague and yet to be fleshed out.
Fortunately they are sufficiently coherent in my own mind that I can be
pretty sure all that will be the less onerous bits to address in GDB/BFD's
DWARF reader code.

The key piece of the plan that is not an extension, but a proper part of
DWARF 3--just one that we have never used or supported before.  That is
DW_TAG_imported_unit (section 3.1.2 of DWARF 3).

This is a magical "middle-level" DWARF feature that is supposed to tell the
reader to graft a physically separate DWARF tree in as a subtree at another
given point in the reader's "logical" view of the overall DIE tree rooted
in a CU.  In this way, common subtrees can be moved off and then shared by
many different containing DIEs (often in multiple different CUs).  I call
it "middle-level" because the tag itself is encoded in the normal format--a
DW_TAG_imported_unit node with a single DW_AT_import attribute--but what is
specifies is not a "consumer"-level semantic node that the exists in the
conceptual DIE tree emitted by the compiler, rather it's a "transparent"
DWARF format detail that the consumer should pretend doesn't exist for
purposes of its semantic DIE-traversal algorithms.

So it needs to be understood by every place that does a DIE tree walk (or
just flat enumeration of a given DIE's children).  (Ideally this would be
consolidated in one "logical tree walker" place, but I imagine the existing
code might not all be quite so nicely-structured.)  Handling it in a walk
is pretty straightforward: you follow the DW_AT_import (which should always
be DW_FORM_ref_addr) to another DIE, which is either DW_TAG_compile_unit or
DW_TAG_partial_unit.  All the children of that DIE appear in the logical
tree in place of the DW_TAG_imported_unit child.

The other main wrinkle of DW_TAG_imported_unit is the "which CU" issue.
The referent DIE is a CU, but logically all the DIEs (its children) you
look at belong to the referring CU.  For the low-level format pointers,
the referent CU is what matters: which line table DW_AT_decl_file refers
to, etc.  For the stuff about using "the CU" as context for how you
understand DIEs at a semantic level, it's the logically-containing
(referring) CU that matters: finding the relevant "top of tree" to
search what's in scope, etc.

So, I didn't really mean to try to cover the whole technical subject.
I'm sure there are many more corners we will stumble across, and as I
said this work anticipates producer/transformer tools that aren't there
yet to drive the need.  But now I've put this task on your radar.


Thanks,
Roland

Re: DW_TAG_imported_unit support

[Tom Tromey]

>>>>> "Roland" == Roland McGrath <roland@redhat.com> writes:

Roland> The key piece of the plan that is not an extension, but a
Roland> proper part of DWARF 3--just one that we have never used or
Roland> supported before.  That is DW_TAG_imported_unit (section 3.1.2
Roland> of DWARF 3).

Thanks for the heads up.  I filed a bug for this:

    http://sourceware.org/bugzilla/show_bug.cgi?id=10240

Tom

Re: DW_TAG_imported_unit support

[Doug Evans]

On Wed, Jun 3, 2009 at 8:58 PM, Roland McGrath<roland@redhat.com> wrote:
> When my DWARF size reduction plan comes along on the producer side,
> there will be some new format elements and/or extensions that GDB
> will want to grok so it can use the packaged debuginfo directly in
> its most compact form.  This remains nebulous since we're still a
> few months away from having any tool to produce any new-style data.
> But I want to let you plan to be ready for what I think we'll have.
>
> I have various ideas for low-level variations/extensions of the format that
> would be used for the most thoroughly compact way to package debuginfo.
> Actuals details of those remain quite vague and yet to be fleshed out.
> Fortunately they are sufficiently coherent in my own mind that I can be
> pretty sure all that will be the less onerous bits to address in GDB/BFD's
> DWARF reader code.
>
> The key piece of the plan that is not an extension, but a proper part of
> DWARF 3--just one that we have never used or supported before.  That is
> DW_TAG_imported_unit (section 3.1.2 of DWARF 3).
>
> This is a magical "middle-level" DWARF feature that is supposed to tell the
> reader to graft a physically separate DWARF tree in as a subtree at another
> given point in the reader's "logical" view of the overall DIE tree rooted
> in a CU.  In this way, common subtrees can be moved off and then shared by
> many different containing DIEs (often in multiple different CUs).  I call
> it "middle-level" because the tag itself is encoded in the normal format--a
> DW_TAG_imported_unit node with a single DW_AT_import attribute--but what is
> specifies is not a "consumer"-level semantic node that the exists in the
> conceptual DIE tree emitted by the compiler, rather it's a "transparent"
> DWARF format detail that the consumer should pretend doesn't exist for
> purposes of its semantic DIE-traversal algorithms.
>
> So it needs to be understood by every place that does a DIE tree walk (or
> just flat enumeration of a given DIE's children).  (Ideally this would be
> consolidated in one "logical tree walker" place, but I imagine the existing
> code might not all be quite so nicely-structured.)  Handling it in a walk
> is pretty straightforward: you follow the DW_AT_import (which should always
> be DW_FORM_ref_addr) to another DIE, which is either DW_TAG_compile_unit or
> DW_TAG_partial_unit.  All the children of that DIE appear in the logical
> tree in place of the DW_TAG_imported_unit child.
>
> The other main wrinkle of DW_TAG_imported_unit is the "which CU" issue.
> The referent DIE is a CU, but logically all the DIEs (its children) you
> look at belong to the referring CU.  For the low-level format pointers,
> the referent CU is what matters: which line table DW_AT_decl_file refers
> to, etc.  For the stuff about using "the CU" as context for how you
> understand DIEs at a semantic level, it's the logically-containing
> (referring) CU that matters: finding the relevant "top of tree" to
> search what's in scope, etc.
>
> So, I didn't really mean to try to cover the whole technical subject.
> I'm sure there are many more corners we will stumble across, and as I
> said this work anticipates producer/transformer tools that aren't there
> yet to drive the need.  But now I've put this task on your radar.

As a data point, reading this made me think of the comdat types
support in DWARF 4.

http://wiki.dwarfstd.org/index.php?title=COMDAT_Type_Sections

I'm not suggesting the two are identical, comdat types is just for,
well, types. :-)
But both tell the reader to (at least conceptually) graft a physically
separate DWARF tree in as a subtree.