Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Diego Novillo]

[ Moved to gcc@gcc.gnu.org ]

On Fri, Aug 1, 2008 at 03:33, Bill Maddox <maddox@google.com> wrote:

> I am continuing to make progress with the LTO streamer on C++ code,
> and finding a considerable amount  of front-end specific crud hanging
> off of the GIMPLE, particularly related to templates.  This is not
> surprising given the current strategy for generating debug information
> (late, in the back-end).   The LTO IR must be language-independent,
> hower, and the current short-term plan of record, as I understand it,
> is to strip out left-over front-end specific parts of the tree in
> tree.c:free_lang_specifics, accepting that this will break debug info
> generation until a satisfactory alternative is devised.

Yes.  That coupled with early debug info emission in the FE.

>
> Finding that DECL_CONTEXT might include portions of what looked like
> template definitions, and surmising that  lexical nesting was a
> language artifact that should no longer be of significance beyond the
> front-end, I attempted to clear DECL_CONTEXT in free_lang_specifics.
> This resulted in an assertion failure in
> cp/mangle.c:write_unscoped_name.  It appears that the name mangling
> code, as one would expect, is very interested in such nesting, and
> examines DECL_CONTEXT in multiple locations.  None of this would be of
> any consequence if the name mangling were complete upon entry to the
> middle-end, but it can in fact be invoked via a langhook from
> DECL_ASSEMBLER_NAME.  Setting the assembler name is performed lazily
> in the back-end, requiring nesting relationships reflected in
> DECL_CONTEXT to be preserved to this point.
>
> My assertion is that the langhook must go, and that mangled names
> should be generated before invoking the middle-end.  Given the baroque
> treatment of the assembler name, however, I don't think it will be
> sufficient simply to make a pass over the declarations in
> free_lang_specifics setting the assembler name.
>
> Any advice from someone who deeply understands this stuff?

I can't claim any kind of knowledge of the C++ FE, but from the
point of view of the middle end, I would prefer if all the names
are decided right before, or during gimplification.

FE folks, any reason why we couldn't set assembler name as the FE
hands off the function to the gimplifier?  In principle,
free_lang_specifics would be a good place for it, I think.


Diego.

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Mark Mitchell]

Diego Novillo wrote:

>> My assertion is that the langhook must go, and that mangled names
>> should be generated before invoking the middle-end.  Given the baroque
>> treatment of the assembler name, however, I don't think it will be
>> sufficient simply to make a pass over the declarations in
>> free_lang_specifics setting the assembler name.
>>
>> Any advice from someone who deeply understands this stuff?

That's certain the only choice if we're going to throw away DECL_CONTEXT 
(and lots of other information).  However, the reason that names are 
mangled lazily is that it was a significant compile-time performance 
improvement.  There are *tons* of functions in the typical C++ 
translation unit when you start #includ'ing big headers.  Typically, 
thought many of them reference one-another, only a few definitions are 
actually output.  They reference other functions whose definitions are 
not emitted, but not that many.  So, you'd like to avoid mangling all of 
them.

My concern is that the path we're heading towards is:

1. C++ front end generates all function bodies, including instantiating 
all templates that might be needed, mangles all names that might be 
needed, and precomputes all debug info that might be needed, and then 
hands the whole mess off to the gimplifier.

2. Middle-end builds call graphs, etc., and throws out 99.5% of the 
functions and debug info, after deciding it's not needed.  (Note, for 
example, that mangled names are typically not needed for inline functions.)

That's going to make -flto slow.  If we do it in a way that affects 
normal -O0 compilation, this would be a bad thing.  Maybe we should 
consider a strategy where the front-end data structures live until the 
point of LTO output, so that we can compute what we need at that point, 
based only on what we are going to output.  (And for DECL_COMDAT things, 
we should be outputting LTO information only if actually referenced.)

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Bill Maddox]

On Fri, Aug 1, 2008 at 7:53 AM, Mark Mitchell <mark@codesourcery.com> wrote:

> That's going to make -flto slow.  If we do it in a way that affects normal
> -O0 compilation, this would be a bad thing.  Maybe we should consider a
> strategy where the front-end data structures live until the point of LTO
> output, so that we can compute what we need at that point, based only on
> what we are going to output.  (And for DECL_COMDAT things, we should be
> outputting LTO information only if actually referenced.)

Before free_lang_specifics was added, I dropped some of the tree while
streaming it out.  The advantage of this is that it doesn't trash the
FE-specfic info for the currently-running compilation, so that debug
info should be good for object code emitted during a -flto compile and
linked normally.

To move toward a language-independent IR, however, it is good hygiene
to strip this stuff so we aren't fooling ourselves.  When the lto IR
is streamed back in, lto1 will pick up the compilation more or less at
the point where it stood when the IR was written out, and proactively
cleaning the tree in free_lang_specifics will help us to anticipate
problems further along in the pipeline that might presently be masked
in our testing of lto1 by other streamer failures.

I agree that doing early name mangling at streamout time, only for the
declarations that are actually referenced and must be streamed out,
could be a huge win in the presence of large but sparsely-used header
files. I will try a few experiments here.  I'm not terribly
comfortable with aggressively cleaning the tree in free_lang_specifics
until we've clarified a GIMPLE type system and a plan for adjusting
early-generated debug info to reflect middle-end optimization.

--Bill

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Mark Mitchell]

Bill Maddox wrote:

> To move toward a language-independent IR, however, it is good hygiene
> to strip this stuff so we aren't fooling ourselves.

Yes, the question is just when to do the stripping.  My point is that 
stripping early implies that the FE must generate everything you might 
ever want early and that is quite possibly expensive.

> I'm not terribly
> comfortable with aggressively cleaning the tree in free_lang_specifics
> until we've clarified a GIMPLE type system and a plan for adjusting
> early-generated debug info to reflect middle-end optimization.

That's kinda what I think too.  I'm we're approaching this as "throw 
everything away, see what breaks, fix it" and that seems like a risky 
method for such a major change.  If we go down this path and don't like 
it, it will be hard to back out.

The only thing actually required for LTO to work is that we not use FE 
callbacks to interpret the intermediate.  There's no inherent reason all 
the FE-specific data has to be freed.  So, we do have to mangle names 
before we write out LTO information (since we won't have an FE to mangle 
them for us later), but we don't necessarily have to do that at the 
point of GIMPLE-fication.

In fact, if we have big pieces of tree that are swapped out (e.g., PCH), 
running around freeing bits of them is probably going to slow things 
down, rather than speed them up.  In the case of PCH, we're going to be 
touching disk.  Maybe we should just not write the data out to the LTO 
file, rather than trying to free it early.  If you like, have a 
free-it-early mode for error-checking (to make sure that we aren't 
relying on that data) in the same way that we have an always-collect 
mode for GC (to make sure we're not depending on things not being 
collected).

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Chris Lattner]

On Aug 1, 2008, at 7:53 AM, Mark Mitchell wrote:
> My concern is that the path we're heading towards is:

...

> 2. Middle-end builds call graphs, etc., and throws out 99.5% of the  
> functions and debug info, after deciding it's not needed.  (Note,  
> for example, that mangled names are typically not needed for inline  
> functions.)

Note that you often still need the mangled form of a symbol name when  
inlined if -g is present.

-Chris

Re: [lto] C++. streaming, front-end specific tree nodes, IR types, and assembler names

[Michael Matz]

Hi Bill,

On Fri, 1 Aug 2008, Diego Novillo wrote:

> > Finding that DECL_CONTEXT might include portions of what looked like
> > template definitions, and surmising that  lexical nesting was a
> > language artifact that should no longer be of significance beyond the
> > front-end, I attempted to clear DECL_CONTEXT in free_lang_specifics.
> > This resulted in an assertion failure in
> > cp/mangle.c:write_unscoped_name.  It appears that the name mangling
> > code, as one would expect, is very interested in such nesting, and
> > examines DECL_CONTEXT in multiple locations.  None of this would be of
> > any consequence if the name mangling were complete upon entry to the
> > middle-end, but it can in fact be invoked via a langhook from
> > DECL_ASSEMBLER_NAME.  Setting the assembler name is performed lazily
> > in the back-end, requiring nesting relationships reflected in
> > DECL_CONTEXT to be preserved to this point.
> >
> > My assertion is that the langhook must go, and that mangled names
> > should be generated before invoking the middle-end.  Given the baroque
> > treatment of the assembler name, however, I don't think it will be
> > sufficient simply to make a pass over the declarations in
> > free_lang_specifics setting the assembler name.
> >
> > Any advice from someone who deeply understands this stuff?

See also my patch from 
http://gcc.gnu.org/ml/gcc-patches/2008-03/msg00349.html that had to deal 
with the same problem.  It indeed simply sets the asm name from 
free_lang_specifics (see set_asm_name() in there).  One issue is to watch 
out for string memory, one potentially generates names for decls that 
later are thrown away.  I found this to be only a minor issue with 
tramp3d, though.


Ciao,
Michael.