From: "H.J. Lu" <hjl.tools@gmail.com>
To: Richard Biener <rguenther@suse.de>
Cc: Jan Hubicka <hubicka@ucw.cz>,
GCC Patches <gcc-patches@gcc.gnu.org>,
Andi Kleen <ak@linux.intel.com>,
Cary Coutant <ccoutant@google.com>,
Ian Lance Taylor <iant@google.com>
Subject: Re: [RFC] Getting LTO incremental linking work
Date: Wed, 25 Nov 2015 15:45:00 -0000 [thread overview]
Message-ID: <CAMe9rOpqi6fM_g1uG2077D+7SrJ5QqBX8mxhPDbah2jQzSpquw@mail.gmail.com> (raw)
In-Reply-To: <alpine.LSU.2.11.1511251155460.4884@t29.fhfr.qr>
On Wed, Nov 25, 2015 at 3:15 AM, Richard Biener <rguenther@suse.de> wrote:
> On Wed, 25 Nov 2015, Jan Hubicka wrote:
>
>> Hi,
>> PR 67548 is about LTO not supporting incremental linking. I never really
>> considered our current incremental linking very useful, because it triggers
>> code generation at the incremental link time basically nullifying any
>> benefits of whole program optimization and in fact I think it is harmful,
>> because it sort of works and w/o any warning produce not very optimized code.
>>
>> Basically there are 3 schemes how to make incremental link work
>> 1) Turn LTO objects to non-LTO as we do now
>> 2) concatenate LTO sections as implemented by Andi and Hj
>> 3) Do actual linking of LTO sections
>>
>> The problem of current implementation of 1) is that GCC thinks the resulting
>> object file will not be used for static linking and thus assume that hidden
>> symbols can be turned to static.
>>
>> In the log of PR67548 HJ actually pointed out that we do have API at linker
>> plugin side which says what type of output is done. This is cool because we
>> can also use it to drop -fpic when building static binary. This is common in
>> Firefox, where some objects are built with -fpic and linked to both binaries
>> and libraries.
>>
>> Moreover we do have all infrastructure ready to implement 3). Our tree merging
>> and symbol table handling is fuly incremental and I think made a patch to
>> implement it today. The scheme is easy:
>>
>> 1) linker plugin is modified to pass -flinker-output to lto wrapper
>> linker-output is either dyn (.so), pie or exec
>> for incremental linking I added .rel for 3) and noltorel for 1)
>>
>> currently it does rel because 3) (nor 2) can not be done when incremnetal
>> linking is done on both LTO and non-LTO objects.
>
> That's because the result would be a "fat" object where both pieces
> would be needed. Btw, I wonder why you are not running into the
> same issues as me when producing linker plugin output (the "merged"
> LTO IL) that is LTO IL. Ah, possibly because the link is incremental,
> and thus all special-handling of LTO sections is disabled.
>
>> In this case linker
>> plugin output warings about code quality loss and switch to
>> noltorel.
>> 2) with -flinker-ouptut the lto wrapper behaves same way as with
>> -flto-partition=none.
>> 3) lto frontend parses -flinker-output and sets our internal flags accordingly.
>> I added new flag_incremental_linking to inform middle-end about the fact
>> that the output is going to be statically linked again. This disables
>> the privatization of hidden symbols and if set to 2 it also triggers
>> the LTO IL streaming
>
> I wonder why it behaves like -flto-partition=none in the case it does
> not need to do LTO IL streaming (which I hope does LTO IL streaming
> only? or does this implement fat objects "correctly"?). Can't
> we still parallelize the build via LTRANS and then incrementally
> link the result (I suppose the linker will do that for us with the
> linker plugin outputs already?)?
>
> -flto-partition=none itself isn't more memory intensive than
> WPA in these days, it's only about compile-time, correct?
>
> Your patch means that Andis/HJs work is no longer needed and we can
> drop the section suffixes again?
>
>
There is a difference between "ld -r " and "gcc -r". "ld -r" may not
perform any LTO.
--
H.J.
next prev parent reply other threads:[~2015-11-25 15:44 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-25 9:04 Jan Hubicka
2015-11-25 11:19 ` Richard Biener
2015-11-25 15:45 ` H.J. Lu [this message]
2015-11-25 19:21 ` Jan Hubicka
2015-11-25 23:09 ` Jan Hubicka
2015-11-25 23:56 ` Jan Hubicka
2015-11-28 10:35 ` Tom de Vries
2015-11-28 12:03 ` Tom de Vries
2015-11-28 16:05 ` Ilya Verbin
2015-11-28 17:41 ` Tom de Vries
2015-11-29 21:15 ` Jan Hubicka
2015-11-25 18:54 ` Jan Hubicka
2015-11-26 10:15 ` Richard Biener
2015-11-26 20:30 ` Jan Hubicka
2015-11-25 23:59 ` Andi Kleen
2015-11-26 0:24 ` Andi Kleen
2015-11-26 0:54 ` Jan Hubicka
2015-11-26 1:55 ` Andi Kleen
2015-11-26 2:02 ` Jan Hubicka
2015-11-26 2:12 ` Andi Kleen
2015-11-26 6:33 ` Jan Hubicka
2015-11-26 10:33 ` Richard Biener
2016-03-16 17:33 ` H.J. Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAMe9rOpqi6fM_g1uG2077D+7SrJ5QqBX8mxhPDbah2jQzSpquw@mail.gmail.com \
--to=hjl.tools@gmail.com \
--cc=ak@linux.intel.com \
--cc=ccoutant@google.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=hubicka@ucw.cz \
--cc=iant@google.com \
--cc=rguenther@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).