From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 72467 invoked by alias); 25 Nov 2015 15:44:11 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 72457 invoked by uid 89); 25 Nov 2015 15:44:11 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.5 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-oi0-f49.google.com Received: from mail-oi0-f49.google.com (HELO mail-oi0-f49.google.com) (209.85.218.49) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Wed, 25 Nov 2015 15:44:09 +0000 Received: by oige206 with SMTP id e206so31060745oig.2 for ; Wed, 25 Nov 2015 07:44:07 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.202.80.204 with SMTP id e195mr24184983oib.75.1448466247562; Wed, 25 Nov 2015 07:44:07 -0800 (PST) Received: by 10.76.72.3 with HTTP; Wed, 25 Nov 2015 07:44:07 -0800 (PST) In-Reply-To: References: <20151125085912.GD58491@kam.mff.cuni.cz> Date: Wed, 25 Nov 2015 15:45:00 -0000 Message-ID: Subject: Re: [RFC] Getting LTO incremental linking work From: "H.J. Lu" To: Richard Biener Cc: Jan Hubicka , GCC Patches , Andi Kleen , Cary Coutant , Ian Lance Taylor Content-Type: text/plain; charset=UTF-8 X-IsSubscribed: yes X-SW-Source: 2015-11/txt/msg03098.txt.bz2 On Wed, Nov 25, 2015 at 3:15 AM, Richard Biener wrote: > On Wed, 25 Nov 2015, Jan Hubicka wrote: > >> Hi, >> PR 67548 is about LTO not supporting incremental linking. I never really >> considered our current incremental linking very useful, because it triggers >> code generation at the incremental link time basically nullifying any >> benefits of whole program optimization and in fact I think it is harmful, >> because it sort of works and w/o any warning produce not very optimized code. >> >> Basically there are 3 schemes how to make incremental link work >> 1) Turn LTO objects to non-LTO as we do now >> 2) concatenate LTO sections as implemented by Andi and Hj >> 3) Do actual linking of LTO sections >> >> The problem of current implementation of 1) is that GCC thinks the resulting >> object file will not be used for static linking and thus assume that hidden >> symbols can be turned to static. >> >> In the log of PR67548 HJ actually pointed out that we do have API at linker >> plugin side which says what type of output is done. This is cool because we >> can also use it to drop -fpic when building static binary. This is common in >> Firefox, where some objects are built with -fpic and linked to both binaries >> and libraries. >> >> Moreover we do have all infrastructure ready to implement 3). Our tree merging >> and symbol table handling is fuly incremental and I think made a patch to >> implement it today. The scheme is easy: >> >> 1) linker plugin is modified to pass -flinker-output to lto wrapper >> linker-output is either dyn (.so), pie or exec >> for incremental linking I added .rel for 3) and noltorel for 1) >> >> currently it does rel because 3) (nor 2) can not be done when incremnetal >> linking is done on both LTO and non-LTO objects. > > That's because the result would be a "fat" object where both pieces > would be needed. Btw, I wonder why you are not running into the > same issues as me when producing linker plugin output (the "merged" > LTO IL) that is LTO IL. Ah, possibly because the link is incremental, > and thus all special-handling of LTO sections is disabled. > >> In this case linker >> plugin output warings about code quality loss and switch to >> noltorel. >> 2) with -flinker-ouptut the lto wrapper behaves same way as with >> -flto-partition=none. >> 3) lto frontend parses -flinker-output and sets our internal flags accordingly. >> I added new flag_incremental_linking to inform middle-end about the fact >> that the output is going to be statically linked again. This disables >> the privatization of hidden symbols and if set to 2 it also triggers >> the LTO IL streaming > > I wonder why it behaves like -flto-partition=none in the case it does > not need to do LTO IL streaming (which I hope does LTO IL streaming > only? or does this implement fat objects "correctly"?). Can't > we still parallelize the build via LTRANS and then incrementally > link the result (I suppose the linker will do that for us with the > linker plugin outputs already?)? > > -flto-partition=none itself isn't more memory intensive than > WPA in these days, it's only about compile-time, correct? > > Your patch means that Andis/HJs work is no longer needed and we can > drop the section suffixes again? > > There is a difference between "ld -r " and "gcc -r". "ld -r" may not perform any LTO. -- H.J.