public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* LTO question
@ 2010-04-27 16:46 Bingfeng Mei
  2010-04-27 20:38 ` Ian Lance Taylor
  2010-04-28  9:40 ` Richard Guenther
  0 siblings, 2 replies; 23+ messages in thread
From: Bingfeng Mei @ 2010-04-27 16:46 UTC (permalink / raw)
  To: gcc

Hello,
I have been playing with LTO. I notice that LTO doesn't work when
object files are achived into static library files and the final
binary is linked against them, although these object files are compiled
with -flto and I can see all the lto related sections in .a files.
Is this what is described in LTO Wiki page? 

"As an added feature, LTO will take advantage of the plugin feature 
in gold. This allows the compiler to pick up object files that may 
have been stored in library archives. "

So do I have to use gold to solve this issue? 

Many thanks,
Bingfeng 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-27 16:46 LTO question Bingfeng Mei
@ 2010-04-27 20:38 ` Ian Lance Taylor
  2010-04-28  9:40 ` Richard Guenther
  1 sibling, 0 replies; 23+ messages in thread
From: Ian Lance Taylor @ 2010-04-27 20:38 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

"Bingfeng Mei" <bmei@broadcom.com> writes:

> I have been playing with LTO. I notice that LTO doesn't work when
> object files are achived into static library files and the final
> binary is linked against them, although these object files are compiled
> with -flto and I can see all the lto related sections in .a files.
> Is this what is described in LTO Wiki page? 
>
> "As an added feature, LTO will take advantage of the plugin feature 
> in gold. This allows the compiler to pick up object files that may 
> have been stored in library archives. "
>
> So do I have to use gold to solve this issue? 

Yes.

Speaking of which, I'd sure like to see a MIPS port for gold....

Ian

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-27 16:46 LTO question Bingfeng Mei
  2010-04-27 20:38 ` Ian Lance Taylor
@ 2010-04-28  9:40 ` Richard Guenther
  2010-04-28 12:34   ` Bingfeng Mei
  1 sibling, 1 reply; 23+ messages in thread
From: Richard Guenther @ 2010-04-28  9:40 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei <bmei@broadcom.com> wrote:
> Hello,
> I have been playing with LTO. I notice that LTO doesn't work when
> object files are achived into static library files and the final
> binary is linked against them, although these object files are compiled
> with -flto and I can see all the lto related sections in .a files.
> Is this what is described in LTO Wiki page?
>
> "As an added feature, LTO will take advantage of the plugin feature
> in gold. This allows the compiler to pick up object files that may
> have been stored in library archives. "
>
> So do I have to use gold to solve this issue?

Yes.  Or you fix collect2 to do processing of archives and hand
lto1 the required information (it expects archive components
with LTO bytecode like archive.a@offset with offset being the
offset of the .o file with LTO bytecode inside the archive).  See
lto/lto-elf.c:lto_obj_file_open for "details".

Richard.

> Many thanks,
> Bingfeng
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
  2010-04-28  9:40 ` Richard Guenther
@ 2010-04-28 12:34   ` Bingfeng Mei
       [not found]     ` <20100428112054.GA22195@atrey.karlin.mff.cuni.cz>
                       ` (2 more replies)
  0 siblings, 3 replies; 23+ messages in thread
From: Bingfeng Mei @ 2010-04-28 12:34 UTC (permalink / raw)
  To: Richard Guenther; +Cc: gcc

Thanks, I will check what I can do with collect2. LTO
seems to save 6-9% code size for applications I tested
and should be very useful for us.

Bingfeng 

> -----Original Message-----
> From: Richard Guenther [mailto:richard.guenther@gmail.com] 
> Sent: 28 April 2010 10:33
> To: Bingfeng Mei
> Cc: gcc@gcc.gnu.org
> Subject: Re: LTO question
> 
> On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei 
> <bmei@broadcom.com> wrote:
> > Hello,
> > I have been playing with LTO. I notice that LTO doesn't work when
> > object files are achived into static library files and the final
> > binary is linked against them, although these object files 
> are compiled
> > with -flto and I can see all the lto related sections in .a files.
> > Is this what is described in LTO Wiki page?
> >
> > "As an added feature, LTO will take advantage of the plugin feature
> > in gold. This allows the compiler to pick up object files that may
> > have been stored in library archives. "
> >
> > So do I have to use gold to solve this issue?
> 
> Yes.  Or you fix collect2 to do processing of archives and hand
> lto1 the required information (it expects archive components
> with LTO bytecode like archive.a@offset with offset being the
> offset of the .o file with LTO bytecode inside the archive).  See
> lto/lto-elf.c:lto_obj_file_open for "details".
> 
> Richard.
> 
> > Many thanks,
> > Bingfeng
> >
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
       [not found]         ` <20100428125856.GE9094@kam.mff.cuni.cz>
@ 2010-04-28 13:22           ` Bingfeng Mei
  2010-04-28 14:48             ` Manuel López-Ibáñez
  0 siblings, 1 reply; 23+ messages in thread
From: Bingfeng Mei @ 2010-04-28 13:22 UTC (permalink / raw)
  To: Jan Hubicka; +Cc: gcc

I just tried -fwhole-program. It can achieve extra 3%-4% saving.
That is brilliant. Thanks.

Bingfeng 

> -----Original Message-----
> From: Jan Hubicka [mailto:hubicka@ucw.cz] 
> Sent: 28 April 2010 13:59
> To: Bingfeng Mei
> Subject: Re: LTO question
> 
> > Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
> > and it is just for solving scaling issue of large program.(These two
> > options do look similar :-). I shall try next.
> 
> Yep, -fwhopr is not ideal name, but I guess there is not much 
> to do about it.
> With -fwhole-program LTO should be a lot more effective.
> 
> Honza
> > 
> > Bingfeng
> > 
> > > -----Original Message-----
> > > From: Jan Hubicka [mailto:hubicka@ucw.cz] 
> > > Sent: 28 April 2010 12:21
> > > To: Bingfeng Mei
> > > Subject: Re: LTO question
> > > 
> > > > Thanks, I will check what I can do with collect2. LTO
> > > > seems to save 6-9% code size for applications I tested
> > > > and should be very useful for us.
> > > Did you also tested with -fwhole-program? :)
> > > 
> > > Honza
> > > > 
> > > > Bingfeng 
> > > > 
> > > > > -----Original Message-----
> > > > > From: Richard Guenther [mailto:richard.guenther@gmail.com] 
> > > > > Sent: 28 April 2010 10:33
> > > > > To: Bingfeng Mei
> > > > > Cc: gcc@gcc.gnu.org
> > > > > Subject: Re: LTO question
> > > > > 
> > > > > On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei 
> > > > > <bmei@broadcom.com> wrote:
> > > > > > Hello,
> > > > > > I have been playing with LTO. I notice that LTO doesn't 
> > > work when
> > > > > > object files are achived into static library files 
> and the final
> > > > > > binary is linked against them, although these object files 
> > > > > are compiled
> > > > > > with -flto and I can see all the lto related sections 
> > > in .a files.
> > > > > > Is this what is described in LTO Wiki page?
> > > > > >
> > > > > > "As an added feature, LTO will take advantage of the 
> > > plugin feature
> > > > > > in gold. This allows the compiler to pick up object 
> > > files that may
> > > > > > have been stored in library archives. "
> > > > > >
> > > > > > So do I have to use gold to solve this issue?
> > > > > 
> > > > > Yes.  Or you fix collect2 to do processing of 
> archives and hand
> > > > > lto1 the required information (it expects archive components
> > > > > with LTO bytecode like archive.a@offset with offset being the
> > > > > offset of the .o file with LTO bytecode inside the 
> archive).  See
> > > > > lto/lto-elf.c:lto_obj_file_open for "details".
> > > > > 
> > > > > Richard.
> > > > > 
> > > > > > Many thanks,
> > > > > > Bingfeng
> > > > > >
> > > > > 
> > > > > 
> > > 
> > > 
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 12:34   ` Bingfeng Mei
       [not found]     ` <20100428112054.GA22195@atrey.karlin.mff.cuni.cz>
@ 2010-04-28 13:46     ` Dave Korn
  2010-04-28 13:58       ` Bingfeng Mei
  2010-04-29 16:24     ` Xinliang David Li
  2 siblings, 1 reply; 23+ messages in thread
From: Dave Korn @ 2010-04-28 13:46 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: Richard Guenther, gcc

On 28/04/2010 10:44, Bingfeng Mei wrote:
> Thanks, I will check what I can do with collect2. 

  I was also planning to work on this, but won't have any objection if you get
there before me!  We have an open PR about this, would you care to use

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376

to co-ordinate?

    cheers,
      DaveK

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
  2010-04-28 13:46     ` Dave Korn
@ 2010-04-28 13:58       ` Bingfeng Mei
  0 siblings, 0 replies; 23+ messages in thread
From: Bingfeng Mei @ 2010-04-28 13:58 UTC (permalink / raw)
  To: Dave Korn; +Cc: Richard Guenther, gcc

Thanks. I am not very familiar with ELF/lto format etc. But 
making LTO work for our port is definitely in high priority.
I shall see what I can achieve in next weeks. 

Bingfeng

> -----Original Message-----
> From: Dave Korn [mailto:dave.korn.cygwin@googlemail.com] 
> Sent: 28 April 2010 14:59
> To: Bingfeng Mei
> Cc: Richard Guenther; gcc@gcc.gnu.org
> Subject: Re: LTO question
> 
> On 28/04/2010 10:44, Bingfeng Mei wrote:
> > Thanks, I will check what I can do with collect2. 
> 
>   I was also planning to work on this, but won't have any 
> objection if you get
> there before me!  We have an open PR about this, would you care to use
> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
> 
> to co-ordinate?
> 
>     cheers,
>       DaveK
> 
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 13:22           ` Bingfeng Mei
@ 2010-04-28 14:48             ` Manuel López-Ibáñez
  2010-04-28 15:56               ` Diego Novillo
  0 siblings, 1 reply; 23+ messages in thread
From: Manuel López-Ibáñez @ 2010-04-28 14:48 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: Jan Hubicka, gcc

>> > Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
>> > and it is just for solving scaling issue of large program.(These two
>> > options do look similar :-). I shall try next.
>>
>> Yep, -fwhopr is not ideal name, but I guess there is not much
>> to do about it.

It is marked as experimental, so if it is going to stay for GCC 4.6,
then we should change the name. I think one possibility discussed
somewhere is that LTO scales back automatically, so the option would
be not necessary.

Cheers,

Manuel.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 14:48             ` Manuel López-Ibáñez
@ 2010-04-28 15:56               ` Diego Novillo
  2010-04-28 22:22                 ` Jan Hubicka
  0 siblings, 1 reply; 23+ messages in thread
From: Diego Novillo @ 2010-04-28 15:56 UTC (permalink / raw)
  To: Manuel López-Ibáñez; +Cc: Bingfeng Mei, Jan Hubicka, gcc

On 4/28/10 10:26 , Manuel López-Ibáñez wrote:
>>>> Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
>>>> and it is just for solving scaling issue of large program.(These two
>>>> options do look similar :-). I shall try next.
>>>
>>> Yep, -fwhopr is not ideal name, but I guess there is not much
>>> to do about it.
> 
> It is marked as experimental, so if it is going to stay for GCC 4.6,
> then we should change the name. I think one possibility discussed
> somewhere is that LTO scales back automatically, so the option would
> be not necessary.

Yes.  I think we should just keep -flto and make it use split
compilation if needed.  -fwhopr is only needed to explicitly enable it.
 My suggestion is to just keep -flto and invoke whopr with -flto=split
or -flto=big (until the automatic threshold is added).


Diego.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 15:56               ` Diego Novillo
@ 2010-04-28 22:22                 ` Jan Hubicka
  2010-04-29  6:56                   ` Jan Hubicka
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Hubicka @ 2010-04-28 22:22 UTC (permalink / raw)
  To: Diego Novillo
  Cc: Manuel López-Ibá?ez, Bingfeng Mei, Jan Hubicka, gcc

> On 4/28/10 10:26 , Manuel LĂłpez-IbĂĄ?ez wrote:
> >>>> Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
> >>>> and it is just for solving scaling issue of large program.(These two
> >>>> options do look similar :-). I shall try next.
> >>>
> >>> Yep, -fwhopr is not ideal name, but I guess there is not much
> >>> to do about it.
> > 
> > It is marked as experimental, so if it is going to stay for GCC 4.6,
> > then we should change the name. I think one possibility discussed
> > somewhere is that LTO scales back automatically, so the option would
> > be not necessary.
> 
> Yes.  I think we should just keep -flto and make it use split
> compilation if needed.  -fwhopr is only needed to explicitly enable it.
>  My suggestion is to just keep -flto and invoke whopr with -flto=split
> or -flto=big (until the automatic threshold is added).

Yep, I like this idea too.  I hope to be able to drop "experimental" status
from mainline whopr soonish (basically I need to implement references and then
I will burn a lot of time fixing how clones are streamed to enable ipa-cp).

We probably want something like

 -flto for "do whatever you think is best"
       (it will be interesting to unify the paths enough so WPA can decide to be
        LTO at its will after reading in cgraph and concluding on program size)
 -flto=n to force whopr optimized for N parallel coputations (we probably will
       end up with more than N units for sufficiently large programs since we anyway
       can't resonably expect that compile times will be even).
 -flto=incremental to force inremental LTO.

Honza
> 
> 
> Diego.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 22:22                 ` Jan Hubicka
@ 2010-04-29  6:56                   ` Jan Hubicka
  2010-04-29  8:57                     ` Richard Guenther
  0 siblings, 1 reply; 23+ messages in thread
From: Jan Hubicka @ 2010-04-29  6:56 UTC (permalink / raw)
  To: Jan Hubicka
  Cc: Diego Novillo, Manuel López-Ibá?ez, Bingfeng Mei, gcc

> > On 4/28/10 10:26 , Manuel LĂłpez-IbĂĄ?ez wrote:
> > >>>> Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
> > >>>> and it is just for solving scaling issue of large program.(These two
> > >>>> options do look similar :-). I shall try next.
> > >>>
> > >>> Yep, -fwhopr is not ideal name, but I guess there is not much
> > >>> to do about it.
> > > 
> > > It is marked as experimental, so if it is going to stay for GCC 4.6,
> > > then we should change the name. I think one possibility discussed
> > > somewhere is that LTO scales back automatically, so the option would
> > > be not necessary.
> > 
> > Yes.  I think we should just keep -flto and make it use split
> > compilation if needed.  -fwhopr is only needed to explicitly enable it.
> >  My suggestion is to just keep -flto and invoke whopr with -flto=split
> > or -flto=big (until the automatic threshold is added).
> 
> Yep, I like this idea too.  I hope to be able to drop "experimental" status
> from mainline whopr soonish (basically I need to implement references and then
> I will burn a lot of time fixing how clones are streamed to enable ipa-cp).

And do something about paralelizing the whopr build.  I guess it means storing
ltrans partition list into file and making collect2 to execute compilation
and re-invent the Makefile code?
It would be great if someone took look at this, I am not at all familiar with that
code and in a way I would preffer it to stay that way ;))

Honza

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-29  6:56                   ` Jan Hubicka
@ 2010-04-29  8:57                     ` Richard Guenther
  2010-04-29  9:19                       ` Jan Hubicka
  0 siblings, 1 reply; 23+ messages in thread
From: Richard Guenther @ 2010-04-29  8:57 UTC (permalink / raw)
  To: Jan Hubicka
  Cc: Diego Novillo, Manuel López-Ibá?ez, Bingfeng Mei, gcc

2010/4/29 Jan Hubicka <hubicka@ucw.cz>:
>> > On 4/28/10 10:26 , Manuel López-Ibá?ez wrote:
>> > >>>> Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
>> > >>>> and it is just for solving scaling issue of large program.(These two
>> > >>>> options do look similar :-). I shall try next.
>> > >>>
>> > >>> Yep, -fwhopr is not ideal name, but I guess there is not much
>> > >>> to do about it.
>> > >
>> > > It is marked as experimental, so if it is going to stay for GCC 4.6,
>> > > then we should change the name. I think one possibility discussed
>> > > somewhere is that LTO scales back automatically, so the option would
>> > > be not necessary.
>> >
>> > Yes.  I think we should just keep -flto and make it use split
>> > compilation if needed.  -fwhopr is only needed to explicitly enable it.
>> >  My suggestion is to just keep -flto and invoke whopr with -flto=split
>> > or -flto=big (until the automatic threshold is added).
>>
>> Yep, I like this idea too.  I hope to be able to drop "experimental" status
>> from mainline whopr soonish (basically I need to implement references and then
>> I will burn a lot of time fixing how clones are streamed to enable ipa-cp).
>
> And do something about paralelizing the whopr build.  I guess it means storing
> ltrans partition list into file and making collect2 to execute compilation
> and re-invent the Makefile code?
> It would be great if someone took look at this, I am not at all familiar with that
> code and in a way I would preffer it to stay that way ;))

I will look at moving the LTRANS driving to the driver, it should be
easy to do parallel execs from it and hopefully make debugging
WPA/LTRANS less of a headache.

Richard.

> Honza
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-29  8:57                     ` Richard Guenther
@ 2010-04-29  9:19                       ` Jan Hubicka
  0 siblings, 0 replies; 23+ messages in thread
From: Jan Hubicka @ 2010-04-29  9:19 UTC (permalink / raw)
  To: Richard Guenther
  Cc: Jan Hubicka, Diego Novillo, Manuel López-Ibá?ez,
	Bingfeng Mei, gcc

> 2010/4/29 Jan Hubicka <hubicka@ucw.cz>:
> >> > On 4/28/10 10:26 , Manuel LĂłpez-IbĂĄ?ez wrote:
> >> > >>>> Not yet, I mistakenly thought -fwhole-program is the same as -fwhopr
> >> > >>>> and it is just for solving scaling issue of large program.(These two
> >> > >>>> options do look similar :-). I shall try next.
> >> > >>>
> >> > >>> Yep, -fwhopr is not ideal name, but I guess there is not much
> >> > >>> to do about it.
> >> > >
> >> > > It is marked as experimental, so if it is going to stay for GCC 4.6,
> >> > > then we should change the name. I think one possibility discussed
> >> > > somewhere is that LTO scales back automatically, so the option would
> >> > > be not necessary.
> >> >
> >> > Yes.  I think we should just keep -flto and make it use split
> >> > compilation if needed.  -fwhopr is only needed to explicitly enable it.
> >> >  My suggestion is to just keep -flto and invoke whopr with -flto=split
> >> > or -flto=big (until the automatic threshold is added).
> >>
> >> Yep, I like this idea too.  I hope to be able to drop "experimental" status
> >> from mainline whopr soonish (basically I need to implement references and then
> >> I will burn a lot of time fixing how clones are streamed to enable ipa-cp).
> >
> > And do something about paralelizing the whopr build.  I guess it means storing
> > ltrans partition list into file and making collect2 to execute compilation
> > and re-invent the Makefile code?
> > It would be great if someone took look at this, I am not at all familiar with that
> > code and in a way I would preffer it to stay that way ;))
> 
> I will look at moving the LTRANS driving to the driver, it should be
> easy to do parallel execs from it and hopefully make debugging
> WPA/LTRANS less of a headache.

That would be great, thanks!
It would be also great to get the parallel build working, but I guess it can be done
inrementally.

One problem is that we output .o files via assembly.  We produce a lot of temporary
data and producing all temporary .s files and processing them alter from collect
will increase memory use etc...
So probably doing as from wpa itself is sort of needed to avoid this bottleneck.

Honza
> 
> Richard.
> 
> > Honza
> >

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-28 12:34   ` Bingfeng Mei
       [not found]     ` <20100428112054.GA22195@atrey.karlin.mff.cuni.cz>
  2010-04-28 13:46     ` Dave Korn
@ 2010-04-29 16:24     ` Xinliang David Li
  2010-04-29 16:41       ` Bingfeng Mei
  2 siblings, 1 reply; 23+ messages in thread
From: Xinliang David Li @ 2010-04-29 16:24 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: Richard Guenther, gcc

Just curious, what is the base line size of your comparison? Did you
turn on GC (-ffunction-sections -fdata-sections -Wl,--gc-sections)?

David

On Wed, Apr 28, 2010 at 2:44 AM, Bingfeng Mei <bmei@broadcom.com> wrote:
> Thanks, I will check what I can do with collect2. LTO
> seems to save 6-9% code size for applications I tested
> and should be very useful for us.
>
> Bingfeng
>
>> -----Original Message-----
>> From: Richard Guenther [mailto:richard.guenther@gmail.com]
>> Sent: 28 April 2010 10:33
>> To: Bingfeng Mei
>> Cc: gcc@gcc.gnu.org
>> Subject: Re: LTO question
>>
>> On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei
>> <bmei@broadcom.com> wrote:
>> > Hello,
>> > I have been playing with LTO. I notice that LTO doesn't work when
>> > object files are achived into static library files and the final
>> > binary is linked against them, although these object files
>> are compiled
>> > with -flto and I can see all the lto related sections in .a files.
>> > Is this what is described in LTO Wiki page?
>> >
>> > "As an added feature, LTO will take advantage of the plugin feature
>> > in gold. This allows the compiler to pick up object files that may
>> > have been stored in library archives. "
>> >
>> > So do I have to use gold to solve this issue?
>>
>> Yes.  Or you fix collect2 to do processing of archives and hand
>> lto1 the required information (it expects archive components
>> with LTO bytecode like archive.a@offset with offset being the
>> offset of the .o file with LTO bytecode inside the archive).  See
>> lto/lto-elf.c:lto_obj_file_open for "details".
>>
>> Richard.
>>
>> > Many thanks,
>> > Bingfeng
>> >
>>
>>
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
  2010-04-29 16:24     ` Xinliang David Li
@ 2010-04-29 16:41       ` Bingfeng Mei
  2010-04-29 16:42         ` Xinliang David Li
  0 siblings, 1 reply; 23+ messages in thread
From: Bingfeng Mei @ 2010-04-29 16:41 UTC (permalink / raw)
  To: Xinliang David Li; +Cc: Richard Guenther, gcc

I turned on -ffunction-sections and compiled with -Os. 
The size gain at -O2 is less though.

Bingfeng

> -----Original Message-----
> From: Xinliang David Li [mailto:davidxl@google.com] 
> Sent: 29 April 2010 17:17
> To: Bingfeng Mei
> Cc: Richard Guenther; gcc@gcc.gnu.org
> Subject: Re: LTO question
> 
> Just curious, what is the base line size of your comparison? Did you
> turn on GC (-ffunction-sections -fdata-sections -Wl,--gc-sections)?
> 
> David
> 
> On Wed, Apr 28, 2010 at 2:44 AM, Bingfeng Mei 
> <bmei@broadcom.com> wrote:
> > Thanks, I will check what I can do with collect2. LTO
> > seems to save 6-9% code size for applications I tested
> > and should be very useful for us.
> >
> > Bingfeng
> >
> >> -----Original Message-----
> >> From: Richard Guenther [mailto:richard.guenther@gmail.com]
> >> Sent: 28 April 2010 10:33
> >> To: Bingfeng Mei
> >> Cc: gcc@gcc.gnu.org
> >> Subject: Re: LTO question
> >>
> >> On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei
> >> <bmei@broadcom.com> wrote:
> >> > Hello,
> >> > I have been playing with LTO. I notice that LTO doesn't work when
> >> > object files are achived into static library files and the final
> >> > binary is linked against them, although these object files
> >> are compiled
> >> > with -flto and I can see all the lto related sections in 
> .a files.
> >> > Is this what is described in LTO Wiki page?
> >> >
> >> > "As an added feature, LTO will take advantage of the 
> plugin feature
> >> > in gold. This allows the compiler to pick up object 
> files that may
> >> > have been stored in library archives. "
> >> >
> >> > So do I have to use gold to solve this issue?
> >>
> >> Yes.  Or you fix collect2 to do processing of archives and hand
> >> lto1 the required information (it expects archive components
> >> with LTO bytecode like archive.a@offset with offset being the
> >> offset of the .o file with LTO bytecode inside the archive).  See
> >> lto/lto-elf.c:lto_obj_file_open for "details".
> >>
> >> Richard.
> >>
> >> > Many thanks,
> >> > Bingfeng
> >> >
> >>
> >>
> >
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2010-04-29 16:41       ` Bingfeng Mei
@ 2010-04-29 16:42         ` Xinliang David Li
  0 siblings, 0 replies; 23+ messages in thread
From: Xinliang David Li @ 2010-04-29 16:42 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: Richard Guenther, gcc

On Thu, Apr 29, 2010 at 9:28 AM, Bingfeng Mei <bmei@broadcom.com> wrote:
> I turned on -ffunction-sections and compiled with -Os.
> The size gain at -O2 is less though.

Interesting.

Thanks,

David
>
> Bingfeng
>
>> -----Original Message-----
>> From: Xinliang David Li [mailto:davidxl@google.com]
>> Sent: 29 April 2010 17:17
>> To: Bingfeng Mei
>> Cc: Richard Guenther; gcc@gcc.gnu.org
>> Subject: Re: LTO question
>>
>> Just curious, what is the base line size of your comparison? Did you
>> turn on GC (-ffunction-sections -fdata-sections -Wl,--gc-sections)?
>>
>> David
>>
>> On Wed, Apr 28, 2010 at 2:44 AM, Bingfeng Mei
>> <bmei@broadcom.com> wrote:
>> > Thanks, I will check what I can do with collect2. LTO
>> > seems to save 6-9% code size for applications I tested
>> > and should be very useful for us.
>> >
>> > Bingfeng
>> >
>> >> -----Original Message-----
>> >> From: Richard Guenther [mailto:richard.guenther@gmail.com]
>> >> Sent: 28 April 2010 10:33
>> >> To: Bingfeng Mei
>> >> Cc: gcc@gcc.gnu.org
>> >> Subject: Re: LTO question
>> >>
>> >> On Tue, Apr 27, 2010 at 6:30 PM, Bingfeng Mei
>> >> <bmei@broadcom.com> wrote:
>> >> > Hello,
>> >> > I have been playing with LTO. I notice that LTO doesn't work when
>> >> > object files are achived into static library files and the final
>> >> > binary is linked against them, although these object files
>> >> are compiled
>> >> > with -flto and I can see all the lto related sections in
>> .a files.
>> >> > Is this what is described in LTO Wiki page?
>> >> >
>> >> > "As an added feature, LTO will take advantage of the
>> plugin feature
>> >> > in gold. This allows the compiler to pick up object
>> files that may
>> >> > have been stored in library archives. "
>> >> >
>> >> > So do I have to use gold to solve this issue?
>> >>
>> >> Yes.  Or you fix collect2 to do processing of archives and hand
>> >> lto1 the required information (it expects archive components
>> >> with LTO bytecode like archive.a@offset with offset being the
>> >> offset of the .o file with LTO bytecode inside the archive).  See
>> >> lto/lto-elf.c:lto_obj_file_open for "details".
>> >>
>> >> Richard.
>> >>
>> >> > Many thanks,
>> >> > Bingfeng
>> >> >
>> >>
>> >>
>> >
>>
>>
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2009-10-13 15:19   ` Bingfeng Mei
@ 2009-10-13 16:49     ` Richard Guenther
  0 siblings, 0 replies; 23+ messages in thread
From: Richard Guenther @ 2009-10-13 16:49 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

On Tue, Oct 13, 2009 at 5:18 PM, Bingfeng Mei <bmei@broadcom.com> wrote:
>
>
>> -----Original Message-----
>> From: Richard Guenther [mailto:richard.guenther@gmail.com]
>> Sent: 13 October 2009 16:15
>> To: Bingfeng Mei
>> Cc: gcc@gcc.gnu.org
>> Subject: Re: LTO question
>>
>> On Tue, Oct 13, 2009 at 2:47 PM, Bingfeng Mei
>> <bmei@broadcom.com> wrote:
>> > Hello,
>> > I just had the first taste with the latest LTO merge on our port.
>> > Compiler is configured with LTO enabled and built correctly.
>> > I tried the following example:
>> >
>> > a.c
>> > extern void foo(int);
>> > int main()
>> > {  foo(20);
>> >  return 1;
>> > }
>> >
>> > b.c
>> > #include <stdio.h>
>> > void foo(int c)
>> > {
>> >  printf("Hello world: %d\n", c);
>> > }
>> >
>> > compiled with:
>> > firepath-elf-gcc -flto a.c b.c -save-temps -O2
>> >
>> > I expected that foo should be inlined into main, but not.
>> Both functions
>> >  of main and foo are present in a.s, while b.s contains the
>> LTO code.
>> >
>> > Did I miss something here? Are there new hooks I should
>> specify or tune for
>> > LTO? I checked the up-to-date internal manual and found nothing.
>>
>> non-inline declared functions are inlined at -O2 only if doing so
>> does not increase program size.  Use -O3 or -finline-functions.
>
> But the function is only called once here. It should always decrease size unless
> my cost function is terribly wrong.  I will check how other targets such as
> x86 do here.

The function is still exported if you do not use -fwhole-program.

Richard.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
  2009-10-13 15:19 ` Richard Guenther
@ 2009-10-13 15:19   ` Bingfeng Mei
  2009-10-13 16:49     ` Richard Guenther
  0 siblings, 1 reply; 23+ messages in thread
From: Bingfeng Mei @ 2009-10-13 15:19 UTC (permalink / raw)
  To: Richard Guenther; +Cc: gcc

 

> -----Original Message-----
> From: Richard Guenther [mailto:richard.guenther@gmail.com] 
> Sent: 13 October 2009 16:15
> To: Bingfeng Mei
> Cc: gcc@gcc.gnu.org
> Subject: Re: LTO question
> 
> On Tue, Oct 13, 2009 at 2:47 PM, Bingfeng Mei 
> <bmei@broadcom.com> wrote:
> > Hello,
> > I just had the first taste with the latest LTO merge on our port.
> > Compiler is configured with LTO enabled and built correctly.
> > I tried the following example:
> >
> > a.c
> > extern void foo(int);
> > int main()
> > {  foo(20);
> >  return 1;
> > }
> >
> > b.c
> > #include <stdio.h>
> > void foo(int c)
> > {
> >  printf("Hello world: %d\n", c);
> > }
> >
> > compiled with:
> > firepath-elf-gcc -flto a.c b.c -save-temps -O2
> >
> > I expected that foo should be inlined into main, but not.  
> Both functions
> >  of main and foo are present in a.s, while b.s contains the 
> LTO code.
> >
> > Did I miss something here? Are there new hooks I should 
> specify or tune for
> > LTO? I checked the up-to-date internal manual and found nothing.
> 
> non-inline declared functions are inlined at -O2 only if doing so
> does not increase program size.  Use -O3 or -finline-functions.

But the function is only called once here. It should always decrease size unless
my cost function is terribly wrong.  I will check how other targets such as 
x86 do here. 


> 
> Richard.
> 
> > Thanks,
> > Bingfeng Mei
> >
> >
> >
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2009-10-13 12:51 Bingfeng Mei
  2009-10-13 13:47 ` Diego Novillo
@ 2009-10-13 15:19 ` Richard Guenther
  2009-10-13 15:19   ` Bingfeng Mei
  1 sibling, 1 reply; 23+ messages in thread
From: Richard Guenther @ 2009-10-13 15:19 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

On Tue, Oct 13, 2009 at 2:47 PM, Bingfeng Mei <bmei@broadcom.com> wrote:
> Hello,
> I just had the first taste with the latest LTO merge on our port.
> Compiler is configured with LTO enabled and built correctly.
> I tried the following example:
>
> a.c
> extern void foo(int);
> int main()
> {  foo(20);
>  return 1;
> }
>
> b.c
> #include <stdio.h>
> void foo(int c)
> {
>  printf("Hello world: %d\n", c);
> }
>
> compiled with:
> firepath-elf-gcc -flto a.c b.c -save-temps -O2
>
> I expected that foo should be inlined into main, but not.  Both functions
>  of main and foo are present in a.s, while b.s contains the LTO code.
>
> Did I miss something here? Are there new hooks I should specify or tune for
> LTO? I checked the up-to-date internal manual and found nothing.

non-inline declared functions are inlined at -O2 only if doing so
does not increase program size.  Use -O3 or -finline-functions.

Richard.

> Thanks,
> Bingfeng Mei
>
>
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2009-10-13 13:47   ` Bingfeng Mei
@ 2009-10-13 14:53     ` Diego Novillo
  0 siblings, 0 replies; 23+ messages in thread
From: Diego Novillo @ 2009-10-13 14:53 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

On Tue, Oct 13, 2009 at 09:46, Bingfeng Mei <bmei@broadcom.com> wrote:

> Thanks. It works. I thought -fwhole-program was used with --combine and they are replaced
> by -flto. Now it seems that -flto is equivalent of --combine, and -fwhole-program is still
> important.

That's approximately correct, yes.  --combine will likely be
deprecated soon in favour of -flto.  The -fwhole-program flag simply
allows the inter-procedural optimizer to make more aggressive
assumptions.


Diego.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: LTO question
  2009-10-13 13:47 ` Diego Novillo
@ 2009-10-13 13:47   ` Bingfeng Mei
  2009-10-13 14:53     ` Diego Novillo
  0 siblings, 1 reply; 23+ messages in thread
From: Bingfeng Mei @ 2009-10-13 13:47 UTC (permalink / raw)
  To: Diego Novillo; +Cc: gcc

Thanks. It works. I thought -fwhole-program was used with --combine and they are replaced
by -flto. Now it seems that -flto is equivalent of --combine, and -fwhole-program is still
important. 

Bingfeng  

> -----Original Message-----
> From: Diego Novillo [mailto:dnovillo@google.com] 
> Sent: 13 October 2009 14:30
> To: Bingfeng Mei
> Cc: gcc@gcc.gnu.org
> Subject: Re: LTO question
> 
> On Tue, Oct 13, 2009 at 08:47, Bingfeng Mei <bmei@broadcom.com> wrote:
> 
> > a.c
> > extern void foo(int);
> > int main()
> > {  foo(20);
> >  return 1;
> > }
> >
> > b.c
> > #include <stdio.h>
> > void foo(int c)
> > {
> >  printf("Hello world: %d\n", c);
> > }
> >
> > compiled with:
> > firepath-elf-gcc -flto a.c b.c -save-temps -O2
> >
> > I expected that foo should be inlined into main, but not.  
> Both functions
> >  of main and foo are present in a.s, while b.s contains the 
> LTO code.
> 
> Try adding -fwhole-program.
> 
> 
> Diego.
> 
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: LTO question
  2009-10-13 12:51 Bingfeng Mei
@ 2009-10-13 13:47 ` Diego Novillo
  2009-10-13 13:47   ` Bingfeng Mei
  2009-10-13 15:19 ` Richard Guenther
  1 sibling, 1 reply; 23+ messages in thread
From: Diego Novillo @ 2009-10-13 13:47 UTC (permalink / raw)
  To: Bingfeng Mei; +Cc: gcc

On Tue, Oct 13, 2009 at 08:47, Bingfeng Mei <bmei@broadcom.com> wrote:

> a.c
> extern void foo(int);
> int main()
> {  foo(20);
>  return 1;
> }
>
> b.c
> #include <stdio.h>
> void foo(int c)
> {
>  printf("Hello world: %d\n", c);
> }
>
> compiled with:
> firepath-elf-gcc -flto a.c b.c -save-temps -O2
>
> I expected that foo should be inlined into main, but not.  Both functions
>  of main and foo are present in a.s, while b.s contains the LTO code.

Try adding -fwhole-program.


Diego.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* LTO question
@ 2009-10-13 12:51 Bingfeng Mei
  2009-10-13 13:47 ` Diego Novillo
  2009-10-13 15:19 ` Richard Guenther
  0 siblings, 2 replies; 23+ messages in thread
From: Bingfeng Mei @ 2009-10-13 12:51 UTC (permalink / raw)
  To: gcc

Hello,
I just had the first taste with the latest LTO merge on our port.  
Compiler is configured with LTO enabled and built correctly. 
I tried the following example: 
 
a.c
extern void foo(int);
int main()
{  foo(20);
  return 1;
}  

b.c
#include <stdio.h>
void foo(int c)
{
  printf("Hello world: %d\n", c);
}

compiled with:
firepath-elf-gcc -flto a.c b.c -save-temps -O2
 
I expected that foo should be inlined into main, but not.  Both functions
 of main and foo are present in a.s, while b.s contains the LTO code. 
 
Did I miss something here? Are there new hooks I should specify or tune for 
LTO? I checked the up-to-date internal manual and found nothing.
 
Thanks,
Bingfeng Mei
 

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2010-04-29 16:41 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-04-27 16:46 LTO question Bingfeng Mei
2010-04-27 20:38 ` Ian Lance Taylor
2010-04-28  9:40 ` Richard Guenther
2010-04-28 12:34   ` Bingfeng Mei
     [not found]     ` <20100428112054.GA22195@atrey.karlin.mff.cuni.cz>
     [not found]       ` <7FB04A5C213E9943A72EE127DB74F0ADA6654A8AB1@SJEXCHCCR02.corp.ad.broadcom.com>
     [not found]         ` <20100428125856.GE9094@kam.mff.cuni.cz>
2010-04-28 13:22           ` Bingfeng Mei
2010-04-28 14:48             ` Manuel López-Ibáñez
2010-04-28 15:56               ` Diego Novillo
2010-04-28 22:22                 ` Jan Hubicka
2010-04-29  6:56                   ` Jan Hubicka
2010-04-29  8:57                     ` Richard Guenther
2010-04-29  9:19                       ` Jan Hubicka
2010-04-28 13:46     ` Dave Korn
2010-04-28 13:58       ` Bingfeng Mei
2010-04-29 16:24     ` Xinliang David Li
2010-04-29 16:41       ` Bingfeng Mei
2010-04-29 16:42         ` Xinliang David Li
  -- strict thread matches above, loose matches on Subject: below --
2009-10-13 12:51 Bingfeng Mei
2009-10-13 13:47 ` Diego Novillo
2009-10-13 13:47   ` Bingfeng Mei
2009-10-13 14:53     ` Diego Novillo
2009-10-13 15:19 ` Richard Guenther
2009-10-13 15:19   ` Bingfeng Mei
2009-10-13 16:49     ` Richard Guenther

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).