Re: How to successfully use autoLIPO on GCC？

Re: How to successfully use autoLIPO on GCC？

[Teresa Johnson]

+davidxl who might remember the necessary options.

It's been a good 3-4 years since we've worked on or used gcc and LIPO, so
my memory is pretty stale on the usage model. I never myself worked on the
autoFDO tool, so I don't know the options there. A few comments/responses
below.

On Fri, Feb 19, 2021 at 7:55 PM ijinma <ijinma@yeah.net> wrote:

> hi, Johnson
>
>    I happened to see your patches(
> https://chromium.googlesource.com/chromiumos/third_party/gcc/+/2eee6942394ea46458d4412d613623fac806247d)
>  about autoLIPO LIPO in Google GCC. Recently, I was studying autoLIPO(
> autoFDO + lipo), but unfortunately, I encountered some problems, so I
> want to seek your help. Although the llvm thinLTO has a great advantage, I
> still want to try autoLIPO on GCC. I read the source code of the autoFDO
> tool briefly, but unfortunately I didn't find a way to enable autoLIPO.
>
>    I want to recreate autoLIPO work, and I think it's worth continuing on
> gcc. I think there should be a proper way to turn on autoLIPO, though it
> might be very old. But I'm a beginner in the compiler direction, and I'm
> not good at all of this.
>
>    So could you give me some help?
>
>    1. GCC(google gcc_4.8 ?)(
> https://chromium.googlesource.com/chromiumos/third_party/gcc/+/refs/heads/svn-mirror/google/gcc-4_8)
> and autoFDO(https://github.com/google/autofdo) versions that have been
> successfully run by autoLIPO, or some suggestions.
>

Whatever is the most recent google gcc branch would support autoFDO. I
believe that google/gcc_4.8 branch is it. Not sure about the autoFDO tool
though, David might know.

>
>    2. The usage or description of autoFDO enabling auxiliary module. That
> is, the imports file generated by AutoFDO stores the grouping algorithm
> result of the LIPO module.
>

I'm not personally familiar with usage of the autoFDO tool, but again,
David might remember.


>
>    3. Or what would I need to do if I wanted to successfully use autoLIPO
> on GCC?
>

If you want to use it with a more recent gcc I'm afraid that is probably
going to be a lot of work. I remember each time we synced our google branch
with a more recent mainline it required months of work to iron out all of
the LIPO interactions, since that wasn't supported by the main gcc branch.

Teresa


>    Looking forward to your reply!
>
>   thanks，
>   ijinma
>


-- 
Teresa Johnson |  Software Engineer |  tejohnson@google.com |

Re: How to successfully use autoLIPO on GCC？

[Xinliang David Li]

For AutoLIPO to work, there is going to be some build tool changes. The
reason is that AutoFDO does not have the training run step and therefore
can not produce dynamic callgraph and module grouping directly. In other
words, it needs a two pass build -- so build tools need to changed for
autoLIPO.
 We use blaze internally, and it is changed to do the two pass build -- the
first pass produces module grouping info which is integrated with the
autoFDO profile, and the second pass turns on LIPO mode.

You will need to do similar things for whatever build system you are using.

Hopefully this is helpful.

David


On Sat, Feb 20, 2021 at 8:31 AM Teresa Johnson <tejohnson@google.com> wrote:

> +davidxl who might remember the necessary options.
>
> It's been a good 3-4 years since we've worked on or used gcc and LIPO, so
> my memory is pretty stale on the usage model. I never myself worked on the
> autoFDO tool, so I don't know the options there. A few comments/responses
> below.
>
> On Fri, Feb 19, 2021 at 7:55 PM ijinma <ijinma@yeah.net> wrote:
>
>> hi, Johnson
>>
>>    I happened to see your patches(
>> https://chromium.googlesource.com/chromiumos/third_party/gcc/+/2eee6942394ea46458d4412d613623fac806247d)
>>  about autoLIPO LIPO in Google GCC. Recently, I was studying autoLIPO(
>> autoFDO + lipo), but unfortunately, I encountered some problems, so I
>> want to seek your help. Although the llvm thinLTO has a great advantage, I
>> still want to try autoLIPO on GCC. I read the source code of the autoFDO
>> tool briefly, but unfortunately I didn't find a way to enable autoLIPO.
>>
>>    I want to recreate autoLIPO work, and I think it's worth continuing
>> on gcc. I think there should be a proper way to turn on autoLIPO, though it
>> might be very old. But I'm a beginner in the compiler direction, and I'm
>> not good at all of this.
>>
>>    So could you give me some help?
>>
>>    1. GCC(google gcc_4.8 ?)(
>> https://chromium.googlesource.com/chromiumos/third_party/gcc/+/refs/heads/svn-mirror/google/gcc-4_8)
>> and autoFDO(https://github.com/google/autofdo) versions that have been
>> successfully run by autoLIPO, or some suggestions.
>>
>
> Whatever is the most recent google gcc branch would support autoFDO. I
> believe that google/gcc_4.8 branch is it. Not sure about the autoFDO tool
> though, David might know.
>
>>
>>    2. The usage or description of autoFDO enabling auxiliary module. That
>> is, the imports file generated by AutoFDO stores the grouping algorithm
>> result of the LIPO module.
>>
>
> I'm not personally familiar with usage of the autoFDO tool, but again,
> David might remember.
>
>
>>
>>    3. Or what would I need to do if I wanted to successfully use autoLIPO
>> on GCC?
>>
>
> If you want to use it with a more recent gcc I'm afraid that is probably
> going to be a lot of work. I remember each time we synced our google branch
> with a more recent mainline it required months of work to iron out all of
> the LIPO interactions, since that wasn't supported by the main gcc branch.
>
> Teresa
>
>
>>    Looking forward to your reply!
>>
>>   thanks，
>>   ijinma
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson@google.com |
>