Re: State of AutoFDO in GCC

[Wei Mi <wmi@google.com>]

On Sun, Apr 25, 2021 at 4:18 PM Xinliang David Li <davidxl@google.com>
wrote:

>
>
> On Sun, Apr 25, 2021 at 12:07 PM Jan Hubicka <hubicka@ucw.cz> wrote:
>
>> David,
>> >
>> > The text format is documented here:
>> > https://clang.llvm.org/docs/UsersManual.html
>> > The binary format is not documented. The binary format is not
>> guaranteed to
>> > be backward compatible, so sharing the same format may not be the best
>> way
>> > as changes for clang may break GCC.
>> >
>> > Since linux perf format does not change, the tool should be relatively
>> > stable with low maintenance cost. Changes are needed only when some new
>> > AutoFDO features are added to the compiler side.
>>
>> I was under impression that it is indeed problem with the tool requiring
>> old format of linux perf. At least with opensuse distro the shipped tool
>> fails for me:
>> jan@skylake:~> create_llvm_prof --binary=./code --out=code.prof
>> E0425 21:01:55.038128 17977 perf_reader.cc:996] Unsupported event type
>> 79
>> F0425 21:01:55.038295 17977 perf_parser.cc:240] Check failed:
>> reader_.ReadPerfSampleInfo(*parsed_event.raw_event, &sample_info)
>> *** Check failure stack trace: ***
>>     @     0x55e6deb6058e  (unknown)
>>     @     0x55e6deb94a49  (unknown)
>>     ..
>>     Aborted (core dumped)
>>
>> I collect data as intstructed here:
>> https://clang.llvm.org/docs/UsersManual.html
>>
>> It is from package autofdo-0.18-4.4.x86_64 and perf 5.11.15.
>>
>> Is there a way to get this working w/o using older perf?
>> Honza
>> >
>
>
> Interesting. That means we will also see the same error when using the
> latest perf.
>
> Wei, are you aware of the issue?
>
> David
>
>
>
My local perf is 4.13 so I cannot look at the problem. I remember we has
fixed such problem in quipper before. Could you try the latest version of
autofdo (using the guideline here: https://github.com/google/autofdo#readme)?
autofdo-0.18 is an old version and it is using an old quipper.

Thanks,
Wei.



>
>
>
>>
>> > Does LLVM's auto-FDO support non-Intel CPUs these days?
>> > >
>> >
>> > It supports LBR like events, so it is CPU vendor dependent. For ARM,
>> using
>> > ETM can achieve the goal, but I don't have detailed knowledge of it.
>> >
>> > David
>> >
>> > >
>> > > Honza
>> > > >
>> > > > David
>> > > > >
>> > > > >
>> > > > >> Honza
>> > > > >> >
>> > > > >> > David
>> > > > >> >
>> > > > >> >
>> > > > >> > > Honza
>> > > > >> > > >
>> > > > >> > > > David
>> > > > >> > > >
>> > > > >> > > > >
>> > > > >> > > > > Thoughts?
>> > > > >> > > > > Martin
>> > > > >> > > > >
>> > > > >> > > > > >
>> > > > >> > > > > > Having the tool third-party makes keeping the whole
>> chain
>> > > > >> working
>> > > > >> > > more
>> > > > >> > > > > > difficult.
>> > > > >> > > > > >
>> > > > >> > > > > > Richard.
>> > > > >> > > > > >
>> > > > >> > > > > >> Thanks,
>> > > > >> > > > > >>
>> > > > >> > > > > >> David
>> > > > >> > > > > >>
>> > > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka <
>> > > hubicka@ucw.cz>
>> > > > >> wrote:
>> > > > >> > > > > >>
>> > > > >> > > > > >>>> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote:
>> > > > >> > > > > >>>>> GCC documentation for AutoFDO points to
>> create_gcov tool
>> > > > >> that
>> > > > >> > > > > converts
>> > > > >> > > > > >>> perf.data file into gcov format that can be consumed
>> by
>> > > gcc
>> > > > >> with
>> > > > >> > > > > >>> -fauto-profile (
>> > > > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
>> ,
>> > > > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial).
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> I noticed that the source code for create_gcov has
>> been
>> > > > >> deleted
>> > > > >> > > from
>> > > > >> > > > > >>> https://github.com/google/autofdo on April 7. I
>> asked
>> > > about
>> > > > >> that
>> > > > >> > > > > change
>> > > > >> > > > > >>> in that repo and got the following reply:
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>>
>> > > > >> > >
>> https://github.com/google/autofdo/pull/107#issuecomment-819108738
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> "Actually we didn't use create_gcov and havn't
>> updated
>> > > > >> > > create_gcov
>> > > > >> > > > > for
>> > > > >> > > > > >>> years, and we also didn't have enough tests to
>> guarantee
>> > > it
>> > > > >> works
>> > > > >> > > (It
>> > > > >> > > > > was
>> > > > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If
>> you
>> > > need
>> > > > >> it, it
>> > > > >> > > is
>> > > > >> > > > > >>> welcomed to update create_gcov and add it to the
>> > > respository."
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> Does this mean that AutoFDO is currently dead in
>> gcc?
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Hello.
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Yes. I know that even basic test cases have been
>> broken
>> > > for
>> > > > >> years
>> > > > >> > > in
>> > > > >> > > > > the
>> > > > >> > > > > >>> GCC.
>> > > > >> > > > > >>>> It's new to me that create_gcov was removed.
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> I tend to send patch to GCC that will remove
>> AutoFDO from
>> > > > >> GCC.
>> > > > >> > > > > >>>> I known Bin spent some time working on AutoFDO, has
>> he
>> > > came
>> > > > >> up to
>> > > > >> > > > > >>> something?
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> The GCC side of auto-FDO is not that hard.  We have
>> most
>> > > of
>> > > > >> > > > > >>> infrastructure in place, but stopping point for me
>> was
>> > > always
>> > > > >> > > > > difficulty
>> > > > >> > > > > >>> to get gcov-tool working.  If some maintainer steps
>> up, I
>> > > > >> think I
>> > > > >> > > can
>> > > > >> > > > > >>> fix GCC side.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> I am bit unsure how important feature it is - we
>> have FDO
>> > > that
>> > > > >> > > works
>> > > > >> > > > > >>> quite well for most users but I know there are some
>> users
>> > > of
>> > > > >> the
>> > > > >> > > LLVM
>> > > > >> > > > > >>> implementation and there is potential to tie this
>> with
>> > > other
>> > > > >> > > hardware
>> > > > >> > > > > >>> events to asist i.e. if conversion (where one wants
>> to
>> > > know
>> > > > >> how
>> > > > >> > > well
>> > > > >> > > > > CPU
>> > > > >> > > > > >>> predicts the jump rather than just the jump
>> probability)
>> > > > >> which I
>> > > > >> > > always
>> > > > >> > > > > >>> found potentially interesting.
>> > > > >> > > > > >>>
>> > > > >> > > > > >>> Honza
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>> Martin
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> Thanks,
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>> Eugene
>> > > > >> > > > > >>>>>
>> > > > >> > > > > >>>>
>> > > > >> > > > > >>>
>> > > > >> > > > >
>> > > > >> > > > >
>> > > > >> > >
>> > > > >>
>> > > > >
>> > >
>>
>