RE: [EXTERNAL] Re: State of AutoFDO in GCC

[Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>]

Is the format produced by create_gcov and expected by GCC under -fauto-rpofile documented somewhere? How is it different from .gcda used in FDO, e.g., as described here: http://src.gnu-darwin.org/src/contrib/gcc/gcov-io.h.html?
My input data is different from perf.data and I'd like to write a tool that produces the format needed for AutoFDO.

I would prefer that AutoFDO is not removed from GCC and it would be helpful if create_gcov were restored in google/autofdo. I checked out a revision before the recent merge and tried it on a simple example and it seems to work.
I'm also interested in contributing improvements for AutoFDO so will try to investigate https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71672 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81379

Thanks,

Eugene

From: Xinliang David Li <davidxl@google.com>
Sent: Friday, April 23, 2021 9:36 AM
To: Richard Biener <richard.guenther@gmail.com>
Cc: Jan Hubicka <hubicka@ucw.cz>; gcc@gcc.gnu.org; Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
Subject: [EXTERNAL] Re: State of AutoFDO in GCC



On Fri, Apr 23, 2021 at 12:00 AM Richard Biener <richard.guenther@gmail.com<mailto:richard.guenther@gmail.com>> wrote:
On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc
<gcc@gcc.gnu.org<mailto:gcc@gcc.gnu.org>> wrote:
>
> Hi, the create_gcov tool was probably removed with the assumption that it
> was only used with Google GCC branch, but it is actually used with GCC
> trunk as well.
>
> Given that, the tool will be restored in the github repo. It seems to build
> and work fine with the regression test.
>
> The tool may ust work as it is right now, but there is no guarantee it
> won't break in the future unless someone in the GCC community tries to
> maintain it.

I think if we want to keep the feature it makes sense to provide create_gcov
functionality either directly from perf (input data producer) or from gcc
(data consumer).  Of course I have no idea about its complexity, license
or implementation language ...

Right. What it takes is a perf data (can be text format) parser to produce the format GCC needs, but someone in the community needs to take the lead. It should not involve too much effort.

David

Having the tool third-party makes keeping the whole chain working more
difficult.

Richard.

> Thanks,
>
> David
>
> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka <hubicka@ucw.cz<mailto:hubicka@ucw.cz>> wrote:
>
> > > On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote:
> > > > GCC documentation for AutoFDO points to create_gcov tool that converts
> > perf.data file into gcov format that can be consumed by gcc with
> > -fauto-profile (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fonlinedocs%2Fgcc%2FOptimize-Options.html&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C08a2bfb9135c41955a7708d90675e9ba%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637547925772845557%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FAYvyG0%2BxQ%2BbgLgEHUckUngTXDLoNJ4AASMeMVAhHWE%3D&reserved=0>,
> > https://gcc.gnu.org/wiki/AutoFDO/Tutorial<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fwiki%2FAutoFDO%2FTutorial&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C08a2bfb9135c41955a7708d90675e9ba%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637547925772855555%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IqMDaxmMTM64eavwksTEXZKwlVhR5UZZiL4tfcyu3io%3D&reserved=0>).
> > > >
> > > > I noticed that the source code for create_gcov has been deleted from
> > https://github.com/google/autofdo<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C08a2bfb9135c41955a7708d90675e9ba%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637547925772865552%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6m86ZH8vmrXFIJOGO8WPYBxap1R4uULFZi5mE04dFbc%3D&reserved=0> on April 7. I asked about that change
> > in that repo and got the following reply:
> > > >
> > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fpull%2F107%23issuecomment-819108738&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C08a2bfb9135c41955a7708d90675e9ba%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637547925772875543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9k945Ma9HRo4RzafSMzr%2FiKx8ochLr77GyIO7pzBVJ8%3D&reserved=0>
> > > >
> > > > "Actually we didn't use create_gcov and havn't updated create_gcov for
> > years, and we also didn't have enough tests to guarantee it works (It was
> > gcc-4.8 when we used and verified create_gcov). If you need it, it is
> > welcomed to update create_gcov and add it to the respository."
> > > >
> > > > Does this mean that AutoFDO is currently dead in gcc?
> > >
> > > Hello.
> > >
> > > Yes. I know that even basic test cases have been broken for years in the
> > GCC.
> > > It's new to me that create_gcov was removed.
> > >
> > > I tend to send patch to GCC that will remove AutoFDO from GCC.
> > > I known Bin spent some time working on AutoFDO, has he came up to
> > something?
> >
> > The GCC side of auto-FDO is not that hard.  We have most of
> > infrastructure in place, but stopping point for me was always difficulty
> > to get gcov-tool working.  If some maintainer steps up, I think I can
> > fix GCC side.
> >
> > I am bit unsure how important feature it is - we have FDO that works
> > quite well for most users but I know there are some users of the LLVM
> > implementation and there is potential to tie this with other hardware
> > events to asist i.e. if conversion (where one wants to know how well CPU
> > predicts the jump rather than just the jump probability) which I always
> > found potentially interesting.
> >
> > Honza
> > >
> > > Martin
> > >
> > > >
> > > > Thanks,
> > > >
> > > > Eugene
> > > >
> > >
> >