Re: AutoFDO tools for GCC

[Andi Kleen <ak@linux.intel.com>]

On Tue, Mar 26, 2024 at 08:45:22AM +0100, Richard Biener wrote:
> On Mon, Mar 25, 2024 at 9:54 PM Eugene Rozenfeld via Gcc
> <gcc@gcc.gnu.org> wrote:
> >
> > Hello,
> >
> > I've been the AutoFDO maintainer for the last 1.5 years. I've resurrected autoprofiledbootstrap build and made a number of other fixes/improvements (e.g., discriminator support).
> >
> > The tools for AutoFDO (create_gcov, etc.) currently live in https://github.com/google/AutoFDO  repo and GCC AutoFDO documentation points users to that repo. That repo also has tools for LLVM AutoFDO.
> > https://github.com/google/AutoFDO  has several submodules: https://github.com/google/autofdo/blob/master/.gitmodules
> >
> > I got a message from Snehasish (cc'd) that google intends to migrate the tools for LLVM to the LLVM repo and wants to archive https://github.com/google/AutoFDO. That will be a problem for AutoFDO in GCC. The idea to find a different home for GCC AutoFDO tools was discussed before on this alias but this becomes more urgent now. One idea was to build these tools from GCC repo and another was to produce gcov from perf tool directly. Andi (cc'd)  had some early unfinished prototype for latter.
> >
> > Please let me know if you have thoughts on how we should proceed.
> 
> I think it makes sense for GCC specific parts to live in the GCC
> repository alongside gcov tools.  I do wonder how much common code
> there is
> between the LLVM and the GCC tooling though and whether it makes sense
> to keep it common (and working with both frontends)?  The
> pragmatic solution would have been to fork the repo on github to a
> place not within the google group ...

In tree would need convincing Google to assign the copyright.

Some of the code has problems however. The main problem is
that it always loads everything into memory, so you cannot do
longer recording session with larger data files.
Even the gcc boot strap is already pushing it on a reasonably memory
sized machine.

The scheme of only processing one target at a time is also quite inefficient.
For systems with shared libraries and multiple binaries it would
be much more efficient to do it all in one pass and write
multiple gcovs, and then just let Makefiles or gcc wrapper pick.

The other big issue is that the perf format keeps adding new 
records and their perf parser is written in a way that
it always asserts on anything it doesn't understand, so there
is constant breakage with new perf versions.
That is fixable however (I had a patch submitted but it wasn't merged)

I think what makes sense to have from the code based are
profile_diff/merge etc. which are needed for scalable collection.
Or perhaps it would be best if gcov just gained those functionalities.

-Andi