RE: [EXTERNAL] Re: State of AutoFDO in GCC

[Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>]

Hi Jan,

Thank you for your reply. I did have you on the TO line in my latest patch:
https://gcc.gnu.org/pipermail/gcc-patches/2022-June/596065.html

That's the patch I need a review on.

I'm looking forward to co-maintaining AutoFDO with you.

Thanks,

Eugene

-----Original Message-----
From: Jan Hubicka <hubicka@ucw.cz> 
Sent: Wednesday, July 27, 2022 12:27 AM
To: David Edelsohn <dje.gcc@gmail.com>
Cc: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; Martin Liska <mliska@suse.cz>; Xinliang David Li <davidxl@google.com>; gcc <gcc@gcc.gnu.org>; Andi Kleen <ak@linux.intel.com>; Joseph Myers <joseph@codesourcery.com>
Subject: [EXTERNAL] Re: State of AutoFDO in GCC

> On Tue, Jul 26, 2022 at 4:13 PM Eugene Rozenfeld via Gcc 
> <gcc@gcc.gnu.org> wrote:
> >
> > Hello GCC community.
> >
> > I started this thread on the state of AutoFDO in GCC more than a 
> > year ago. Here is the first message in the thread: 
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgc
> > c.gnu.org%2Fpipermail%2Fgcc%2F2021-April%2F235860.html&amp;data=05%7
> > C01%7CEugene.Rozenfeld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa
> > 1619e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63794503618637077
> > 0%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=9%2Bnv2ShWxKh88K%
> > 2BsOeqPgQX3lOCJQ0lnF%2F7SUs4K4uI%3D&amp;reserved=0
> >
> > Since then I committed a number of patches to revive AutoFDO in GCC:
> >
> > Fix a typo in an AutoFDO error 
> > string<https://nam06.safelinks.protection.outlook.com/?url=https%3A%
> > 2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D23691ddd3
> > aa3ffe55892b2bff54f9a15a89de2b4&amp;data=05%7C01%7CEugene.Rozenfeld%
> > 40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141a
> > f91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbGZsb
> > 3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
> > D%7C3000%7C%7C%7C&amp;sdata=qkecfE9uH5gy91vILQQlCk9RpExqPZxO4q02wiN1
> > EFw%3D&amp;reserved=0> Update gen_autofdo_event.py and 
> > gcc-auto-profile.<https://nam06.safelinks.protection.outlook.com/?ur
> > l=https%3A%2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%
> > 3D01d402c5e0ac1ddf5618bbe316b50067625fda46&amp;data=05%7C01%7CEugene
> > .Rozenfeld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f9
> > 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%
> > 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC
> > JXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=E52qVFlfdfFGnW9yDsBNhh4k2ey8g
> > 3aJEGzH40MuSOc%3D&amp;reserved=0> Fixes for AutoFDO 
> > tests<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
> > F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Df9ad3d5339
> > faaaed6e15a7b27d90fbc66eb72f37&amp;data=05%7C01%7CEugene.Rozenfeld%4
> > 0microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af
> > 91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbGZsb3
> > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C3000%7C%7C%7C&amp;sdata=XYlFoY3OTTXHp18O1v8BY47A17NyNPXvUWWYsVnbD
> > 0U%3D&amp;reserved=0> Fix indir-call-prof-2.c with 
> > AutoFDO<https://nam06.safelinks.protection.outlook.com/?url=https%3A
> > %2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D0ed093c7
> > c3f755bc1cd80e5186abeb2f5c50ee0c&amp;data=05%7C01%7CEugene.Rozenfeld
> > %40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141
> > af91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbGZs
> > b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> > 3D%7C3000%7C%7C%7C&amp;sdata=sLftY6hjvzSuE9ZgkGmXZLDpRMjlDo%2FEAyDyP
> > CviY5Q%3D&amp;reserved=0> Fixes for AutoFDO 
> > testing<https://nam06.safelinks.protection.outlook.com/?url=https%3A
> > %2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D9265b378
> > 531391498ec1727f67a45da72a6c07e9&amp;data=05%7C01%7CEugene.Rozenfeld
> > %40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141
> > af91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbGZs
> > b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> > 3D%7C3000%7C%7C%7C&amp;sdata=lXZ%2F%2FbcfYD%2BQyIiXMAaCxOujEAfDXSY1p
> > 78kUb2md7w%3D&amp;reserved=0> Fix indirect call inlining with 
> > AutoFDO<https://nam06.safelinks.protection.outlook.com/?url=https%3A
> > %2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D285aa689
> > 5d479bed8e72ad363290846645b6faa0&amp;data=05%7C01%7CEugene.Rozenfeld
> > %40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141
> > af91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbGZs
> > b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> > 3D%7C3000%7C%7C%7C&amp;sdata=ypoF%2BZnEe3eC3Gat6%2FjUJLV2XiltdjJHe68
> > pue64fSU%3D&amp;reserved=0> Improve AutoFDO count propagation 
> > algorithm<https://nam06.safelinks.protection.outlook.com/?url=https%
> > 3A%2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D3d9e67
> > 67939e9658260e2506e81ec32b37cba041&amp;data=05%7C01%7CEugene.Rozenfe
> > ld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f1
> > 41af91ab2d7cd011db47%7C1%7C0%7C637945036186370770%7CUnknown%7CTWFpbG
> > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn
> > 0%3D%7C3000%7C%7C%7C&amp;sdata=IyKR391AJksu3p85Z5q5xS%2FT7i0IC%2B9oX
> > Y3VKE49s58%3D&amp;reserved=0>
> > AutoFDO: don't set param_early_inliner_max_iterations to 
> > 10.<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
> > 2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Dc17975d81aae
> > d49ff759c20c68b31304a6953d58&amp;data=05%7C01%7CEugene.Rozenfeld%40m
> > icrosoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af91
> > ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3d8
> > eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> > C3000%7C%7C%7C&amp;sdata=d%2Fj8IyRYVjCA1grtov7PNzhp4eRszLEflY21kKCN8
> > eo%3D&amp;reserved=0>
> > AutoFDO: Don't try to promote indirect calls that result in 
> > recursive direct 
> > calls<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
> > F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Dba125745d9
> > e9fe90a18a2af8701b3269c5fdd468&amp;data=05%7C01%7CEugene.Rozenfeld%4
> > 0microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af
> > 91ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3
> > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C3000%7C%7C%7C&amp;sdata=L4bypORscLpmmhxnaAupzTWepno447nbMcPUGiah2
> > b0%3D&amp;reserved=0> Fix profile count maintenance in vectorizer 
> > peeling.<https://nam06.safelinks.protection.outlook.com/?url=https%3
> > A%2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D5af2202
> > 4f62f1f596a35d3c138d41d47d5697ca0&amp;data=05%7C01%7CEugene.Rozenfel
> > d%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f14
> > 1af91ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZ
> > sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
> > %3D%7C3000%7C%7C%7C&amp;sdata=LTaHWhkpp2wH%2FaqoJZQ%2Bvcj5o%2FKZ4kQP
> > OstxpsLuOKI%3D&amp;reserved=0>
> >
> > I also made a number of fixes and improvements to create_gcov tool in https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&amp;data=05%7C01%7CEugene.Rozenfeld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=6i14y0lx0m%2FSaN2jw3DKykIun5XiKj17x5%2BBL%2BGkBd4%3D&amp;reserved=0 .
> >
> > AutoFDO in GCC is in a much better shape now.
> >
> > I have a further set of patches that improve DWARF discriminator 
> > support in GCC and enable AutoFDO to use discriminators. It's based on commits in an old Google vendor branch as described in Andi's mail below but uses a different approach for keeping track of per-instruction discriminators.
> >
> > I submitted the first (and the biggest) of these patches almost 2 
> > months ago on June 2: 
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgc
> > c.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D5af22024f62f1f59
> > 6a35d3c138d41d47d5697ca0&amp;data=05%7C01%7CEugene.Rozenfeld%40micro
> > soft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af91ab2d
> > 7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3d8eyJW
> > IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> > 0%7C%7C%7C&amp;sdata=LTaHWhkpp2wH%2FaqoJZQ%2Bvcj5o%2FKZ4kQPOstxpsLuO
> > KI%3D&amp;reserved=0 but only got a review from Andi 
> > (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fpipermail%2Fgcc-patches%2F2022-June%2F596549.html&amp;data=05%7C01%7CEugene.Rozenfeld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=G2578gq1F7h5s8ussOZVZtERXNKkgQD%2FJOvJ0KB7ct8%3D&amp;reserved=0) who is not allowed to approve patches for commit. I pinged gcc-patches twice with no success.
> >
> > I would appreciate help in getting a review on this patch so that I can get it committed and submit patches that depend on it.
> 
> Hi, Eugene
> 
> Thanks for your efforts to fix and improve AutoFDO in GCC.  I believe 
> that part of the difficulty with obtaining a review of the patches is 
> that the original authors have dispersed and no one in the GCC 
> community officially is the maintainer for the feature.  Because you 
> seem to be one of the primary users and developers, would you be 
> interested to take on the responsibility of maintaining the 
> AutoFDO-specific portions of the code, with guidance and mentorship 
> from other GCC maintainers, especially the ones responsible for gcov 
> and PDO?

I missed the patches (it would help to add me to CC :) and will review the FDO/profile facing parts.  Since it also extends debug info generation and front-ends I think we also need reviewer for that part.

Having auto-FDO co-maintainer would be welcome.

Honza
> 
> Thanks, David
> 
> >
> > Thank you,
> >
> > Eugene
> >
> > -----Original Message-----
> > From: Andi Kleen <ak@linux.intel.com>
> > Sent: Monday, May 10, 2021 10:21 AM
> > To: Joseph Myers <joseph@codesourcery.com>
> > Cc: Jan Hubicka <hubicka@ucw.cz>; gcc <gcc@gcc.gnu.org>; Eugene 
> > Rozenfeld <Eugene.Rozenfeld@microsoft.com>
> > Subject: [EXTERNAL] Re: State of AutoFDO in GCC
> >
> > On Mon, May 10, 2021 at 04:55:50PM +0000, Joseph Myers wrote:
> > > On Mon, 10 May 2021, Andi Kleen via Gcc wrote:
> > >
> > > > It's difficult to find now because it was a branch in the old 
> > > > SVN that wasn't converted. Sadly the great git conversion was quite lossy.
> > >
> > > All branches and tags, including deleted ones, were converted 
> > > (under not-fetched-by-default refs in some cases); the git 
> > > repository has everything that might plausibly be useful, omitting 
> > > only a few things that would have been meaningless to convert, 
> > > such as mistaken branch creations in the root of the repository and the SVN hooks directory.
> > > Use "git ls-remote git://gcc.gnu.org/git/gcc.git" to see the full 
> > > list of over 5000 refs available in the repository (or do a clone 
> > > with --mirror to fetch them all).
> >
> > Okay thanks. I don't see them in any of the web interfaces, neither 
> > on
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgc
> > c.gnu.org%2Fgit%2Fgitweb.cgi%3Fp%3Dgcc.git&amp;data=05%7C01%7CEugene
> > .Rozenfeld%40microsoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f9
> > 88bf86f141af91ab2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%
> > 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC
> > JXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=BC%2BdUNgy2eSrnK7dWA%2B1hj1my
> > OhW7O94Mdcywbx1AI8%3D&amp;reserved=0
> > nor on
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > thub.com%2Fgcc-mirror%2Fgcc&amp;data=05%7C01%7CEugene.Rozenfeld%40mi
> > crosoft.com%7Cfe5184091a18487fd92d08da6fa1619e%7C72f988bf86f141af91a
> > b2d7cd011db47%7C1%7C0%7C637945036186683243%7CUnknown%7CTWFpbGZsb3d8e
> > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > 3000%7C%7C%7C&amp;sdata=TcRqYfDpYTTRgh7y%2FxRmuAjLDrLDUMLwrnJsklD2mZ
> > I%3D&amp;reserved=0
> > but
> > git fetch origin vendors/google/heads/gcc-4_8 does the trick for fetching the commits, but not the symbolic branches.
> >
> > Anyways with that it looks like the discriminator changes are:
> >
> > commit fd9de90d750e3588b1e5a218b28102b6c8bb8434
> > Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> > Date:   Thu Oct 10 14:39:31 2013 +0000
> >
> >     Use only lineno+discriminator (remove the callee function name) as the key to represent callsite. Because each callsite will have its discriminator if in the same line.
> >
> >     2013-10-10  Dehao Chen  
> > <dehao@google.com<mailto:dehao@google.com>>
> >
> >             * gcc/auto-profile.c (get_function_instance_by_decl): Remove
> >             callee_name from callsite.
> >             (read_function_instance): Likewise.
> >
> >     From-SVN: r203379
> >
> > commit 3987da76affbfbe7195c0a16b33beedc649ec14f
> > Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> > Date:   Tue Aug 27 16:46:49 2013 +0000
> >
> >     Refactor AutoFDO to:
> >
> >     1. Now that we have discriminator for inlined callsite, we do not need special handling for callsite location any more.
> >     2. If a source line is mapped to multiple BBs, only the first BB will be annotated.
> >     3. Before actual annotation, mark everythin BB/edge as not annotated.
> >
> >     2013-08-27  Dehao Chen  
> > <dehao@google.com<mailto:dehao@google.com>>
> >
> >             * gcc/auto-profile.c (location_set): New data structure.
> >             (get_count_info): Add new parameter.
> >             (get_combined_location): Remove unused parameter.
> >             (get_inline_stack): Remove unused parameter.
> >             (afdo_get_bb_count): Add new parameter.
> >             (afdo_annotate_cfg): Reset annotated flags.
> > commit 1e6c4a7a8fb8e20545bb9f9032d3854f3f794c18
> > Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> > Date:   Thu Aug 22 17:20:29 2013 +0000
> >
> >     Set discriminator for call stmts within a same basic block.
> >
> >     2013-08-22  Dehao Chen  
> > <dehao@google.com<mailto:dehao@google.com>>
> >
> >             * gcc/tree-cfg.c (assign_discriminators): assign discriminator for
> >             call stmt in a same BB if it is mapped to a same line.
> >
> > commit b0be0175f4cf18fdd77bb013b181eead3a0a4773
> > Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> > Date:   Mon Aug 19 21:26:33 2013 +0000
> >
> >     Fix the discriminator assignment bug during hashing.
> >
> >     2013-08-19  Dehao Chen  
> > <dehao@google.com<mailto:dehao@google.com>>
> >
> >             * tree-cfg.c (next_discriminator_for_locus): Fix discriminator
> >             assignment bug.
> >
> >     From-SVN: r201857
> > commit 9fa26998a63d4b22b637ed8702520819e408a694
> > Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> > Date:   Mon Aug 19 20:16:47 2013 +0000
> >
> >     Add discrminator for inlined callsites.
> >
> >     2013-08-19  Dehao Chen  
> > (dehao@google.com<mailto:dehao@google.com>)
> >
> >             * include/dwarf2.def (DW_AT_GNU_discriminator): New attribute.
> >             * gcc/dwarf2out.c (add_call_src_coords_attributes): Emit discriminator
> >             attribute for inlined callsite.
> >
> >     From-SVN: r201856
> >
> > -Andi
> >