RE: [EXTERNAL] Re: State of AutoFDO in GCC

[Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>]

Hi David,

Thank you for your offer to take on the responsibility of maintaining the AutoFDO-specific portions of the code. I'll be happy to do that with Jan's and other GGC maintainers' help.
I hope more people will start using and contributing to AutoFDO.

Thanks,

Eugene

-----Original Message-----
From: David Edelsohn <dje.gcc@gmail.com> 
Sent: Tuesday, July 26, 2022 3:38 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; Jan Hubicka <hubicka@ucw.cz>; Martin Liska <mliska@suse.cz>; Xinliang David Li <davidxl@google.com>
Cc: Andi Kleen <ak@linux.intel.com>; Joseph Myers <joseph@codesourcery.com>; gcc <gcc@gcc.gnu.org>
Subject: [EXTERNAL] Re: State of AutoFDO in GCC

[You don't often get email from dje.gcc@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On Tue, Jul 26, 2022 at 4:13 PM Eugene Rozenfeld via Gcc <gcc@gcc.gnu.org> wrote:
>
> Hello GCC community.
>
> I started this thread on the state of AutoFDO in GCC more than a year 
> ago. Here is the first message in the thread: 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.
> gnu.org%2Fpipermail%2Fgcc%2F2021-April%2F235860.html&amp;data=05%7C01%
> 7CEugene.Rozenfeld%40microsoft.com%7Ccabaec504f234632a03a08da6f5780d2%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637944718882043585%7CUnkn
> own%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwi
> LCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=5RoiLypTlkNRIZW8yAufYR4qiO573
> 0PADO%2BFsS6InIU%3D&amp;reserved=0
>
> Since then I committed a number of patches to revive AutoFDO in GCC:
>
> Fix a typo in an AutoFDO error 
> string<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F
> %2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D23691ddd3aa3f
> fe55892b2bff54f9a15a89de2b4&amp;data=05%7C01%7CEugene.Rozenfeld%40micr
> osoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7
> cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjo
> iMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%
> 7C%7C&amp;sdata=LRgW4qyh%2FAdrlKluUXnaDTFCNW8tnS1WX8bCs1WAT7s%3D&amp;r
> eserved=0> Update gen_autofdo_event.py and 
> gcc-auto-profile.<https://nam06.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D01
> d402c5e0ac1ddf5618bbe316b50067625fda46&amp;data=05%7C01%7CEugene.Rozen
> feld%40microsoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f1
> 41af91ab2d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZs
> b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> %7C3000%7C%7C%7C&amp;sdata=oBJvTW6RrkY6uURmr0UK5gaNiVS1b8vFfIyOoqq8AkM
> %3D&amp;reserved=0> Fixes for AutoFDO 
> tests<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
> 2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Df9ad3d5339faaa
> ed6e15a7b27d90fbc66eb72f37&amp;data=05%7C01%7CEugene.Rozenfeld%40micro
> soft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi
> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7
> C%7C&amp;sdata=6evlP10%2BQJyUuVZ3u%2Fv%2FP9nSIsASLjtWETyqeBfnQ2k%3D&am
> p;reserved=0> Fix indir-call-prof-2.c with 
> AutoFDO<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D0ed093c7c3f7
> 55bc1cd80e5186abeb2f5c50ee0c&amp;data=05%7C01%7CEugene.Rozenfeld%40mic
> rosoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d
> 7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> %7C%7C&amp;sdata=mtMd21By6r8DiIIXzk5ePHP0iU%2FHfnDQG8%2FXJbAr5qE%3D&am
> p;reserved=0> Fixes for AutoFDO 
> testing<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D9265b3785313
> 91498ec1727f67a45da72a6c07e9&amp;data=05%7C01%7CEugene.Rozenfeld%40mic
> rosoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d
> 7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> %7C%7C&amp;sdata=QnKrxt7W9wfTYY3Drjm%2FSn7D4yHwxjInvI8dj9KEIs4%3D&amp;
> reserved=0> Fix indirect call inlining with 
> AutoFDO<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2
> F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D285aa6895d47
> 9bed8e72ad363290846645b6faa0&amp;data=05%7C01%7CEugene.Rozenfeld%40mic
> rosoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d
> 7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> %7C%7C&amp;sdata=695ieVftlFRwICzxxMmmIb9%2F%2FDBlYBy9jJn%2ByfzkhP0%3D&
> amp;reserved=0> Improve AutoFDO count propagation 
> algorithm<https://nam06.safelinks.protection.outlook.com/?url=https%3A
> %2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D3d9e676793
> 9e9658260e2506e81ec32b37cba041&amp;data=05%7C01%7CEugene.Rozenfeld%40m
> icrosoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab
> 2d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJW
> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> 7C%7C%7C&amp;sdata=IXpOMFrH%2FSsrI7f1WbEJQHtl4Nt1ClVYzvM%2FCbR%2FDm0%3
> D&amp;reserved=0>
> AutoFDO: don't set param_early_inliner_max_iterations to 
> 10.<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> gcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Dc17975d81aaed49f
> f759c20c68b31304a6953d58&amp;data=05%7C01%7CEugene.Rozenfeld%40microso
> ft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7cd0
> 11db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%
> 7C&amp;sdata=%2FM2iGyPxGCi8%2FnhqMQ4eUafsNyWy1IrNABa2TgVdBdQ%3D&amp;re
> served=0>
> AutoFDO: Don't try to promote indirect calls that result in recursive 
> direct 
> calls<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%
> 2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3Dba125745d9e9fe
> 90a18a2af8701b3269c5fdd468&amp;data=05%7C01%7CEugene.Rozenfeld%40micro
> soft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7c
> d011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi
> MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7
> C%7C&amp;sdata=%2BMV7YxH617Nf6PjEF8Z3ImdEkvBptM9y47fp1ja4wVY%3D&amp;re
> served=0> Fix profile count maintenance in vectorizer 
> peeling.<https://nam06.safelinks.protection.outlook.com/?url=https%3A%
> 2F%2Fgcc.gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D5af22024f62
> f1f596a35d3c138d41d47d5697ca0&amp;data=05%7C01%7CEugene.Rozenfeld%40mi
> crosoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2
> d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWI
> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7
> C%7C%7C&amp;sdata=aiXW3IyLauhHOvpsPcqDXwg9VT3nn7o1Zh1KPKLxlDo%3D&amp;r
> eserved=0>
>
> I also made a number of fixes and improvements to create_gcov tool in https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&amp;data=05%7C01%7CEugene.Rozenfeld%40microsoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=zGQgQdD%2F6a9tk8EcaFLcQ1DiX5nGgzIjWUrWqzEIjk4%3D&amp;reserved=0 .
>
> AutoFDO in GCC is in a much better shape now.
>
> I have a further set of patches that improve DWARF discriminator 
> support in GCC and enable AutoFDO to use discriminators. It's based on commits in an old Google vendor branch as described in Andi's mail below but uses a different approach for keeping track of per-instruction discriminators.
>
> I submitted the first (and the biggest) of these patches almost 2 
> months ago on June 2: 
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.
> gnu.org%2Fgit%2F%3Fp%3Dgcc.git%3Ba%3Dcommit%3Bh%3D5af22024f62f1f596a35
> d3c138d41d47d5697ca0&amp;data=05%7C01%7CEugene.Rozenfeld%40microsoft.c
> om%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7cd011db
> 47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj
> AwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&a
> mp;sdata=aiXW3IyLauhHOvpsPcqDXwg9VT3nn7o1Zh1KPKLxlDo%3D&amp;reserved=0
> but only got a review from Andi (https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.gnu.org%2Fpipermail%2Fgcc-patches%2F2022-June%2F596549.html&amp;data=05%7C01%7CEugene.Rozenfeld%40microsoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=TC0cBmoLsDJ3rx9paqvfFFTR9mSVuW%2Fb%2Bu4Y8jFyf%2FE%3D&amp;reserved=0) who is not allowed to approve patches for commit. I pinged gcc-patches twice with no success.
>
> I would appreciate help in getting a review on this patch so that I can get it committed and submit patches that depend on it.

Hi, Eugene

Thanks for your efforts to fix and improve AutoFDO in GCC.  I believe that part of the difficulty with obtaining a review of the patches is that the original authors have dispersed and no one in the GCC community officially is the maintainer for the feature.  Because you seem to be one of the primary users and developers, would you be interested to take on the responsibility of maintaining the AutoFDO-specific portions of the code, with guidance and mentorship from other GCC maintainers, especially the ones responsible for gcov and PDO?

Thanks, David

>
> Thank you,
>
> Eugene
>
> -----Original Message-----
> From: Andi Kleen <ak@linux.intel.com>
> Sent: Monday, May 10, 2021 10:21 AM
> To: Joseph Myers <joseph@codesourcery.com>
> Cc: Jan Hubicka <hubicka@ucw.cz>; gcc <gcc@gcc.gnu.org>; Eugene 
> Rozenfeld <Eugene.Rozenfeld@microsoft.com>
> Subject: [EXTERNAL] Re: State of AutoFDO in GCC
>
> On Mon, May 10, 2021 at 04:55:50PM +0000, Joseph Myers wrote:
> > On Mon, 10 May 2021, Andi Kleen via Gcc wrote:
> >
> > > It's difficult to find now because it was a branch in the old SVN 
> > > that wasn't converted. Sadly the great git conversion was quite lossy.
> >
> > All branches and tags, including deleted ones, were converted (under 
> > not-fetched-by-default refs in some cases); the git repository has 
> > everything that might plausibly be useful, omitting only a few 
> > things that would have been meaningless to convert, such as mistaken 
> > branch creations in the root of the repository and the SVN hooks directory.
> > Use "git ls-remote git://gcc.gnu.org/git/gcc.git" to see the full 
> > list of over 5000 refs available in the repository (or do a clone 
> > with --mirror to fetch them all).
>
> Okay thanks. I don't see them in any of the web interfaces, neither on
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgcc.
> gnu.org%2Fgit%2Fgitweb.cgi%3Fp%3Dgcc.git&amp;data=05%7C01%7CEugene.Roz
> enfeld%40microsoft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86
> f141af91ab2d7cd011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbG
> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C3000%7C%7C%7C&amp;sdata=spoRRd6x520PaarYjzVmTcmjhIXTtVZWw%2Bh0LJc
> jshE%3D&amp;reserved=0
> nor on
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fgcc-mirror%2Fgcc&amp;data=05%7C01%7CEugene.Rozenfeld%40micros
> oft.com%7Ccabaec504f234632a03a08da6f5780d2%7C72f988bf86f141af91ab2d7cd
> 011db47%7C1%7C0%7C637944718882199821%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiM
> C4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> %7C&amp;sdata=442RFALNd91XWlWDIo4482Q7YUuJBBlETZwE1WZDZP8%3D&amp;reser
> ved=0
> but
> git fetch origin vendors/google/heads/gcc-4_8 does the trick for fetching the commits, but not the symbolic branches.
>
> Anyways with that it looks like the discriminator changes are:
>
> commit fd9de90d750e3588b1e5a218b28102b6c8bb8434
> Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> Date:   Thu Oct 10 14:39:31 2013 +0000
>
>     Use only lineno+discriminator (remove the callee function name) as the key to represent callsite. Because each callsite will have its discriminator if in the same line.
>
>     2013-10-10  Dehao Chen  
> <dehao@google.com<mailto:dehao@google.com>>
>
>             * gcc/auto-profile.c (get_function_instance_by_decl): Remove
>             callee_name from callsite.
>             (read_function_instance): Likewise.
>
>     From-SVN: r203379
>
> commit 3987da76affbfbe7195c0a16b33beedc649ec14f
> Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> Date:   Tue Aug 27 16:46:49 2013 +0000
>
>     Refactor AutoFDO to:
>
>     1. Now that we have discriminator for inlined callsite, we do not need special handling for callsite location any more.
>     2. If a source line is mapped to multiple BBs, only the first BB will be annotated.
>     3. Before actual annotation, mark everythin BB/edge as not annotated.
>
>     2013-08-27  Dehao Chen  
> <dehao@google.com<mailto:dehao@google.com>>
>
>             * gcc/auto-profile.c (location_set): New data structure.
>             (get_count_info): Add new parameter.
>             (get_combined_location): Remove unused parameter.
>             (get_inline_stack): Remove unused parameter.
>             (afdo_get_bb_count): Add new parameter.
>             (afdo_annotate_cfg): Reset annotated flags.
> commit 1e6c4a7a8fb8e20545bb9f9032d3854f3f794c18
> Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> Date:   Thu Aug 22 17:20:29 2013 +0000
>
>     Set discriminator for call stmts within a same basic block.
>
>     2013-08-22  Dehao Chen  
> <dehao@google.com<mailto:dehao@google.com>>
>
>             * gcc/tree-cfg.c (assign_discriminators): assign discriminator for
>             call stmt in a same BB if it is mapped to a same line.
>
> commit b0be0175f4cf18fdd77bb013b181eead3a0a4773
> Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> Date:   Mon Aug 19 21:26:33 2013 +0000
>
>     Fix the discriminator assignment bug during hashing.
>
>     2013-08-19  Dehao Chen  
> <dehao@google.com<mailto:dehao@google.com>>
>
>             * tree-cfg.c (next_discriminator_for_locus): Fix discriminator
>             assignment bug.
>
>     From-SVN: r201857
> commit 9fa26998a63d4b22b637ed8702520819e408a694
> Author: Dehao Chen <dehao@gcc.gnu.org<mailto:dehao@gcc.gnu.org>>
> Date:   Mon Aug 19 20:16:47 2013 +0000
>
>     Add discrminator for inlined callsites.
>
>     2013-08-19  Dehao Chen  
> (dehao@google.com<mailto:dehao@google.com>)
>
>             * include/dwarf2.def (DW_AT_GNU_discriminator): New attribute.
>             * gcc/dwarf2out.c (add_call_src_coords_attributes): Emit discriminator
>             attribute for inlined callsite.
>
>     From-SVN: r201856
>
> -Andi
>