From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by sourceware.org (Postfix) with ESMTPS id 006803854835 for ; Fri, 23 Apr 2021 17:04:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 006803854835 Received: by mail-pf1-x432.google.com with SMTP id q2so2508913pfk.9 for ; Fri, 23 Apr 2021 10:04:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xfjcsB2ToykRA/R6jW7jKPl01xFXmvjjePx2fTjxKjs=; b=r8f7MuOYBYomPmVeunAPCSLZUoOVlWyeH0Xu0uQkj5jZifkdzLsFjQbkCE4piSS5tB m+OnRN1e1zLT8cKuV4RZKzOMdfZWaH3Hvg/nzCprp0Pnu4lBbFN0UdJnAKv6WobTHUkC 5PLjozBPQmVs5a/sDvR9FRLA7v+yQPDzm19IF/zhGAFFjwlITuJxfgjaA7SjlM5Nw3Bj JBUhnBE07iLXTeOuD78EcfrC/cO144OnDc2OzCS7AwrYU9hxbfrlGYnAMI50DefkdZk2 mVqyhXX//LhN7m3FixdvqkAGTPFKGRCdaqbeNUusKEK+H95sKMemJKIH0YbjbndISXYe 1m2w== X-Gm-Message-State: AOAM531Htbn8jjRCMxGmkQpI9pdNGNjzXv1FR/HwGwUb2oHXAGBpYF+p ECodAkWvrBPlxFsaXSIc+O03ky/ZnZA56wjk9vR2pg== X-Google-Smtp-Source: ABdhPJylWIawGN+dNqwkKR7+zShgyV+edg12fp2M/WUpVRov+SC14D3LJGSjkwgduDt5lSoZcEnNpYQdI4Zh/DMgGac= X-Received: by 2002:a62:1ec1:0:b029:24d:b3de:25be with SMTP id e184-20020a621ec10000b029024db3de25bemr4717399pfe.17.1619197481780; Fri, 23 Apr 2021 10:04:41 -0700 (PDT) MIME-Version: 1.0 References: <62330f82-201d-af7d-d1ed-1c8c529cc0f7@suse.cz> <20210422222906.GB5803@kam.mff.cuni.cz> <20210423165449.GC56452@kam.mff.cuni.cz> In-Reply-To: <20210423165449.GC56452@kam.mff.cuni.cz> From: Xinliang David Li Date: Fri, 23 Apr 2021 10:04:31 -0700 Message-ID: Subject: Re: State of AutoFDO in GCC To: Jan Hubicka Cc: =?UTF-8?Q?Martin_Li=C5=A1ka?= , Richard Biener , "gcc@gcc.gnu.org" , Eugene Rozenfeld X-Spam-Status: No, score=-17.1 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, HTML_MESSAGE, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Apr 2021 17:04:45 -0000 On Fri, Apr 23, 2021 at 9:54 AM Jan Hubicka wrote: > > On Fri, Apr 23, 2021 at 12:18 AM Martin Li=C5=A1ka wro= te: > > > > > On 4/23/21 9:00 AM, Richard Biener via Gcc wrote: > > > > On Fri, Apr 23, 2021 at 7:28 AM Xinliang David Li via Gcc > > > > wrote: > > > >> > > > >> Hi, the create_gcov tool was probably removed with the assumption > that > > > it > > > >> was only used with Google GCC branch, but it is actually used with > GCC > > > >> trunk as well. > > > >> > > > >> Given that, the tool will be restored in the github repo. It seems > to > > > build > > > >> and work fine with the regression test. > > > >> > > > >> The tool may ust work as it is right now, but there is no guarante= e > it > > > >> won't break in the future unless someone in the GCC community trie= s > to > > > >> maintain it. > > > > > > Hi. > > > > > > The current situation is that AutoFDO doesn't work with pretty simple > > > test-cases > > > we have in testsuite: > > > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D71672 > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D81379 > > > > > > These are ~5 years old and nothing has happened. > > Basic functionality is to identify hot parts of programs and give basic > idea of branch probabilities. Bugs above are about other optimizations > (indirect inlining, hot-cold partitioning and peeling). First one ought > to be working but still indirect call profiling accounts for relatively > minor portion of FDO benefits, so auto-FDO should be usefable even w/o > these working if tooling was fixed. Expecting hot-cold partitioning and > peeling to work reliably for testcases designed for normalFDO is IMO bit > unrealistic. > > Perf is not limited to x86, so we should get basic usability for all > major architectures. David, what CPUs are supported with auto-FDO on > LLVM side? > > > > This is a major feature in Clang. Deprecating it in GCC will be > unfortunate. > > I also like overall idea of auto-FDO just never had much time to get it > into shape. Every stage1 I am trying to fix something that was broken > for a while (like ICF last stage1) and fix it, so option would be to > focus on autoFDO this stage1 even though I have quite few other things > in TODO list. I never looked into the tools translating perf data to > gcc compatible format and how easy would be to keep this alive. > > David, how does perf->LLVM path work these days? > It uses create_llvm_prof tool which is in the same git repo. The data parsing part is shared with create_gcov, but the writer is obviously different for the two tools. David > Honza > > > > David > > > > > > > > Thoughts? > > > Martin > > > > > > > > > > > Having the tool third-party makes keeping the whole chain working > more > > > > difficult. > > > > > > > > Richard. > > > > > > > >> Thanks, > > > >> > > > >> David > > > >> > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka wrote= : > > > >> > > > >>>> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > >>>>> GCC documentation for AutoFDO points to create_gcov tool that > > > converts > > > >>> perf.data file into gcov format that can be consumed by gcc with > > > >>> -fauto-profile ( > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > >>>>> > > > >>>>> I noticed that the source code for create_gcov has been deleted > from > > > >>> https://github.com/google/autofdo on April 7. I asked about that > > > change > > > >>> in that repo and got the following reply: > > > >>>>> > > > >>>>> > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > >>>>> > > > >>>>> "Actually we didn't use create_gcov and havn't updated > create_gcov > > > for > > > >>> years, and we also didn't have enough tests to guarantee it works > (It > > > was > > > >>> gcc-4.8 when we used and verified create_gcov). If you need it, i= t > is > > > >>> welcomed to update create_gcov and add it to the respository." > > > >>>>> > > > >>>>> Does this mean that AutoFDO is currently dead in gcc? > > > >>>> > > > >>>> Hello. > > > >>>> > > > >>>> Yes. I know that even basic test cases have been broken for year= s > in > > > the > > > >>> GCC. > > > >>>> It's new to me that create_gcov was removed. > > > >>>> > > > >>>> I tend to send patch to GCC that will remove AutoFDO from GCC. > > > >>>> I known Bin spent some time working on AutoFDO, has he came up t= o > > > >>> something? > > > >>> > > > >>> The GCC side of auto-FDO is not that hard. We have most of > > > >>> infrastructure in place, but stopping point for me was always > > > difficulty > > > >>> to get gcov-tool working. If some maintainer steps up, I think I > can > > > >>> fix GCC side. > > > >>> > > > >>> I am bit unsure how important feature it is - we have FDO that > works > > > >>> quite well for most users but I know there are some users of the > LLVM > > > >>> implementation and there is potential to tie this with other > hardware > > > >>> events to asist i.e. if conversion (where one wants to know how > well > > > CPU > > > >>> predicts the jump rather than just the jump probability) which I > always > > > >>> found potentially interesting. > > > >>> > > > >>> Honza > > > >>>> > > > >>>> Martin > > > >>>> > > > >>>>> > > > >>>>> Thanks, > > > >>>>> > > > >>>>> Eugene > > > >>>>> > > > >>>> > > > >>> > > > > > > >