From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by sourceware.org (Postfix) with ESMTPS id 64098393C853 for ; Fri, 23 Apr 2021 19:58:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 64098393C853 Received: by mail-pg1-x529.google.com with SMTP id p12so35907307pgj.10 for ; Fri, 23 Apr 2021 12:58:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Lsmiy8IPAT3b4g6NcNvPcTVicnyKOd9vqMwS5PprCOE=; b=MXiwPX48XDar5MG6BBP9KaPvw5SbVi4rVjlCrMNw1oDdRF9gAqFpl/BHqmiwRxp3Au 9RaEEjVXUjtWccG8VAy6aAP4//L8zp81Nuzv5akSv0A4CypsZuppu9XlGx2JLPqhx54F 8oufWZsPOcAMwsSAzjXvF+inp1NxSRljAoh4o4eW0sudqCL0Hs2G0xZWAtbSOVb7S0W9 y8u4lNVBZunOD0x2GgF95xoTOTTEUU5EbzkfEsDjEYMAa/dx2I8Zzt0J/3+00KK/NytB NJQG/acYV+9d/iIEYw/4kLb88lMQVy4FNblJwlGYdgpYwpEiUxQNeTTcDzbbiyiUgZfx 2Z7A== X-Gm-Message-State: AOAM5330bfW6PuVLOzcRdmT2HYhehXmr8E/Q2iggcNUgjn5LhY92Up/a HNAcojbIA6lAto4qm5hXqUtmlKYO4VtGFQPJs49EVA== X-Google-Smtp-Source: ABdhPJwITn+RHr1rnsOJCcP6475RyE6UaQZ7e18koS0WJAy176T8ftpqS498Un95ZTS57btYEJZ+biFu6WZadUWyRoU= X-Received: by 2002:a62:1ec1:0:b029:24d:b3de:25be with SMTP id e184-20020a621ec10000b029024db3de25bemr5343309pfe.17.1619207938099; Fri, 23 Apr 2021 12:58:58 -0700 (PDT) MIME-Version: 1.0 References: <20210422222906.GB5803@kam.mff.cuni.cz> <20210423165449.GC56452@kam.mff.cuni.cz> <20210423171611.GA82007@kam.mff.cuni.cz> <20210423192834.GA1949@kam.mff.cuni.cz> In-Reply-To: <20210423192834.GA1949@kam.mff.cuni.cz> From: Xinliang David Li Date: Fri, 23 Apr 2021 12:58:47 -0700 Message-ID: Subject: Re: State of AutoFDO in GCC To: Jan Hubicka Cc: "gcc@gcc.gnu.org" , Eugene Rozenfeld , Wei Mi X-Spam-Status: No, score=-16.8 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, HTML_MESSAGE, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Apr 2021 19:59:01 -0000 On Fri, Apr 23, 2021 at 12:28 PM Jan Hubicka wrote: > > On Fri, Apr 23, 2021 at 10:27 AM Xinliang David Li > > wrote: > > > > > > > > > > > On Fri, Apr 23, 2021 at 10:16 AM Jan Hubicka wrote: > > > > > >> > > > >> > It uses create_llvm_prof tool which is in the same git repo. The > data > > >> > parsing part is shared with create_gcov, but the writer is obviously > > >> > different for the two tools. > > >> > > >> OK and what are the main differences between llvmand gcc format? > > >> > > >> GCC expects GCOV format, I think while LLVM uses a newly designed > binary > > > format. > > > > > > > > Sorry I missed your next reply. I forgot about the details of GCC' > format. > > Thanks for explanation. How hard would be to make GCC consume this > format? Is is reasonably stable and documented somewhere? > The text format is documented here: https://clang.llvm.org/docs/UsersManual.html The binary format is not documented. The binary format is not guaranteed to be backward compatible, so sharing the same format may not be the best way as changes for clang may break GCC. Since linux perf format does not change, the tool should be relatively stable with low maintenance cost. Changes are needed only when some new AutoFDO features are added to the compiler side. Does LLVM's auto-FDO support non-Intel CPUs these days? > It supports LBR like events, so it is CPU vendor dependent. For ARM, using ETM can achieve the goal, but I don't have detailed knowledge of it. David > > Honza > > > > David > > > > > > > > >> Honza > > >> > > > >> > David > > >> > > > >> > > > >> > > Honza > > >> > > > > > >> > > > David > > >> > > > > > >> > > > > > > >> > > > > Thoughts? > > >> > > > > Martin > > >> > > > > > > >> > > > > > > > >> > > > > > Having the tool third-party makes keeping the whole chain > > >> working > > >> > > more > > >> > > > > > difficult. > > >> > > > > > > > >> > > > > > Richard. > > >> > > > > > > > >> > > > > >> Thanks, > > >> > > > > >> > > >> > > > > >> David > > >> > > > > >> > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka < > hubicka@ucw.cz> > > >> wrote: > > >> > > > > >> > > >> > > > > >>>> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > >> > > > > >>>>> GCC documentation for AutoFDO points to create_gcov tool > > >> that > > >> > > > > converts > > >> > > > > >>> perf.data file into gcov format that can be consumed by > gcc > > >> with > > >> > > > > >>> -fauto-profile ( > > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > >> > > > > >>>>> > > >> > > > > >>>>> I noticed that the source code for create_gcov has been > > >> deleted > > >> > > from > > >> > > > > >>> https://github.com/google/autofdo on April 7. I asked > about > > >> that > > >> > > > > change > > >> > > > > >>> in that repo and got the following reply: > > >> > > > > >>>>> > > >> > > > > >>>>> > > >> > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > >> > > > > >>>>> > > >> > > > > >>>>> "Actually we didn't use create_gcov and havn't updated > > >> > > create_gcov > > >> > > > > for > > >> > > > > >>> years, and we also didn't have enough tests to guarantee > it > > >> works > > >> > > (It > > >> > > > > was > > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you > need > > >> it, it > > >> > > is > > >> > > > > >>> welcomed to update create_gcov and add it to the > respository." > > >> > > > > >>>>> > > >> > > > > >>>>> Does this mean that AutoFDO is currently dead in gcc? > > >> > > > > >>>> > > >> > > > > >>>> Hello. > > >> > > > > >>>> > > >> > > > > >>>> Yes. I know that even basic test cases have been broken > for > > >> years > > >> > > in > > >> > > > > the > > >> > > > > >>> GCC. > > >> > > > > >>>> It's new to me that create_gcov was removed. > > >> > > > > >>>> > > >> > > > > >>>> I tend to send patch to GCC that will remove AutoFDO from > > >> GCC. > > >> > > > > >>>> I known Bin spent some time working on AutoFDO, has he > came > > >> up to > > >> > > > > >>> something? > > >> > > > > >>> > > >> > > > > >>> The GCC side of auto-FDO is not that hard. We have most > of > > >> > > > > >>> infrastructure in place, but stopping point for me was > always > > >> > > > > difficulty > > >> > > > > >>> to get gcov-tool working. If some maintainer steps up, I > > >> think I > > >> > > can > > >> > > > > >>> fix GCC side. > > >> > > > > >>> > > >> > > > > >>> I am bit unsure how important feature it is - we have FDO > that > > >> > > works > > >> > > > > >>> quite well for most users but I know there are some users > of > > >> the > > >> > > LLVM > > >> > > > > >>> implementation and there is potential to tie this with > other > > >> > > hardware > > >> > > > > >>> events to asist i.e. if conversion (where one wants to > know > > >> how > > >> > > well > > >> > > > > CPU > > >> > > > > >>> predicts the jump rather than just the jump probability) > > >> which I > > >> > > always > > >> > > > > >>> found potentially interesting. > > >> > > > > >>> > > >> > > > > >>> Honza > > >> > > > > >>>> > > >> > > > > >>>> Martin > > >> > > > > >>>> > > >> > > > > >>>>> > > >> > > > > >>>>> Thanks, > > >> > > > > >>>>> > > >> > > > > >>>>> Eugene > > >> > > > > >>>>> > > >> > > > > >>>> > > >> > > > > >>> > > >> > > > > > > >> > > > > > > >> > > > > >> > > > >