From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by sourceware.org (Postfix) with ESMTPS id 1D99A3858020 for ; Sun, 25 Apr 2021 23:18:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1D99A3858020 Received: by mail-pg1-x535.google.com with SMTP id f29so1766258pgm.8 for ; Sun, 25 Apr 2021 16:18:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lS0PRx28jc4/uFBHqQGk+KNqOwxSfnGZd9wCfcD501M=; b=OyTXhFVEzEeO7iHaVT0J28Awpnqk1LVzlkV9yBpGQ/Kb9smwhZkn7XKIZclmjP9bxP 6v+VPvo4OhRkBy0phWfNfpVkAJL457Ku/DqLCIQjscmrzBcd54TdwTEn+N4mXtJ/hzmK xqlWgmwhSGGp6TGPhFqG5onfOmyXt5fNIo/IAarNDK4j/gyezfCMi4gMlmd7O+GUlZEs 1YGlwTtKNqwG61Kec8NZcXK/xmHfvbjA4Ztxin9S+oW9NrDSPkI/CaOQdqVQ9G+mGof1 7qoftc6abBD+OZfSiJtMMcn6xNZuq4kigAXhOxhano+r4wNyi+cyUowMZpprHo9QImI+ F8Vg== X-Gm-Message-State: AOAM531j/LZ1YGA7DhLUJeJfcq/k8JoP9KGAyhu9xpzuDWKCOYQ2HDBe m2YcyO/jIK3pkKhT/ToAddRzsY4QrAzmPKkfs/Ye5g== X-Google-Smtp-Source: ABdhPJzd/QgXXJNVm8YMLBi4jwBEqLAr08VXSe8j53pJOpK8L7gbQK3C7eJsoE8CJj41mvWF6bmoSItnJA1/U1JalSA= X-Received: by 2002:a63:ea06:: with SMTP id c6mr14375028pgi.401.1619392713001; Sun, 25 Apr 2021 16:18:33 -0700 (PDT) MIME-Version: 1.0 References: <20210423165449.GC56452@kam.mff.cuni.cz> <20210423171611.GA82007@kam.mff.cuni.cz> <20210423192834.GA1949@kam.mff.cuni.cz> <20210425190754.GA88857@kam.mff.cuni.cz> In-Reply-To: <20210425190754.GA88857@kam.mff.cuni.cz> From: Xinliang David Li Date: Sun, 25 Apr 2021 16:18:21 -0700 Message-ID: Subject: Re: State of AutoFDO in GCC To: Jan Hubicka Cc: "gcc@gcc.gnu.org" , Eugene Rozenfeld , Wei Mi X-Spam-Status: No, score=-17.4 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, HTML_MESSAGE, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Apr 2021 23:18:36 -0000 On Sun, Apr 25, 2021 at 12:07 PM Jan Hubicka wrote: > David, > > > > The text format is documented here: > > https://clang.llvm.org/docs/UsersManual.html > > The binary format is not documented. The binary format is not guaranteed > to > > be backward compatible, so sharing the same format may not be the best > way > > as changes for clang may break GCC. > > > > Since linux perf format does not change, the tool should be relatively > > stable with low maintenance cost. Changes are needed only when some new > > AutoFDO features are added to the compiler side. > > I was under impression that it is indeed problem with the tool requiring > old format of linux perf. At least with opensuse distro the shipped tool > fails for me: > jan@skylake:~> create_llvm_prof --binary=./code --out=code.prof > E0425 21:01:55.038128 17977 perf_reader.cc:996] Unsupported event type > 79 > F0425 21:01:55.038295 17977 perf_parser.cc:240] Check failed: > reader_.ReadPerfSampleInfo(*parsed_event.raw_event, &sample_info) > *** Check failure stack trace: *** > @ 0x55e6deb6058e (unknown) > @ 0x55e6deb94a49 (unknown) > .. > Aborted (core dumped) > > I collect data as intstructed here: > https://clang.llvm.org/docs/UsersManual.html > > It is from package autofdo-0.18-4.4.x86_64 and perf 5.11.15. > > Is there a way to get this working w/o using older perf? > Honza > > Interesting. That means we will also see the same error when using the latest perf. Wei, are you aware of the issue? David > > > Does LLVM's auto-FDO support non-Intel CPUs these days? > > > > > > > It supports LBR like events, so it is CPU vendor dependent. For ARM, > using > > ETM can achieve the goal, but I don't have detailed knowledge of it. > > > > David > > > > > > > > Honza > > > > > > > > David > > > > > > > > > > > > > > >> Honza > > > > >> > > > > > >> > David > > > > >> > > > > > >> > > > > > >> > > Honza > > > > >> > > > > > > > >> > > > David > > > > >> > > > > > > > >> > > > > > > > > >> > > > > Thoughts? > > > > >> > > > > Martin > > > > >> > > > > > > > > >> > > > > > > > > > >> > > > > > Having the tool third-party makes keeping the whole > chain > > > > >> working > > > > >> > > more > > > > >> > > > > > difficult. > > > > >> > > > > > > > > > >> > > > > > Richard. > > > > >> > > > > > > > > > >> > > > > >> Thanks, > > > > >> > > > > >> > > > > >> > > > > >> David > > > > >> > > > > >> > > > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka < > > > hubicka@ucw.cz> > > > > >> wrote: > > > > >> > > > > >> > > > > >> > > > > >>>> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote: > > > > >> > > > > >>>>> GCC documentation for AutoFDO points to create_gcov > tool > > > > >> that > > > > >> > > > > converts > > > > >> > > > > >>> perf.data file into gcov format that can be consumed > by > > > gcc > > > > >> with > > > > >> > > > > >>> -fauto-profile ( > > > > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, > > > > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial). > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> I noticed that the source code for create_gcov has > been > > > > >> deleted > > > > >> > > from > > > > >> > > > > >>> https://github.com/google/autofdo on April 7. I asked > > > about > > > > >> that > > > > >> > > > > change > > > > >> > > > > >>> in that repo and got the following reply: > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> > > > > >> > > > https://github.com/google/autofdo/pull/107#issuecomment-819108738 > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> "Actually we didn't use create_gcov and havn't > updated > > > > >> > > create_gcov > > > > >> > > > > for > > > > >> > > > > >>> years, and we also didn't have enough tests to > guarantee > > > it > > > > >> works > > > > >> > > (It > > > > >> > > > > was > > > > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you > > > need > > > > >> it, it > > > > >> > > is > > > > >> > > > > >>> welcomed to update create_gcov and add it to the > > > respository." > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> Does this mean that AutoFDO is currently dead in > gcc? > > > > >> > > > > >>>> > > > > >> > > > > >>>> Hello. > > > > >> > > > > >>>> > > > > >> > > > > >>>> Yes. I know that even basic test cases have been > broken > > > for > > > > >> years > > > > >> > > in > > > > >> > > > > the > > > > >> > > > > >>> GCC. > > > > >> > > > > >>>> It's new to me that create_gcov was removed. > > > > >> > > > > >>>> > > > > >> > > > > >>>> I tend to send patch to GCC that will remove AutoFDO > from > > > > >> GCC. > > > > >> > > > > >>>> I known Bin spent some time working on AutoFDO, has > he > > > came > > > > >> up to > > > > >> > > > > >>> something? > > > > >> > > > > >>> > > > > >> > > > > >>> The GCC side of auto-FDO is not that hard. We have > most > > > of > > > > >> > > > > >>> infrastructure in place, but stopping point for me was > > > always > > > > >> > > > > difficulty > > > > >> > > > > >>> to get gcov-tool working. If some maintainer steps > up, I > > > > >> think I > > > > >> > > can > > > > >> > > > > >>> fix GCC side. > > > > >> > > > > >>> > > > > >> > > > > >>> I am bit unsure how important feature it is - we have > FDO > > > that > > > > >> > > works > > > > >> > > > > >>> quite well for most users but I know there are some > users > > > of > > > > >> the > > > > >> > > LLVM > > > > >> > > > > >>> implementation and there is potential to tie this with > > > other > > > > >> > > hardware > > > > >> > > > > >>> events to asist i.e. if conversion (where one wants to > > > know > > > > >> how > > > > >> > > well > > > > >> > > > > CPU > > > > >> > > > > >>> predicts the jump rather than just the jump > probability) > > > > >> which I > > > > >> > > always > > > > >> > > > > >>> found potentially interesting. > > > > >> > > > > >>> > > > > >> > > > > >>> Honza > > > > >> > > > > >>>> > > > > >> > > > > >>>> Martin > > > > >> > > > > >>>> > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> Thanks, > > > > >> > > > > >>>>> > > > > >> > > > > >>>>> Eugene > > > > >> > > > > >>>>> > > > > >> > > > > >>>> > > > > >> > > > > >>> > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > > > >