From: Xinliang David Li <davidxl@google.com>
To: Jan Hubicka <hubicka@ucw.cz>
Cc: "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>,
Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>,
Wei Mi <wmi@google.com>
Subject: Re: State of AutoFDO in GCC
Date: Sun, 25 Apr 2021 16:18:21 -0700 [thread overview]
Message-ID: <CAAkRFZLfGd3WStyQ6beJXb4PaTm1PjdYwVBoit2ijbn5r-n_xQ@mail.gmail.com> (raw)
In-Reply-To: <20210425190754.GA88857@kam.mff.cuni.cz>
On Sun, Apr 25, 2021 at 12:07 PM Jan Hubicka <hubicka@ucw.cz> wrote:
> David,
> >
> > The text format is documented here:
> > https://clang.llvm.org/docs/UsersManual.html
> > The binary format is not documented. The binary format is not guaranteed
> to
> > be backward compatible, so sharing the same format may not be the best
> way
> > as changes for clang may break GCC.
> >
> > Since linux perf format does not change, the tool should be relatively
> > stable with low maintenance cost. Changes are needed only when some new
> > AutoFDO features are added to the compiler side.
>
> I was under impression that it is indeed problem with the tool requiring
> old format of linux perf. At least with opensuse distro the shipped tool
> fails for me:
> jan@skylake:~> create_llvm_prof --binary=./code --out=code.prof
> E0425 21:01:55.038128 17977 perf_reader.cc:996] Unsupported event type
> 79
> F0425 21:01:55.038295 17977 perf_parser.cc:240] Check failed:
> reader_.ReadPerfSampleInfo(*parsed_event.raw_event, &sample_info)
> *** Check failure stack trace: ***
> @ 0x55e6deb6058e (unknown)
> @ 0x55e6deb94a49 (unknown)
> ..
> Aborted (core dumped)
>
> I collect data as intstructed here:
> https://clang.llvm.org/docs/UsersManual.html
>
> It is from package autofdo-0.18-4.4.x86_64 and perf 5.11.15.
>
> Is there a way to get this working w/o using older perf?
> Honza
> >
Interesting. That means we will also see the same error when using the
latest perf.
Wei, are you aware of the issue?
David
>
> > Does LLVM's auto-FDO support non-Intel CPUs these days?
> > >
> >
> > It supports LBR like events, so it is CPU vendor dependent. For ARM,
> using
> > ETM can achieve the goal, but I don't have detailed knowledge of it.
> >
> > David
> >
> > >
> > > Honza
> > > >
> > > > David
> > > > >
> > > > >
> > > > >> Honza
> > > > >> >
> > > > >> > David
> > > > >> >
> > > > >> >
> > > > >> > > Honza
> > > > >> > > >
> > > > >> > > > David
> > > > >> > > >
> > > > >> > > > >
> > > > >> > > > > Thoughts?
> > > > >> > > > > Martin
> > > > >> > > > >
> > > > >> > > > > >
> > > > >> > > > > > Having the tool third-party makes keeping the whole
> chain
> > > > >> working
> > > > >> > > more
> > > > >> > > > > > difficult.
> > > > >> > > > > >
> > > > >> > > > > > Richard.
> > > > >> > > > > >
> > > > >> > > > > >> Thanks,
> > > > >> > > > > >>
> > > > >> > > > > >> David
> > > > >> > > > > >>
> > > > >> > > > > >> On Thu, Apr 22, 2021 at 3:29 PM Jan Hubicka <
> > > hubicka@ucw.cz>
> > > > >> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >>>> On 4/22/21 9:58 PM, Eugene Rozenfeld via Gcc wrote:
> > > > >> > > > > >>>>> GCC documentation for AutoFDO points to create_gcov
> tool
> > > > >> that
> > > > >> > > > > converts
> > > > >> > > > > >>> perf.data file into gcov format that can be consumed
> by
> > > gcc
> > > > >> with
> > > > >> > > > > >>> -fauto-profile (
> > > > >> > > > > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html,
> > > > >> > > > > >>> https://gcc.gnu.org/wiki/AutoFDO/Tutorial).
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> I noticed that the source code for create_gcov has
> been
> > > > >> deleted
> > > > >> > > from
> > > > >> > > > > >>> https://github.com/google/autofdo on April 7. I asked
> > > about
> > > > >> that
> > > > >> > > > > change
> > > > >> > > > > >>> in that repo and got the following reply:
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>>
> > > > >> > >
> https://github.com/google/autofdo/pull/107#issuecomment-819108738
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> "Actually we didn't use create_gcov and havn't
> updated
> > > > >> > > create_gcov
> > > > >> > > > > for
> > > > >> > > > > >>> years, and we also didn't have enough tests to
> guarantee
> > > it
> > > > >> works
> > > > >> > > (It
> > > > >> > > > > was
> > > > >> > > > > >>> gcc-4.8 when we used and verified create_gcov). If you
> > > need
> > > > >> it, it
> > > > >> > > is
> > > > >> > > > > >>> welcomed to update create_gcov and add it to the
> > > respository."
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> Does this mean that AutoFDO is currently dead in
> gcc?
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Hello.
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Yes. I know that even basic test cases have been
> broken
> > > for
> > > > >> years
> > > > >> > > in
> > > > >> > > > > the
> > > > >> > > > > >>> GCC.
> > > > >> > > > > >>>> It's new to me that create_gcov was removed.
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> I tend to send patch to GCC that will remove AutoFDO
> from
> > > > >> GCC.
> > > > >> > > > > >>>> I known Bin spent some time working on AutoFDO, has
> he
> > > came
> > > > >> up to
> > > > >> > > > > >>> something?
> > > > >> > > > > >>>
> > > > >> > > > > >>> The GCC side of auto-FDO is not that hard. We have
> most
> > > of
> > > > >> > > > > >>> infrastructure in place, but stopping point for me was
> > > always
> > > > >> > > > > difficulty
> > > > >> > > > > >>> to get gcov-tool working. If some maintainer steps
> up, I
> > > > >> think I
> > > > >> > > can
> > > > >> > > > > >>> fix GCC side.
> > > > >> > > > > >>>
> > > > >> > > > > >>> I am bit unsure how important feature it is - we have
> FDO
> > > that
> > > > >> > > works
> > > > >> > > > > >>> quite well for most users but I know there are some
> users
> > > of
> > > > >> the
> > > > >> > > LLVM
> > > > >> > > > > >>> implementation and there is potential to tie this with
> > > other
> > > > >> > > hardware
> > > > >> > > > > >>> events to asist i.e. if conversion (where one wants to
> > > know
> > > > >> how
> > > > >> > > well
> > > > >> > > > > CPU
> > > > >> > > > > >>> predicts the jump rather than just the jump
> probability)
> > > > >> which I
> > > > >> > > always
> > > > >> > > > > >>> found potentially interesting.
> > > > >> > > > > >>>
> > > > >> > > > > >>> Honza
> > > > >> > > > > >>>>
> > > > >> > > > > >>>> Martin
> > > > >> > > > > >>>>
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> Thanks,
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>> Eugene
> > > > >> > > > > >>>>>
> > > > >> > > > > >>>>
> > > > >> > > > > >>>
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > >
> > > > >>
> > > > >
> > >
>
next prev parent reply other threads:[~2021-04-25 23:18 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 19:58 Eugene Rozenfeld
2021-04-22 20:16 ` Martin Liška
2021-04-22 22:29 ` Jan Hubicka
2021-04-23 4:14 ` Xinliang David Li
2021-04-23 7:00 ` Richard Biener
2021-04-23 7:18 ` Martin Liška
2021-04-23 9:32 ` Richard Biener
2021-04-23 16:41 ` Xinliang David Li
2021-04-23 16:54 ` Jan Hubicka
2021-04-23 17:04 ` Xinliang David Li
2021-04-23 17:16 ` Jan Hubicka
2021-04-23 17:27 ` Xinliang David Li
2021-04-23 17:28 ` Xinliang David Li
2021-04-23 19:28 ` Jan Hubicka
2021-04-23 19:58 ` Xinliang David Li
2021-04-25 19:07 ` Jan Hubicka
2021-04-25 23:18 ` Xinliang David Li [this message]
2021-04-26 4:22 ` Wei Mi
2021-04-26 15:11 ` Andi Kleen
2021-04-26 16:57 ` Xinliang David Li
2021-04-26 18:00 ` Andi Kleen
2021-04-26 18:05 ` Xinliang David Li
2021-04-26 18:40 ` Hongtao Yu
2021-04-26 19:13 ` Andi Kleen
2021-04-29 5:40 ` Andi Kleen
2021-04-29 14:45 ` 172060045
2021-04-30 21:43 ` Andi Kleen
2021-05-08 11:25 ` 172060045
2021-05-09 16:28 ` Andi Kleen
2021-05-09 17:01 ` Jan Hubicka
2021-05-10 15:36 ` Andi Kleen
2021-05-10 16:55 ` Joseph Myers
2021-05-10 17:21 ` Andi Kleen
2022-07-26 20:12 ` Eugene Rozenfeld
2022-07-26 22:37 ` David Edelsohn
2022-07-27 7:26 ` Jan Hubicka
2022-07-27 18:30 ` [EXTERNAL] " Eugene Rozenfeld
2022-07-27 18:24 ` Eugene Rozenfeld
2022-07-27 1:31 ` Xionghu Luo
2022-07-27 1:41 ` Xionghu Luo
2022-07-27 18:38 ` [EXTERNAL] " Eugene Rozenfeld
2021-05-10 23:46 ` Wei Mi
2021-05-22 1:28 ` [EXTERNAL] " Eugene Rozenfeld
2021-05-22 16:36 ` Wei Mi
2021-05-25 1:39 ` Eugene Rozenfeld
2021-05-25 3:11 ` Wei Mi
2021-05-25 3:33 ` Eugene Rozenfeld
2021-05-25 3:54 ` Wei Mi
2021-05-25 7:01 ` Eugene Rozenfeld
2021-05-25 16:16 ` Wei Mi
2021-05-25 20:49 ` Eugene Rozenfeld
2021-05-26 3:06 ` Wei Mi
2021-05-26 23:39 ` Eugene Rozenfeld
2021-05-27 2:51 ` Wei Mi
2021-06-12 1:14 ` Eugene Rozenfeld
2021-06-14 17:00 ` Wei Mi
2021-04-23 17:20 ` Jan Hubicka
2021-04-23 16:36 ` Xinliang David Li
2021-04-30 18:48 ` [EXTERNAL] " Eugene Rozenfeld
2021-04-30 21:45 ` Andi Kleen
2021-06-24 21:45 ` Eugene Rozenfeld
2021-04-23 1:46 ` Bin.Cheng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAAkRFZLfGd3WStyQ6beJXb4PaTm1PjdYwVBoit2ijbn5r-n_xQ@mail.gmail.com \
--to=davidxl@google.com \
--cc=Eugene.Rozenfeld@microsoft.com \
--cc=gcc@gcc.gnu.org \
--cc=hubicka@ucw.cz \
--cc=wmi@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).