From: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
To: Wei Mi <wmi@google.com>
Cc: Andi Kleen <ak@linux.intel.com>, Hongtao Yu <hoy@fb.com>,
Xinliang David Li <davidxl@google.com>,
Jan Hubicka <hubicka@ucw.cz>, "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>,
Wenlei He <wenlei@fb.com>
Subject: RE: [EXTERNAL] Re: State of AutoFDO in GCC
Date: Tue, 25 May 2021 07:01:06 +0000 [thread overview]
Message-ID: <MWHPR21MB0798525139634A6EA53606B991259@MWHPR21MB0798.namprd21.prod.outlook.com> (raw)
In-Reply-To: <CA+4CFy7Mp8iXXa_u+fpY3i5ZWzVHxw5QsVsGrM_ffqS-++3WCQ@mail.gmail.com>
That eliminates the previous error but there is a new one:
eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja
[3/199] Building CXX object CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o
FAILED: CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o
/usr/bin/c++ -I../third_party/perf_data_converter/src -I../third_party/perf_data_converter/src/quipper -I../ -I../third_party/glog/src -I../third_party/abseil -I../util -I. -Ithird_party/glog -std=gnu++1z -MD -MT CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o -MF CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o.d -o CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/perf_reader.cc.o -c ../third_party/perf_data_converter/src/quipper/perf_reader.cc
../third_party/perf_data_converter/src/quipper/perf_reader.cc: In member function 'bool quipper::PerfReader::ReadCPUTopologyMetadata(quipper::DataReader*, size_t)':
../third_party/perf_data_converter/src/quipper/perf_reader.cc:1518:46: error: no match for 'operator[]' (operand types are 'const google::protobuf::RepeatedField<unsigned int>' and 'int')
nrcpus = proto_uint32_metadata.data()[0];
-----Original Message-----
From: Wei Mi <wmi@google.com>
Sent: Monday, May 24, 2021 8:54 PM
To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
Cc: Andi Kleen <ak@linux.intel.com>; Hongtao Yu <hoy@fb.com>; Xinliang David Li <davidxl@google.com>; Jan Hubicka <hubicka@ucw.cz>; gcc@gcc.gnu.org; Wenlei He <wenlei@fb.com>
Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC
It isn't exposed on my platform either. Looks like a bug in perf_data_converter (i.e., quipper). Could you try adding #include <unordered_map> in third_party/perf_data_converter/src/quipper/huge_page_deducer.cc and see if it fixes the problem? If it works, I will need to file a bug against perf_data_converter.
Thanks,
Wei.
On Mon, May 24, 2021 at 8:33 PM Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com> wrote:
>
> That fixed the error I saw before but the build still fails. The
> errors start with
>
>
>
> eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja
>
> [2/217] Building CXX object
> CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quippe
> r/huge_page_deducer.cc.o
>
> FAILED:
> CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quippe
> r/huge_page_deducer.cc.o
>
> /usr/bin/c++ -I../third_party/perf_data_converter/src -I../third_party/perf_data_converter/src/quipper -I../ -I../third_party/glog/src -I../third_party/abseil -I../util -I. -Ithird_party/glog -std=gnu++1z -MD -MT CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o -MF CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o.d -o CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o -c ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc
>
> ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc:14
> 4:26: error: 'unordered_map' in namespace 'std' does not name a
> template type
>
> using container = std::unordered_map<key_t, value_t>;
>
> ^~~~~~~~~~~~~
>
>
>
>
>
>
>
> From: Wei Mi <wmi@google.com>
> Sent: Monday, May 24, 2021 8:12 PM
> To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
> Cc: Andi Kleen <ak@linux.intel.com>; Hongtao Yu <hoy@fb.com>; Xinliang
> David Li <davidxl@google.com>; Jan Hubicka <hubicka@ucw.cz>;
> gcc@gcc.gnu.org; Wenlei He <wenlei@fb.com>
> Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC
>
>
>
> Sorry, I added dependency for create_gcov but missed it for dump_gcov. Fixed it at https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F6ca36cdc30986f13583a3aef3e27746ca4fc5bf6&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575116816277204%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hwJ%2BG64Yw%2BEGMAV7kzilOjAOkHoOv3TQpdqkzcHGO%2FM%3D&reserved=0.
>
>
>
> Thanks,
>
> Wei.
>
>
>
> On Mon, May 24, 2021 at 6:39 PM Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com> wrote:
>
> Thank you Wei. Looks like something is still missing. This time perf_data.pb.h is not found. I'm getting the error below (on Ubuntu 18.04 with cmake 3.12.1):
>
> eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/241] Building
> CXX object CMakeFiles/dump_gcov_lib.dir/profile.cc.o
> FAILED: CMakeFiles/dump_gcov_lib.dir/profile.cc.o
> /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil -I../third_party/perf_data_converter/src -I../third_party/perf_data_converter/src/quipper -I../util -I. -Ithird_party/glog -std=gnu++1z -MD -MT CMakeFiles/dump_gcov_lib.dir/profile.cc.o -MF CMakeFiles/dump_gcov_lib.dir/profile.cc.o.d -o CMakeFiles/dump_gcov_lib.dir/profile.cc.o -c ../profile.cc
> In file included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0:
> warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define
> DISALLOW_COPY_AND_ASSIGN(TypeName) \
>
> In file included from ../profile.h:14:0,
> from ../profile.cc:5:
> ../base/macros.h:114:0: note: this is the location of the previous
> definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \
>
> In file included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0:
> warning: "arraysize" redefined #define arraysize(x) (sizeof(x) /
> sizeof(*x))
>
> In file included from ../profile.h:14:0,
> from ../profile.cc:5:
> ../base/macros.h:162:0: note: this is the location of the previous
> definition #define arraysize(array) (sizeof(ArraySizeHelper(array)))
>
> In file included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/compat/proto.h:14:10:
> fatal error: perf_data.pb.h: No such file or directory #include "perf_data.pb.h"
> ^~~~~~~~~~~~~~~~
> compilation terminated.
> [6/241] Building CXX object
> CMakeFiles/dump_gcov_lib.dir/symbol_map.cc.o
> ninja: build stopped: subcommand failed.
>
> Thanks,
>
> Eugene
>
> From: Wei Mi <wmi@google.com>
> Sent: Saturday, May 22, 2021 9:37 AM
> To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
> Cc: Andi Kleen <ak@linux.intel.com>; Hongtao Yu <hoy@fb.com>; Xinliang
> David Li <davidxl@google.com>; Jan Hubicka <hubicka@ucw.cz>;
> gcc@gcc.gnu.org; Wenlei He <wenlei@fb.com>
> Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC
>
> It is a proto library build dependency issue which didn't expose on my platform. I fix it at https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F98269aee9674cc885cc5eb1bd917eb2d12731710&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PikMs8RtJHBsTUqdjCVWAKNSC5Rls%2B71pSEhq4z0XsM%3D&reserved=0. Please try again.
>
> Thanks,
> Wei.
>
> On Fri, May 21, 2021 at 6:28 PM Eugene Rozenfeld <mailto:Eugene.Rozenfeld@microsoft.com> wrote:
> I tried following the instructions in "2.2 Build autofdo tool for gcc" in https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=EwHtN1OqRmBwL6qMgTc1lAU6cz5%2BJzQZxpd5nos11FM%3D&reserved=0 got build failures:
>
> eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja [1/228] Building
> CXX object CMakeFiles/create_gcov_lib.dir/profile.cc.o
> FAILED: CMakeFiles/create_gcov_lib.dir/profile.cc.o
> /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil
> -I../third_party/perf_data_converter/src -I../third_party/perf_data_converter/src/quipper -I../util -I. -Ithird_party/glog -std=gnu++1z -MD -MT CMakeFiles/create_gcov_lib.dir/profile.cc.o -MF CMakeFiles/create_gcov_lib.dir/profile.cc.o.d -o CMakeFiles/create_gcov_lib.dir/profile.cc.o -c ../profile.cc In file included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0:
> warning: "DISALLOW_COPY_AND_ASSIGN" redefined #define
> DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file included from ../profile.h:14:0,
> from ../profile.cc:5:
> ../base/macros.h:114:0: note: this is the location of the previous
> definition #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ In file
> included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:18:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0:
> warning: "arraysize" redefined #define arraysize(x) (sizeof(x) /
> sizeof(*x)) In file included from ../profile.h:14:0,
> from ../profile.cc:5:
> ../base/macros.h:162:0: note: this is the location of the previous
> definition #define arraysize(array) (sizeof(ArraySizeHelper(array)))
> In file included from ../third_party/perf_data_converter/src/quipper/perf_parser.h:21:0,
> from ../sample_reader.h:18,
> from ../profile.h:15,
> from ../profile.cc:5:
> ../third_party/perf_data_converter/src/quipper/compat/proto.h:16:10:
> fatal error: perf_stat.pb.h: No such file or directory #include "perf_stat.pb.h"
> ^~~~~~~~~~~~~~~~
> compilation terminated.
>
> What is supposed to generate perf_stat.pb.h?
>
> Thanks,
>
> Eugene
>
> From: Wei Mi <mailto:wmi@google.com>
> Sent: Monday, May 10, 2021 4:47 PM
> To: Andi Kleen <mailto:ak@linux.intel.com>
> Cc: Hongtao Yu <mailto:hoy@fb.com>; Xinliang David Li
> <mailto:davidxl@google.com>; Jan Hubicka <mailto:hubicka@ucw.cz>;
> mailto:gcc@gcc.gnu.org; Eugene Rozenfeld
> <mailto:Eugene.Rozenfeld@microsoft.com>; Wenlei He
> <mailto:wenlei@fb.com>
> Subject: [EXTERNAL] Re: State of AutoFDO in GCC
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=a9gGf48MYY2jzbSucOAlhsmdLuYt%2BcM8V5hvGoA39Ms%3D&reserved=0 has been updated. Now create_gcov/dump_gcov are added back and can be built separately.
>
> Please look at "2.2 Build autofdo tool for gcc" in
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fgoogle%2Fautofdo%23readme&data=04%7C01%7CEugene.Rozenfeld
> %40microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af
> 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1
> 000&sdata=EwHtN1OqRmBwL6qMgTc1lAU6cz5%2BJzQZxpd5nos11FM%3D&res
> erved=0 On Wed, Apr 28, 2021 at 10:40 PM Andi Kleen
> <mailto:ak@linux.intel.com> wrote:
> >
> > On Mon, Apr 26, 2021 at 06:40:56PM +0000, Hongtao Yu wrote:
> > > Andi, thanks for pointing out the perf script issues. Can you please
> > > elaborate a bit on the exact issue you have seen? We've been using
> > > specific output of perf script such as mmap, LBR and callstack events
> > > filtered by process id. It works fine so far but may certainly hit issues
> > > in the future with extended uses.
> >
> > Okay I took a look at the latest autofdo now. It seems to be
> > basically a LLVM project now that depends on LLVM to even build with
> > all kinds of dependency hell on some old LLVM version and other packages.
> >
> > I guess gcc will really need a replacement that doesn't pull in all
> > of LLVM if it wants to continue supporting autofdo.
> >
> > I'm myself unable to build now.
> >
> > I'm using the old version I had a git fork of and that was before
> > all of this. I added a patch to make it work with the latest perf by
> > ignoring increased perf_attr and unknown perf events.
> >
> > Honza please use
> >
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > thub.com%2Fandikleen%2Fautofdo&data=04%7C01%7CEugene.Rozenfeld%4
> > 0microsoft.com%7C1bcb2fda4fce4f173c1808d91f30d1fc%7C72f988bf86f141af
> > 91ab2d7cd011db47%7C1%7C0%7C637575116816287200%7CUnknown%7CTWFpbGZsb3
> > d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D
> > %7C1000&sdata=Dilu5PUjpAbA6e6d6GBd5idoKLAMMr7H5eCve%2FGdC8g%3D&a
> > mp;reserved=0 -b perf-future
> >
> > for testing.
> >
> > -Andi
> >
next prev parent reply other threads:[~2021-05-25 7:01 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 19:58 Eugene Rozenfeld
2021-04-22 20:16 ` Martin Liška
2021-04-22 22:29 ` Jan Hubicka
2021-04-23 4:14 ` Xinliang David Li
2021-04-23 7:00 ` Richard Biener
2021-04-23 7:18 ` Martin Liška
2021-04-23 9:32 ` Richard Biener
2021-04-23 16:41 ` Xinliang David Li
2021-04-23 16:54 ` Jan Hubicka
2021-04-23 17:04 ` Xinliang David Li
2021-04-23 17:16 ` Jan Hubicka
2021-04-23 17:27 ` Xinliang David Li
2021-04-23 17:28 ` Xinliang David Li
2021-04-23 19:28 ` Jan Hubicka
2021-04-23 19:58 ` Xinliang David Li
2021-04-25 19:07 ` Jan Hubicka
2021-04-25 23:18 ` Xinliang David Li
2021-04-26 4:22 ` Wei Mi
2021-04-26 15:11 ` Andi Kleen
2021-04-26 16:57 ` Xinliang David Li
2021-04-26 18:00 ` Andi Kleen
2021-04-26 18:05 ` Xinliang David Li
2021-04-26 18:40 ` Hongtao Yu
2021-04-26 19:13 ` Andi Kleen
2021-04-29 5:40 ` Andi Kleen
2021-04-29 14:45 ` 172060045
2021-04-30 21:43 ` Andi Kleen
2021-05-08 11:25 ` 172060045
2021-05-09 16:28 ` Andi Kleen
2021-05-09 17:01 ` Jan Hubicka
2021-05-10 15:36 ` Andi Kleen
2021-05-10 16:55 ` Joseph Myers
2021-05-10 17:21 ` Andi Kleen
2022-07-26 20:12 ` Eugene Rozenfeld
2022-07-26 22:37 ` David Edelsohn
2022-07-27 7:26 ` Jan Hubicka
2022-07-27 18:30 ` [EXTERNAL] " Eugene Rozenfeld
2022-07-27 18:24 ` Eugene Rozenfeld
2022-07-27 1:31 ` Xionghu Luo
2022-07-27 1:41 ` Xionghu Luo
2022-07-27 18:38 ` [EXTERNAL] " Eugene Rozenfeld
2021-05-10 23:46 ` Wei Mi
2021-05-22 1:28 ` [EXTERNAL] " Eugene Rozenfeld
2021-05-22 16:36 ` Wei Mi
2021-05-25 1:39 ` Eugene Rozenfeld
2021-05-25 3:11 ` Wei Mi
2021-05-25 3:33 ` Eugene Rozenfeld
2021-05-25 3:54 ` Wei Mi
2021-05-25 7:01 ` Eugene Rozenfeld [this message]
2021-05-25 16:16 ` Wei Mi
2021-05-25 20:49 ` Eugene Rozenfeld
2021-05-26 3:06 ` Wei Mi
2021-05-26 23:39 ` Eugene Rozenfeld
2021-05-27 2:51 ` Wei Mi
2021-06-12 1:14 ` Eugene Rozenfeld
2021-06-14 17:00 ` Wei Mi
2021-04-23 17:20 ` Jan Hubicka
2021-04-23 16:36 ` Xinliang David Li
2021-04-30 18:48 ` [EXTERNAL] " Eugene Rozenfeld
2021-04-30 21:45 ` Andi Kleen
2021-06-24 21:45 ` Eugene Rozenfeld
2021-04-23 1:46 ` Bin.Cheng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MWHPR21MB0798525139634A6EA53606B991259@MWHPR21MB0798.namprd21.prod.outlook.com \
--to=eugene.rozenfeld@microsoft.com \
--cc=ak@linux.intel.com \
--cc=davidxl@google.com \
--cc=gcc@gcc.gnu.org \
--cc=hoy@fb.com \
--cc=hubicka@ucw.cz \
--cc=wenlei@fb.com \
--cc=wmi@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).