From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf1-x129.google.com (mail-lf1-x129.google.com [IPv6:2a00:1450:4864:20::129]) by sourceware.org (Postfix) with ESMTPS id D47BA384840A for ; Tue, 25 May 2021 03:54:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D47BA384840A Received: by mail-lf1-x129.google.com with SMTP id j10so43780118lfb.12 for ; Mon, 24 May 2021 20:54:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=6R53mr3PjVWZr0s0sQhjX5J9WGtjyjAwouAj7e+hoyE=; b=hnDE84XQvGsPoUCTbV8H7E8J8HjfbgIXrpPNgFfnoiBB+habZcjZyXGhq4coeIQK77 Hr3mI//NZvaxw0U7fe70Fl1SGjqruN3EIWnt3GcQJwBYPvBROSwrTiY9O1KHxFQ9hjku UJhy21IqbSRW554NNRbnGxKTXSqlpar5xFSvibjKlsc/XOAHlf8SnyGlzyy387pEIX2p B6NJ7SyypHzG+2NvSc/bVpnkcCgdNAJYAD8qRk9JDcipgCMRWk5y13QDKcueU+cx5xvS M8OJv9XQJJTHWId8lFHoL0GsH3zBCy0UWls6kK1jXxUNN4/nDb/unZ1AvW5tUyKDAWvF HMGA== X-Gm-Message-State: AOAM531jI9+j/LtpcabkuGNuXIrYN27zTUozVwwQFxPfz/YoXljgVJTO LSG25vGyTZkFw4UZXofKC5SR3o7B78izBAQyY1IBAg== X-Google-Smtp-Source: ABdhPJwus/2fv0jN9S+gViO9pj12mpxYehSVAawS1s2dlbugxSOcn5rNfVyMwsl+ZssNNtuqG3ETWJBPxklWlpQp1nQ= X-Received: by 2002:a05:6512:228f:: with SMTP id f15mr13231854lfu.537.1621914877291; Mon, 24 May 2021 20:54:37 -0700 (PDT) MIME-Version: 1.0 References: <20210423192834.GA1949@kam.mff.cuni.cz> <20210425190754.GA88857@kam.mff.cuni.cz> <87a6plulkz.fsf@linux.intel.com> <20210426180011.GA1401198@tassilo.jf.intel.com> <20210429054025.GB4032392@tassilo.jf.intel.com> In-Reply-To: From: Wei Mi Date: Mon, 24 May 2021 20:54:25 -0700 Message-ID: Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC To: Eugene Rozenfeld Cc: Andi Kleen , Hongtao Yu , Xinliang David Li , Jan Hubicka , "gcc@gcc.gnu.org" , Wenlei He Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-16.6 required=5.0 tests=BAYES_00, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, KAM_LINEPADDING, KAM_LOTSOFHASH, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2021 03:54:41 -0000 It isn't exposed on my platform either. Looks like a bug in perf_data_converter (i.e., quipper). Could you try adding #include in third_party/perf_data_converter/src/quipper/huge_page_deducer.cc and see if it fixes the problem? If it works, I will need to file a bug against perf_data_converter. Thanks, Wei. On Mon, May 24, 2021 at 8:33 PM Eugene Rozenfeld wrote: > > That fixed the error I saw before but the build still fails. The errors s= tart with > > > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja > > [2/217] Building CXX object CMakeFiles/quipper_perf.dir/third_party/perf_= data_converter/src/quipper/huge_page_deducer.cc.o > > FAILED: CMakeFiles/quipper_perf.dir/third_party/perf_data_converter/src/q= uipper/huge_page_deducer.cc.o > > /usr/bin/c++ -I../third_party/perf_data_converter/src -I../third_party/= perf_data_converter/src/quipper -I../ -I../third_party/glog/src -I../third_= party/abseil -I../util -I. -Ithird_party/glog -std=3Dgnu++1z -MD -MT CMakeF= iles/quipper_perf.dir/third_party/perf_data_converter/src/quipper/huge_page= _deducer.cc.o -MF CMakeFiles/quipper_perf.dir/third_party/perf_data_convert= er/src/quipper/huge_page_deducer.cc.o.d -o CMakeFiles/quipper_perf.dir/thir= d_party/perf_data_converter/src/quipper/huge_page_deducer.cc.o -c ../third_= party/perf_data_converter/src/quipper/huge_page_deducer.cc > > ../third_party/perf_data_converter/src/quipper/huge_page_deducer.cc:144:2= 6: error: =E2=80=98unordered_map=E2=80=99 in namespace =E2=80=98std=E2=80= =99 does not name a template type > > using container =3D std::unordered_map; > > ^~~~~~~~~~~~~ > > > > > > > > From: Wei Mi > Sent: Monday, May 24, 2021 8:12 PM > To: Eugene Rozenfeld > Cc: Andi Kleen ; Hongtao Yu ; Xinliang Da= vid Li ; Jan Hubicka ; gcc@gcc.gnu.org;= Wenlei He > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > > > Sorry, I added dependency for create_gcov but missed it for dump_gcov. Fi= xed it at https://github.com/google/autofdo/commit/6ca36cdc30986f13583a3aef= 3e27746ca4fc5bf6. > > > > Thanks, > > Wei. > > > > On Mon, May 24, 2021 at 6:39 PM Eugene Rozenfeld wrote: > > Thank you Wei. Looks like something is still missing. This time perf_data= .pb.h is not found. I'm getting the error below (on Ubuntu 18.04 with cmake= 3.12.1): > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja > [1/241] Building CXX object CMakeFiles/dump_gcov_lib.dir/profile.cc.o > FAILED: CMakeFiles/dump_gcov_lib.dir/profile.cc.o > /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil -I= ../third_party/perf_data_converter/src -I../third_party/perf_data_converter= /src/quipper -I../util -I. -Ithird_party/glog -std=3Dgnu++1z -MD -MT CMakeF= iles/dump_gcov_lib.dir/profile.cc.o -MF CMakeFiles/dump_gcov_lib.dir/profil= e.cc.o.d -o CMakeFiles/dump_gcov_lib.dir/profile.cc.o -c ../profile.cc > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:18:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: warning= : "DISALLOW_COPY_AND_ASSIGN" redefined > #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > In file included from ../profile.h:14:0, > from ../profile.cc:5: > ../base/macros.h:114:0: note: this is the location of the previous defini= tion > #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:18:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: warnin= g: "arraysize" redefined > #define arraysize(x) (sizeof(x) / sizeof(*x)) > > In file included from ../profile.h:14:0, > from ../profile.cc:5: > ../base/macros.h:162:0: note: this is the location of the previous defini= tion > #define arraysize(array) (sizeof(ArraySizeHelper(array))) > > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:21:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/compat/proto.h:14:10: fata= l error: perf_data.pb.h: No such file or directory > #include "perf_data.pb.h" > ^~~~~~~~~~~~~~~~ > compilation terminated. > [6/241] Building CXX object CMakeFiles/dump_gcov_lib.dir/symbol_map.cc.o > ninja: build stopped: subcommand failed. > > Thanks, > > Eugene > > From: Wei Mi > Sent: Saturday, May 22, 2021 9:37 AM > To: Eugene Rozenfeld > Cc: Andi Kleen ; Hongtao Yu ; Xinliang Da= vid Li ; Jan Hubicka ; gcc@gcc.gnu.org;= Wenlei He > Subject: Re: [EXTERNAL] Re: State of AutoFDO in GCC > > It is a proto library build dependency issue which didn't expose on my pl= atform. I fix it at https://nam06.safelinks.protection.outlook.com/?url=3Dh= ttps%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo%2Fcommit%2F98269aee9674cc885cc5e= b1bd917eb2d12731710&data=3D04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C414= e79d9b075428ceffc08d91d3fd7ff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C= 637572982342270322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu= MzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DTxMivTXj9w0W5OFLunWcOpqf9= wefrrUd0m3RqV8JLaA%3D&reserved=3D0. Please try again. > > Thanks, > Wei. > > On Fri, May 21, 2021 at 6:28 PM Eugene Rozenfeld wrote: > I tried following the instructions in "2.2 Build autofdo tool for gcc" in= https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub= .com%2Fgoogle%2Fautofdo%23readme&data=3D04%7C01%7CEugene.Rozenfeld%40micros= oft.com%7C414e79d9b075428ceffc08d91d3fd7ff%7C72f988bf86f141af91ab2d7cd011db= 47%7C1%7C0%7C637572982342280318%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDA= iLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DKBPIcf5k1v23= 0tbLke9%2FdSt25nbPrHhpBvPb8PiebPY%3D&reserved=3D0and got build failures: > > eugene@eugene-Virtual-Machine:~/autofdo1/build$ ninja > [1/228] Building CXX object CMakeFiles/create_gcov_lib.dir/profile.cc.o > FAILED: CMakeFiles/create_gcov_lib.dir/profile.cc.o > /usr/bin/c++ -I../ -I../third_party/glog/src -I../third_party/abseil -I.= ./third_party/perf_data_converter/src -I../third_party/perf_data_converter/= src/quipper -I../util -I. -Ithird_party/glog -std=3Dgnu++1z -MD -MT CMakeFi= les/create_gcov_lib.dir/profile.cc.o -MF CMakeFiles/create_gcov_lib.dir/pro= file.cc.o.d -o CMakeFiles/create_gcov_lib.dir/profile.cc.o -c ../profile.cc > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:18:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/base/macros.h:8:0: warning= : "DISALLOW_COPY_AND_ASSIGN" redefined > #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > In file included from ../profile.h:14:0, > from ../profile.cc:5: > ../base/macros.h:114:0: note: this is the location of the previous defini= tion > #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:18:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/base/macros.h:12:0: warnin= g: "arraysize" redefined > #define arraysize(x) (sizeof(x) / sizeof(*x)) > In file included from ../profile.h:14:0, > from ../profile.cc:5: > ../base/macros.h:162:0: note: this is the location of the previous defini= tion > #define arraysize(array) (sizeof(ArraySizeHelper(array))) > In file included from ../third_party/perf_data_converter/src/quipper/perf= _parser.h:21:0, > from ../sample_reader.h:18, > from ../profile.h:15, > from ../profile.cc:5: > ../third_party/perf_data_converter/src/quipper/compat/proto.h:16:10: fata= l error: perf_stat.pb.h: No such file or directory > #include "perf_stat.pb.h" > ^~~~~~~~~~~~~~~~ > compilation terminated. > > What is supposed to generate perf_stat.pb.h? > > Thanks, > > Eugene > > From: Wei Mi > Sent: Monday, May 10, 2021 4:47 PM > To: Andi Kleen > Cc: Hongtao Yu ; Xinliang David Li ; Jan Hubicka ; mailto:gcc@gcc.gnu.org; Euge= ne Rozenfeld ; Wenlei He > Subject: [EXTERNAL] Re: State of AutoFDO in GCC > > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithu= b.com%2Fgoogle%2Fautofdo&data=3D04%7C01%7CEugene.Rozenfeld%40microsoft.com%= 7C414e79d9b075428ceffc08d91d3fd7ff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7= C0%7C637572982342290320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo= iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DUWJlvkaOdGT5p8ZLmeZY= kZ6lPhSmdmzc0tEnBULcNsY%3D&reserved=3D0 has been updated. Now create_gcov/d= ump_gcov are added back and can be built separately. > > Please look at "2.2 Build autofdo tool for gcc" in https://nam06.safelink= s.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgithub.com%2Fgoogle%2Fautofdo= %23readme&data=3D04%7C01%7CEugene.Rozenfeld%40microsoft.com%7C414e79d9b0754= 28ceffc08d91d3fd7ff%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6375729823= 42290320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi= I6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3D3fXLah0%2B0YHbo%2BHlbcgUFwlFQD9YwCU= ZApmZHoasiW0%3D&reserved=3D0 > On Wed, Apr 28, 2021 at 10:40 PM Andi Kleen w= rote: > > > > On Mon, Apr 26, 2021 at 06:40:56PM +0000, Hongtao Yu wrote: > > > Andi, thanks for pointing out the perf script issues. Can you plea= se > > > elaborate a bit on the exact issue you have seen? We've been using > > > specific output of perf script such as mmap, LBR and callstack eve= nts > > > filtered by process id. It works fine so far but may certainly hit= issues > > > in the future with extended uses. > > > > Okay I took a look at the latest autofdo now. It seems to be basically > > a LLVM project now that depends on LLVM to even build with all kinds > > of dependency hell on some old LLVM version and other packages. > > > > I guess gcc will really need a replacement that doesn't pull in > > all of LLVM if it wants to continue supporting autofdo. > > > > I'm myself unable to build now. > > > > I'm using the old version I had a git fork of and that > > was before all of this. I added a patch to make it work > > with the latest perf by ignoring increased perf_attr > > and unknown perf events. > > > > Honza please use > > > > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgit= hub.com%2Fandikleen%2Fautofdo&data=3D04%7C01%7CEugene.Rozenfeld%40microsoft= .com%7C414e79d9b075428ceffc08d91d3fd7ff%7C72f988bf86f141af91ab2d7cd011db47%= 7C1%7C0%7C637572982342300314%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC= JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DaNe2qZeEjsE2%2F= m7ZbaIFPFAcnF3wtQEkOJr0fG5tHWo%3D&reserved=3D0 -b perf-future > > > > for testing. > > > > -Andi > >