From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by sourceware.org (Postfix) with ESMTPS id DE2EA3896819 for ; Mon, 3 May 2021 09:01:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org DE2EA3896819 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rguenther@suse.de X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A7DFEAEE5; Mon, 3 May 2021 09:01:40 +0000 (UTC) Date: Mon, 3 May 2021 11:01:40 +0200 (CEST) From: Richard Biener To: David Faust cc: "Jose E. Marchesi" , Indu Bhagat , gcc-patches@gcc.gnu.org Subject: Re: [PATCH V7 4/7] CTF/BTF debug formats In-Reply-To: <51ebab8c-4474-f3fd-79cf-1a07edec2073@oracle.com> Message-ID: References: <875z03kb0p.fsf@oracle.com> <51ebab8c-4474-f3fd-79cf-1a07edec2073@oracle.com> User-Agent: Alpine 2.21 (LSU 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2021 09:01:43 -0000 On Fri, 30 Apr 2021, David Faust wrote: > > On 4/30/21 9:11 AM, Jose E. Marchesi via Gcc-patches wrote: > > > >>> For a moment, for the sake of this question, if we establish that CTF/BTF > >>> generation always feeds off DWARF DIEs (so there is no need to access > >>> type/decl tree nodes), what will it take to keep LTO support while keeping > >>> ctf_debug_finalize in dwarf2out_finish ? > >> > >> I don't think it's possible without major surgery. Now, it looks like > >> you want to emit .ctf at the LTO compile stage and BTF at the LTRANS > >> stage. Note you do not have access to the CTF data produced at the > >> LTRANS stage. What DWARF has access to is pointers (symbol name + offset) > >> to the early DWARF generated DIEs for GCCs tree decls from what it > >> adds DW_AT_abstract_origin references and then amending those entities > >> with furthr DWARF attributes. > >> > >> Now, the "major surgery" could for example entail streaming the CTF > >> data you produce from compile through WPA to the LTRANS stage. Or > >> alternatively simply reading it from the compile time produced objects > >> by means of the DWARF DIE references. I don't know what kind of (if any) > >> references you need to produce from BTF to the CTF data (I suppose they > >> might share indexes into a same symbol table...). > > > > I am thinking, given that we have both "early DWARF" and "late DWARF", > > we may want to have three init/fini hooks for debug formats instead of > > two: > > > > DEBUG_FORMAT_initialize (called in dwarf2out_early_finish) > > > > [... traverse early DWARF (DEBUG_FORMAT_do_die) ...] > > > > DEBUG_FORMAT_early_finish (called in dwarf2out_early_finish) > > DEBUG_FORMAT_finish (called in dwarf2out_finish) > > > > Note that CTF and BTF share the ctfc internal data structure (which is > > built from traversing the early DWARF) but these are otherwise distinct > > debug formats: CTF can be completely emitted in ctf_early_finish, > > whereas BTF needs additional information that is only available in > > ctf_finish.) > > > > I like this idea. > > As Indu mentioned, the change to writing CTF/BTF in dwarf2out_finish > rather than dwar2out_early_finish is motivated purely by the needs of BPF. > > Specifically, it is to support BPF Compile Once - Run Everywhere (BPF > CO-RE). The BPF backend needs BTF information to generate CO-RE > 'relocation' records, and it needs to add to the BTF string table. The > relocation records themselves are placed in a .BTF.ext section, but > refer to BTF records by ID. > > Note that CO-RE only comes into play when generating BTF information for > a BPF target. Otherwise, the BTF information can also be emitted in > early_finish. That is, BTF is effectively "late DWARF" ONLY for the BPF > target. Of course, this is also the usual use case for BTF. > > I do not have much experience with LTO. I am still trying to digest the > LTO section of the manual. But from what what I do understand, and what > is discussed here, I think the following may work: > > 1. Always emit CTF in early_finish. There is no reason to delay it. This > simplifies both LTO and non-LTO cases. > > 2. For non-BPF targets, emit BTF in early_finish as well. > > 3. For non-LTO + BTF + BPF, emit BTF in in (late) finish, to support > generating CO-RE information in the BPF backend. > > 4. For LTO + BTF + BPF, either emit BTF in early_finish then read it > back later, or stream it through WPA to LTRANS stage. I am not sure > exactly what this streaming means. In either case, end up with enough > information at LTRANS stage to re-emit the BTF, now with CO-RE information. > > Does the above make any sense? Is this overly complicated? So DWARF would likely emit BTF early and amend it late with the CO-RE information. Whatever exactly CO-RE is - I'm assuming it's backend stuff like DWARF location expressions. Note when emitting CTF/BTF early there's still the issue of fat objects, but as said, eventually the "early" CTF/BTF can be literally shared. Of course the "fat" part would have the CO-RE stuff emitted at compile-time and the "lto" part at link time. I suppose we might want to introduce a target hook that says whether a target supports fat LTO objects or not, side-stepping this issue. Richard. > Thank you very much Richard for the review, and for raising this LTO > discussion. > > David > -- Richard Biener SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)