From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by sourceware.org (Postfix) with ESMTPS id 7B98E3858C2F for ; Mon, 15 Aug 2022 20:29:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7B98E3858C2F Received: by mail-wm1-x32b.google.com with SMTP id m10-20020a05600c3b0a00b003a603fc3f81so458270wms.0 for ; Mon, 15 Aug 2022 13:29:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state :from:to:cc; bh=F2lccXv3ESHEyAui8FXBvZHXBDaALbWigpL1g7I6wfM=; b=uKorE0B6tpD1pqEzbQJj0wfYeztn6nRh1UoDsYwKhZx8HdYa6sa7bFfVLRXqZfHgHL Z59yRnYAP7q9T1l7rKFjzrxuPd4QoHRBGHmxuliB0DGJYOKBGre86r1QE+eoyqHsCNif e/X127ujvjHvmvo/IBQBw2aLegvD9g7XlzWMz3EsJBHGz4BbmjSV38QbZlfhhk8SMgnz CQkBwOvGS3l129tGOk89Knv/ERQWvX76sG+0Ev1L7kamRlMWtEg20/HWXxcuMKd0ejQX yn76GSQCznsW+vRxwpFWhxd3Kyc6qaWl7S7D3Bzbpl/YQ27lRv8dkkVz2AsygIOiL9f0 4jPw== X-Gm-Message-State: ACgBeo1muHF4BHvJxUm1rc51T2N56g8CFhX9/6Ov3Bef5ZzKoyV1UhMK oi4bhTNfhxmIDXxy0gUBJSgUzGJ10r/baw== X-Google-Smtp-Source: AA6agR69UfwopMq88ONhDx7PblQMcNmfRMTZ49bIG4EUdw4Mq8xylg5tAnIiZP9rE8f7xwdC8QiIQA== X-Received: by 2002:a7b:c402:0:b0:3a5:3c7:9741 with SMTP id k2-20020a7bc402000000b003a503c79741mr11280715wmi.72.1660595344204; Mon, 15 Aug 2022 13:29:04 -0700 (PDT) Received: from ?IPv6:2001:8b0:aba:5f3c:1640:f887:39f5:69b1? ([2001:8b0:aba:5f3c:1640:f887:39f5:69b1]) by smtp.gmail.com with ESMTPSA id u18-20020a05600c19d200b003a2cf1ba9e2sm11357034wmq.6.2022.08.15.13.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Aug 2022 13:29:03 -0700 (PDT) Message-ID: Subject: Re: Reproducible builds - supporting relative paths in *-prefix-map From: Richard Purdie To: Mark Wielaard Cc: gcc@gcc.gnu.org, Khem Raj , Sergio Durigan Junior Date: Mon, 15 Aug 2022 21:29:03 +0100 In-Reply-To: References: <1c0ab23fd0cc1bc34a229287529963526141327a.camel@linuxfoundation.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.1-0ubuntu1 MIME-Version: 1.0 X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Aug 2022 20:29:07 -0000 Hi Mark, Thanks for the reply. On Mon, 2022-08-15 at 21:55 +0200, Mark Wielaard wrote: > I added Sergio to the CC since he was looking at debuginfo/DWARF > relative paths created by -fdebug-prefix-map in Debian and having trouble > making them work correctly. Maybe he has some feedback how to make > this work. >=20 > On Mon, Aug 15, 2022 at 12:13:28PM +0100, Richard Purdie via Gcc wrote: > > I'm wondering if we'd be able to improve path handling in the -f*- > > prefix-map compiler options to cover relative paths? > > [...] > > In case it helps, my background on what I'm working on and why we'd > > find this useful follows:=C2=A0 > >=20 > > I work on Yocto Project which cross compiles complete software stacks, > > we use gcc heavily and the resulting builds are all around us. We care > > deeply about build reproducibility. > >=20 > > With autotools, we long ago realised that we were best off separating > > the source and the build output where we could and to do it we used > > relative paths, which removed a lot of problematic hardcoded paths in > > our output. > >=20 > > More recently we've been using -fdebug-prefix-map and -fmacro-prefix- > > map (we need both since some of binutils doesn't seem to support the > > combined version yet). We use the debug information (split separately) > > to support on target debugging but where relative paths were used, we > > miss information as the remapping either doesn't happen correctly or is > > mangled. We'd like to fix our debug output but we're finding that hard > > without relative path support for *prefix-map. >=20 > I might be misinterpreting the issue you are seeing. >=20 > But one problem with debuginfo/DWARF is that relative source paths > aren't clearly defined. If you move or install the executable or > (split) debug file out of the build directory a DWARF reader has no > way to know what the paths are relative to. >=20 > So for DWARF the paths always have to be absolute (they can still be > relative to the compilation dir (DW_AT_comp_dir), but at least that > has to be absolute (and the compiler should turn any relative path > into an absolute one or make sure they are relative to an absolute > compilation directory path). It gets slightly more complicated as we build in a directory separate to the source where we can. Some source files are generated source files and placed in the build directory whilst many are in the source directory. DW_AT_comp_dir can be set to one or the other but it is the relative path between build and source which is problematic. > The problem with that is a) it makes the binary (debuginfo) output > dependent on the build srcdir and builddir and b) to support > debugging/tracing/profiling tools the user has to install the (debug) > sources under those exact same directories on their local machine. >=20 > One way around this is to use -fdebug-prefix-map with an absolute > paths under which you will also install any source files. Or to use > debugedit [*] after the build to rewrite the build dir path to > something like /usr/debug/src/ which is > what some distros do. >=20 > e.g. Fedora has (one or more) binary package, a debuginfo package for > the .debug files (optionally installed under /usr/lib/debug) and > debugsource package for the debug source files (optionally installed > under /usr/debug/src). These packages can then also be used to > dynamically find any debug or source file when indexed by debuginfod > [**]. We split the debuginfo into a separate package. We also look at the sources it references and those go into a different separate package too. We support populating a remote debuginfod server with these or installing them onto the target. > Using known absolute paths generated with debugedit or > -fdebug-prefix-map makes sure the paths used in the debuginfo/DWARF > are always the same independent from the current srcdir or builddir to > make them reproducible. And the user/tools don't have to guess what > the relative paths are relative to. We have that working and set debug-prefix-map today. What is problematic is trying to recreate the relative paths on target between our source and build directories. Currently, most generated files in the build directory just don't get handled correctly on target. We'd like to fix that. There is currently no way to remap a relative path though, at least as far as I could determine. Cheers, Richard