From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 29236 invoked by alias); 4 Aug 2017 08:40:03 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 29100 invoked by uid 89); 4 Aug 2017 08:40:01 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM,SPF_PASS autolearn=no version=3.3.2 spammy= X-HELO: mail-pg0-f44.google.com Received: from mail-pg0-f44.google.com (HELO mail-pg0-f44.google.com) (74.125.83.44) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 04 Aug 2017 08:39:59 +0000 Received: by mail-pg0-f44.google.com with SMTP id u185so5417596pgb.1 for ; Fri, 04 Aug 2017 01:39:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=e9yG/oWc3C++IcJ06D6vlv0zgnlmginfLCKUc5MYJJg=; b=P4OJx2TTmBI1IAYEsiGCobhi4jpKlRkEni9VO8QpluYBCwhjjMpPLqCFO8LeWqYbOk g1rNoZbtGw7/tjjD6YgZujlpC7LS+5Pl1KgxvBLx+pjPUFN5FAGW3aOJJWFxnQe0KNK0 Y29p5kiVqjvMzhEaRuc3+oyqvzTnq1BPm52kb4IN2r0a5QufTvBZnejhnuzEekr0TS5r trRliDUG5IGd952GOW2BpSgDVf1oVJYhaSd2IS1Q5HcgtLDU9IvjMGonPfM7glkNw6UN PZfa/jOs1vNLHeewyJNqmq4opPCWxNly0p8u/bgehM+qLdfzpKCMDmZQCKsD26aLkfOw KrXw== X-Gm-Message-State: AIVw113NPEmjjGBWI93kuOmm/ZHMYRhZ5qZTkiD+Vw9p32PcyEq8eh21 v0DnWwMRPlYcmEGSsFL1Moi0kxcSCA== X-Received: by 10.98.42.139 with SMTP id q133mr1659190pfq.39.1501835997591; Fri, 04 Aug 2017 01:39:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.133.2 with HTTP; Fri, 4 Aug 2017 01:39:56 -0700 (PDT) In-Reply-To: References: <20170721161538.7508-1-infinity0@pwned.gg> <3136125b-bd88-7c0b-504e-a4e4de545bbb@redhat.com> <4b6c844f-9a46-4a4b-48c5-9dfeac54b97f@pwned.gg> <565c6de9-2d85-24f9-faaf-6d8ac476b3a8@gmail.com> From: Yury Gribov Date: Fri, 04 Aug 2017 08:40:00 -0000 Message-ID: Subject: Re: [PING^4][PATCH v2] Generate reproducible output independently of the build-path To: Ximin Luo Cc: Jeff Law , GCC Patches Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2017-08/txt/msg00353.txt.bz2 On Thu, Aug 3, 2017 at 12:45 PM, Ximin Luo wrote: > Yury Gribov: >> On 03.08.2017 3:06, Ximin Luo wrote: >>> Jeff Law: >>>> On 07/21/2017 10:15 AM, Ximin Luo wrote: >>>>> (Please keep me on CC, I am not subscribed) >>>>> >>>>> >>>>> Proposal >>>>> =3D=3D=3D=3D=3D=3D=3D=3D >>>>> >>>>> This patch series adds a new environment variable BUILD_PATH_PREFIX_M= AP. When >>>>> this is set, GCC will treat this as extra implicit "-fdebug-prefix-ma= p=3D$value" >>>>> command-line arguments that precede any explicit ones. This makes the= final >>>>> binary output reproducible, and also hides the unreproducible value (= the source >>>>> path prefixes) from CFLAGS et. al. which many build tools (understand= ably) >>>>> embed as-is into their build output. >>>> I'd *really* avoid doing this with magic environment variables. Make = it >>>> a first class option to the compiler. Yes, it means projects that want >>>> this behavior have to arrange to pass that flag to their compiler, but >>>> IMHO that's much preferred over environment variables. >>>> >>>> Jeff >>>> >>> >>> Hi Jeff, >>> >>> If by "first class option" you meant a command-line flag, GCC *already = has* that (-fdebug-prefix-map) > and it wasn't enough to achieve reproducib= ility in many cases we tested. >> >> Shouldn't -fdebug-prefix-map be updated to use the same syntax as BUILD_= PATH_PREFIX_MAP? >> > > -fdebug-prefix-map is a CLI option and can be given multiple times, each = flag given is in the form of $from=3D$to where $from can't contain a '=3D'. > > BUILD_PATH_PREFIX_MAP is a single envvar that encodes a list-of-pairs of = the form $to=3D$from:$to=3D$from with some escaping for flexibility and to = support things like windows paths. Since it's a new envvar, Ian Jackson sug= gested $to=3D$from to emphasise the reproducible ($to) part. I liked the id= ea so I implemented it like that. (We did a lot of bikeshedding over on the= rb-general mailing list about the exact format and this is what we settled= on, I'd like to avoid getting into that again but would nevertheless do it= , if it's necessary to get this patch accepted.) > > Because -fdebug-prefix-map currently only encodes one $from=3D$to pair, i= t would be a very disruptive and highly backward-incompatible change to mak= e it use the same syntax as B_P_P_M. A slightly less disruptive but still b= ackward-incompatible change would be to make it encode a single $to=3D$from= pair, but I don't really see the advantage to doing so - what were your th= oughts on this? I believe it would much easier to reason about environment variable behavior when it boils down to "prepend some standard flag to command-line flags". It would also simplify maintenance of local compiler patch as core functionality can be merged to mainline GCC whereas debatable environment variable part stays in the distro. > If by "first class option" you meant a command-line flag, GCC *already ha= s* that (-fdebug-prefix-map) and it wasn't enough to achieve reproducibilit= y in many cases we tested. > dpkg-buildflags actually already adds these flags to CFLAGS CXXFLAGS etc = on Debian. However, with this patch using the environment variable, we are = able to reproduce 1800 more packages out of 26000. Just curious, why -fdebug-prefix-map (maybe modified to support multiple renames) was not enough for these packages (and why they can't be fixed instead)? -Y