Re: [PING^4][PATCH v2] Generate reproducible output independently of the build-path

[Matthias Klose <doko@debian.org>]

On 03.08.2017 12:05, Jeff Law wrote:
> On 08/02/2017 08:06 PM, Ximin Luo wrote:
>> Jeff Law:
>>> On 07/21/2017 10:15 AM, Ximin Luo wrote:
>>>> (Please keep me on CC, I am not subscribed)
>>>>
>>>>
>>>> Proposal
>>>> ========
>>>>
>>>> This patch series adds a new environment variable BUILD_PATH_PREFIX_MAP. When
>>>> this is set, GCC will treat this as extra implicit "-fdebug-prefix-map=$value"
>>>> command-line arguments that precede any explicit ones. This makes the final
>>>> binary output reproducible, and also hides the unreproducible value (the source
>>>> path prefixes) from CFLAGS et. al. which many build tools (understandably)
>>>> embed as-is into their build output.
>>> I'd *really* avoid doing this with magic environment variables.  Make it
>>> a first class option to the compiler.  Yes, it means projects that want
>>> this behavior have to arrange to pass that flag to their compiler, but
>>> IMHO that's much preferred over environment variables.
>>>
>>> Jeff
>>>
>>
>> Hi Jeff,
>>
>> If by "first class option" you meant a command-line flag, GCC *already has* that (-fdebug-prefix-map) and it wasn't enough to achieve reproducibility in many cases we tested. dpkg-buildflags actually already adds these flags to CFLAGS CXXFLAGS etc on Debian. However, with this patch using the environment variable, we are able to reproduce 1800 more packages out of 26000.Then take what you've done with the environment variable and instead
> implement it on top of a switch.  An environment variable is absolutely
> the wrong thing to do here.
> 
>>
>> GCC already supports a similar environment variable SOURCE_DATE_EPOCH, which was accepted about 2 years ago in a patch written by one of our GSoC students. We are not planning any more environment variables like this, and are committed to fixing other sources of non-determinism by patching the relevant build scripts.
> I would have rejected that as well :-)  One of the few times I would
> have disagreed with Bernd.

You can argue that gcc has command line options to set these, but most build
systems can be influenced by well documented environment variables like CFLAGS,
CXXFLAGS, LDFLAGS, so adding one more variable like SOURCE_DATE_EPOCH makes
sense, considering that many tools dealing with timestamps don't even have
command line options to do these (and there it's not just about compilers).

BUILD_PATH_PREFIX_MAP might be passed as well in CFLAGS / CXXFLAGS, but there
are build systems that even ignore these environment variables, and it's common
to require the use of environment variables for distributions, like

http://pkgs.fedoraproject.org/cgit/rpms/gcc.git/tree/gcc7-foffload-default.patch

If changes like that are not allowed to go upstream, where do you want to keep
these instead, and keeping these in sync across distributions?

Matthias