Bit-reproduciblity issue for gfortran .mod files

Bit-reproduciblity issue for gfortran .mod files

[Alastair McKinstry]

[-- Attachment #1.1: Type: text/plain, Size: 1670 bytes --]

Hi,

gfortran .mod files compressed files, contains a header, which looks like:
(viewed with e.g. zmore /usr/include/cgns.mod ):

GFORTRAN module version '14' created from /build/libcgns-XmhIxt/libcgns-3.3.0/src/cgns_f.F90
(() () () () () () () () () () () () () () () () () () () () () () () ()
() () ())

The full path of the compiled fortran file is included, which can break b=
it-reproducible builds.
(see https://reproducible-builds.org/)

A trivial patch is included, to strip the path.

Best regards
Alastair McKinstry

Index: gcc-7-7.2.0/src/gcc/fortran/module.c
===================================================================
--- gcc-7-7.2.0.orig/src/gcc/fortran/module.c
+++ gcc-7-7.2.0/src/gcc/fortran/module.c
@@ -79,6 +79,10 @@ along with GCC; see the file COPYING3.
 #include "scanner.h"
 #include <zlib.h>
 
+/* for GNU / POSIX basename */
+#define _GNU_SOURCE
+#include <string.h>
+
 #define MODULE_EXTENSION ".mod"
 #define SUBMODULE_EXTENSION ".smod"
 
@@ -6063,8 +6067,9 @@ dump_module (const char *name, int dump_
     gfc_fatal_error ("Can't open module file %qs for writing at %C: %s",
 		     filename_tmp, xstrerror (errno));
 
+  /* Use basename to make builds bit-reproducible */
   gzprintf (module_fp, "GFORTRAN module version '%s' created from %s\n",
-	    MOD_VERSION, gfc_source_file);
+            MOD_VERSION, basename(gfc_source_file));
 
   /* Write the module itself.  */
   iomode = IO_OUTPUT;



-- 
Alastair McKinstry, <alastair@sceal.ie>, <mckinstry@debian.org>, https://diaspora.sceal.ie/u/amckinstry
Misentropy: doubting that the Universe is becoming more disordered. 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Bit-reproduciblity issue for gfortran .mod files

[Steve Kargl]

On Fri, Feb 02, 2018 at 01:22:17PM +0000, Alastair McKinstry wrote:
> 
> Index: gcc-7-7.2.0/src/gcc/fortran/module.c
> ===================================================================
> --- gcc-7-7.2.0.orig/src/gcc/fortran/module.c
> +++ gcc-7-7.2.0/src/gcc/fortran/module.c
> @@ -79,6 +79,10 @@ along with GCC; see the file COPYING3.
>  #include "scanner.h"
>  #include <zlib.h>
>  
> +/* for GNU / POSIX basename */
> +#define _GNU_SOURCE
> +#include <string.h>
> +

What is the above for?  basename(3) is declared in libgen.h.

-- 
Steve

Re: Bit-reproduciblity issue for gfortran .mod files

[Alastair McKinstry]

On 03/02/2018 01:58, Steve Kargl wrote:

>
>> +/* for GNU / POSIX basename */
>> +#define _GNU_SOURCE
>> +#include <string.h>
>> +
> What is the above for?  basename(3) is declared in libgen.h.
>
I hadn't tested the libgen.h version, only this one. The manual page
points out that there are two versions, and "The  GNU  version  never 
modifies its argument", which made me wary of using the libgen.h one.


-- 
Alastair McKinstry, <alastair@sceal.ie>, <mckinstry@debian.org>, https://diaspora.sceal.ie/u/amckinstry
Commander Vimes didn’t like the phrase “The innocent have nothing to fear,”
 believing the innocent had everything to fear, mostly from the guilty but in the longer term
 even more from those who say things like “The innocent have nothing to fear.”
 - T. Pratchett, Snuff

Re: Bit-reproduciblity issue for gfortran .mod files

[Janne Blomqvist]

On Mon, Feb 5, 2018 at 11:03 AM, Alastair McKinstry
<alastair.mckinstry@sceal.ie> wrote:
> On 03/02/2018 01:58, Steve Kargl wrote:
>
>>
>>> +/* for GNU / POSIX basename */
>>> +#define _GNU_SOURCE
>>> +#include <string.h>
>>> +
>> What is the above for?  basename(3) is declared in libgen.h.
>>
> I hadn't tested the libgen.h version, only this one. The manual page
> points out that there are two versions, and "The  GNU  version  never
> modifies its argument", which made me wary of using the libgen.h one.

All the world is not Debian; there are even poor misguided souls out
there stuck on non-POSIX systems.

So while reproducible builds are important and this is a good catch,
your patch as-is is not really usable. But luckily gcc includes a copy
of libiberty, including lbasename(). That function seems to be used by
other frontends, so I'd suggest using that instead.


-- 
Janne Blomqvist

Re: Bit-reproduciblity issue for gfortran .mod files

[Alastair McKinstry]

On 05/02/2018 09:53, Janne Blomqvist wrote:
> On Mon, Feb 5, 2018 at 11:03 AM, Alastair McKinstry
> <alastair.mckinstry@sceal.ie> wrote:
>> On 03/02/2018 01:58, Steve Kargl wrote:
>>
>>>> +/* for GNU / POSIX basename */
>>>> +#define _GNU_SOURCE
>>>> +#include <string.h>
>>>> +
>>> What is the above for?  basename(3) is declared in libgen.h.
>>>
>> I hadn't tested the libgen.h version, only this one. The manual page
>> points out that there are two versions, and "The  GNU  version  never
>> modifies its argument", which made me wary of using the libgen.h one.
> All the world is not Debian; there are even poor misguided souls out
> there stuck on non-POSIX systems.
Indeed.
> So while reproducible builds are important and this is a good catch,
> your patch as-is is not really usable. But luckily gcc includes a copy
> of libiberty, including lbasename(). That function seems to be used by
> other frontends, so I'd suggest using that instead.

Thanks, I hadn't heard of lbasename(). I've just tested it and it works.
No header inclusion needed.


-- 
Alastair McKinstry, <alastair@sceal.ie>, <mckinstry@debian.org>, https://diaspora.sceal.ie/u/amckinstry
Commander Vimes didn’t like the phrase “The innocent have nothing to fear,”
 believing the innocent had everything to fear, mostly from the guilty but in the longer term
 even more from those who say things like “The innocent have nothing to fear.”
 - T. Pratchett, Snuff

Re: Bit-reproduciblity issue for gfortran .mod files

[Janne Blomqvist]

On Mon, Feb 5, 2018 at 11:13 PM, Alastair McKinstry
<alastair.mckinstry@sceal.ie> wrote:
>
> On 05/02/2018 09:53, Janne Blomqvist wrote:
>> On Mon, Feb 5, 2018 at 11:03 AM, Alastair McKinstry
>> <alastair.mckinstry@sceal.ie> wrote:
>>> On 03/02/2018 01:58, Steve Kargl wrote:
>>>
>>>>> +/* for GNU / POSIX basename */
>>>>> +#define _GNU_SOURCE
>>>>> +#include <string.h>
>>>>> +
>>>> What is the above for?  basename(3) is declared in libgen.h.
>>>>
>>> I hadn't tested the libgen.h version, only this one. The manual page
>>> points out that there are two versions, and "The  GNU  version  never
>>> modifies its argument", which made me wary of using the libgen.h one.
>> All the world is not Debian; there are even poor misguided souls out
>> there stuck on non-POSIX systems.
> Indeed.
>> So while reproducible builds are important and this is a good catch,
>> your patch as-is is not really usable. But luckily gcc includes a copy
>> of libiberty, including lbasename(). That function seems to be used by
>> other frontends, so I'd suggest using that instead.
>
> Thanks, I hadn't heard of lbasename(). I've just tested it and it works.
> No header inclusion needed.

This is Ok for trunk with a proper ChangeLog entry. Since it's
effectively a change in a comment I think we can dispense with nagging
the release managers, even though we're in stage 4.

Do you, or anybody else, have any particular opinion on whether it
should be backported to older release branches?

Do you have commit access, or do you need somebody to do it for you?


-- 
Janne Blomqvist

Re: Bit-reproduciblity issue for gfortran .mod files

[Alastair McKinstry]

>>> So while reproducible builds are important and this is a good catch,
>>> your patch as-is is not really usable. But luckily gcc includes a copy
>>> of libiberty, including lbasename(). That function seems to be used by
>>> other frontends, so I'd suggest using that instead.
>> Thanks, I hadn't heard of lbasename(). I've just tested it and it works.
>> No header inclusion needed.
> This is Ok for trunk with a proper ChangeLog entry. Since it's
> effectively a change in a comment I think we can dispense with nagging
> the release managers, even though we're in stage 4.
>
> Do you, or anybody else, have any particular opinion on whether it
> should be backported to older release branches?
>
> Do you have commit access, or do you need somebody to do it for you?
>
I have no opinion on whether it should be backported; there is a patch
for it in Debian at the moment; it would help reproducible builds
elsewhere (eg OpenSUSE have a reproducible builds project too).

I don't have commit access.

-- 
Alastair McKinstry, <alastair@sceal.ie>, <mckinstry@debian.org>, https://diaspora.sceal.ie/u/amckinstry
Commander Vimes didn’t like the phrase “The innocent have nothing to fear,”
 believing the innocent had everything to fear, mostly from the guilty but in the longer term
 even more from those who say things like “The innocent have nothing to fear.”
 - T. Pratchett, Snuff

Re: Bit-reproduciblity issue for gfortran .mod files

[Janne Blomqvist]

On Tue, Feb 13, 2018 at 8:55 AM, Alastair McKinstry
<alastair.mckinstry@sceal.ie> wrote:
>
>>>> So while reproducible builds are important and this is a good catch,
>>>> your patch as-is is not really usable. But luckily gcc includes a copy
>>>> of libiberty, including lbasename(). That function seems to be used by
>>>> other frontends, so I'd suggest using that instead.
>>> Thanks, I hadn't heard of lbasename(). I've just tested it and it works.
>>> No header inclusion needed.
>> This is Ok for trunk with a proper ChangeLog entry. Since it's
>> effectively a change in a comment I think we can dispense with nagging
>> the release managers, even though we're in stage 4.
>>
>> Do you, or anybody else, have any particular opinion on whether it
>> should be backported to older release branches?
>>
>> Do you have commit access, or do you need somebody to do it for you?
>>
> I have no opinion on whether it should be backported; there is a patch
> for it in Debian at the moment; it would help reproducible builds
> elsewhere (eg OpenSUSE have a reproducible builds project too).
>
> I don't have commit access.

Hi,

here's the patch I tested and committed:
https://gcc.gnu.org/ml/gcc-patches/2018-02/msg00706.html

Committed to trunk as r257613, gcc-7 branch as r257614, and gcc-6
branch as r257615.

Thanks for bringing up this issue!

-- 
Janne Blomqvist