* [patch RFA] support 'deterministic mode' in ar
@ 2009-03-06 8:43 Chris Demetriou
2009-03-10 8:54 ` Nick Clifton
0 siblings, 1 reply; 3+ messages in thread
From: Chris Demetriou @ 2009-03-06 8:43 UTC (permalink / raw)
To: binutils
[-- Attachment #1: Type: text/plain, Size: 2602 bytes --]
The attached patch adds support for a new 'deterministic mode' in
'ar', which causes uids, gids, and timestamps in archives to be set to
0, and file modes to consistently be set to 0644.
(I suggested the possibility of this feature a month or so ago, with a
prelim patch... and finally got around to finishing the patch.)
I've used this to successfully build a (cross-)gcc which produces
deterministic archives by setting AR_FOR_TARGET and RANLIB_FOR_TARGET
to, respectively:
* a wrapper script (around the normal target 'ar') that adds the 'D'
flag into the command options, and
* a wrapper script containing only #!/bin/true.
(if people call ranlib to update the archive map timestamp ... that'll
update the timestamp, defeating use of the new deterministic mode in
ar.)
also built for i686-linux (build/host/target), ran make check, all
tests passed. Also looked at the resulting 'ar' manual page, it
looked sane.
chris
----
[bfd/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* bfd.c (BFD_DETERMINISTIC_OUTPUT): New flag.
* bfd-in2.h: Regenerate.
* ar.c (bfd_ar_hdr_from_filesystem): If BFD_DETERMINISTIC_OUTPUT flag
is set, use 0 for uid, gid, and timestamp, and use 0644 for file mode.
(bsd_write_armap): Likewise.
(_bfd_archive_bsd_update_armap_timestamp): If BFD_DETERMINISTIC_OUTPUT
flag is set, do nothing.
[cgd@cdemetriou-corp ~]$ head -50 !$
head -50 bu_deterministic.patch
[bfd/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* bfd.c (BFD_DETERMINISTIC_OUTPUT): New flag.
* bfd-in2.h: Regenerate.
* ar.c (bfd_ar_hdr_from_filesystem): If BFD_DETERMINISTIC_OUTPUT flag
is set, use 0 for uid, gid, and timestamp, and use 0644 for file mode.
(bsd_write_armap): Likewise.
(_bfd_archive_bsd_update_armap_timestamp): If BFD_DETERMINISTIC_OUTPUT
flag is set, do nothing.
(coff_write_armap): If BFD_DETERMINISTIC_OUTPUT flag is set, use 0
for timestamp.
[binutils/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* ar.c (deterministic): New global variable.
(main): Recognize new 'D' option, which enables 'deterministic mode'.
(usage): Document new 'D' option.
(write_archive): Set BFD_DETERMINISTIC_OUTPUT in output archive's
flags if deterministic mode was requested.
* doc/binutils.texi (ar): Document deterministic mode ('D' option).
[binutils/testsuite/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* binutils-all/ar.exp (deterministic_archive): New test.
[-- Attachment #2: bu_deterministic.patch --]
[-- Type: application/octet-stream, Size: 11177 bytes --]
[bfd/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* bfd.c (BFD_DETERMINISTIC_OUTPUT): New flag.
* bfd-in2.h: Regenerate.
* ar.c (bfd_ar_hdr_from_filesystem): If BFD_DETERMINISTIC_OUTPUT flag
is set, use 0 for uid, gid, and timestamp, and use 0644 for file mode.
(bsd_write_armap): Likewise.
(_bfd_archive_bsd_update_armap_timestamp): If BFD_DETERMINISTIC_OUTPUT
flag is set, do nothing.
(coff_write_armap): If BFD_DETERMINISTIC_OUTPUT flag is set, use 0
for timestamp.
[binutils/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* ar.c (deterministic): New global variable.
(main): Recognize new 'D' option, which enables 'deterministic mode'.
(usage): Document new 'D' option.
(write_archive): Set BFD_DETERMINISTIC_OUTPUT in output archive's
flags if deterministic mode was requested.
* doc/binutils.texi (ar): Document deterministic mode ('D' option).
[binutils/testsuite/ChangeLog]
2009-03-06 Chris Demetriou <cgd@google.com>
* binutils-all/ar.exp (deterministic_archive): New test.
Index: bfd/archive.c
===================================================================
RCS file: /cvs/src/src/bfd/archive.c,v
retrieving revision 1.55
diff -u -u -p -r1.55 archive.c
--- bfd/archive.c 10 Aug 2008 18:49:09 -0000 1.55
+++ bfd/archive.c 6 Mar 2009 08:33:11 -0000
@@ -1652,6 +1652,16 @@ bfd_ar_hdr_from_filesystem (bfd *abfd, c
return NULL;
}
+ /* If the caller requested that the BFD generate deterministic output,
+ fake values for modification time, UID, GID, and file mode. */
+ if ((abfd->flags & BFD_DETERMINISTIC_OUTPUT) != 0)
+ {
+ status.st_mtime = 0;
+ status.st_uid = 0;
+ status.st_gid = 0;
+ status.st_mode = 0644;
+ }
+
amt = sizeof (struct ar_hdr) + sizeof (struct areltdata);
ared = bfd_zalloc (abfd, amt);
if (ared == NULL)
@@ -2220,20 +2230,39 @@ bsd_write_armap (bfd *arch,
unsigned int count;
struct ar_hdr hdr;
struct stat statbuf;
+ long uid, gid;
firstreal = mapsize + elength + sizeof (struct ar_hdr) + SARMAG;
stat (arch->filename, &statbuf);
+ if ((arch->flags & BFD_DETERMINISTIC_OUTPUT) == 0)
+ {
+ /* Remember the timestamp, to keep it holy. But fudge it a little. */
+ bfd_ardata (arch)->armap_timestamp = (statbuf.st_mtime
+ + ARMAP_TIME_OFFSET);
+ uid = getuid();
+ gid = getgid();
+ }
+ else
+ {
+ /* If deterministic, we use 0 as the timestamp in the map.
+ Some linkers may require that the archive filesystem modification
+ time is less than (or near to) the archive map timestamp. Those
+ linkers should not be used with deterministic mode. (GNU ld and
+ Gold do not have this restriction.) */
+ bfd_ardata (arch)->armap_timestamp = 0;
+ uid = 0;
+ gid = 0;
+ }
+
memset (&hdr, ' ', sizeof (struct ar_hdr));
memcpy (hdr.ar_name, RANLIBMAG, strlen (RANLIBMAG));
- /* Remember the timestamp, to keep it holy. But fudge it a little. */
- bfd_ardata (arch)->armap_timestamp = statbuf.st_mtime + ARMAP_TIME_OFFSET;
bfd_ardata (arch)->armap_datepos = (SARMAG
+ offsetof (struct ar_hdr, ar_date[0]));
_bfd_ar_spacepad (hdr.ar_date, sizeof (hdr.ar_date), "%ld",
bfd_ardata (arch)->armap_timestamp);
- _bfd_ar_spacepad (hdr.ar_uid, sizeof (hdr.ar_uid), "%ld", getuid ());
- _bfd_ar_spacepad (hdr.ar_gid, sizeof (hdr.ar_gid), "%ld", getgid ());
+ _bfd_ar_spacepad (hdr.ar_uid, sizeof (hdr.ar_uid), "%ld", uid);
+ _bfd_ar_spacepad (hdr.ar_gid, sizeof (hdr.ar_gid), "%ld", gid);
_bfd_ar_spacepad (hdr.ar_size, sizeof (hdr.ar_size), "%-10ld", mapsize);
memcpy (hdr.ar_fmag, ARFMAG, 2);
if (bfd_bwrite (&hdr, sizeof (struct ar_hdr), arch)
@@ -2301,6 +2330,10 @@ _bfd_archive_bsd_update_armap_timestamp
struct stat archstat;
struct ar_hdr hdr;
+ /* If creating deterministic archives, just leave the timestamp as-is. */
+ if ((arch->flags & BFD_DETERMINISTIC_OUTPUT) != 0)
+ return TRUE;
+
/* Flush writes, get last-write timestamp from file, and compare it
to the timestamp IN the file. */
bfd_flush (arch);
@@ -2385,7 +2418,8 @@ coff_write_armap (bfd *arch,
_bfd_ar_spacepad (hdr.ar_size, sizeof (hdr.ar_size), "%-10ld",
mapsize);
_bfd_ar_spacepad (hdr.ar_date, sizeof (hdr.ar_date), "%ld",
- time (NULL));
+ ((arch->flags & BFD_DETERMINISTIC_OUTPUT) == 0
+ ? time (NULL) : 0));
/* This, at least, is what Intel coff sets the values to. */
_bfd_ar_spacepad (hdr.ar_uid, sizeof (hdr.ar_uid), "%ld", 0);
_bfd_ar_spacepad (hdr.ar_gid, sizeof (hdr.ar_gid), "%ld", 0);
Index: bfd/bfd-in2.h
===================================================================
RCS file: /cvs/src/src/bfd/bfd-in2.h,v
retrieving revision 1.467
diff -u -u -p -r1.467 bfd-in2.h
--- bfd/bfd-in2.h 4 Mar 2009 05:50:49 -0000 1.467
+++ bfd/bfd-in2.h 6 Mar 2009 08:33:11 -0000
@@ -4769,6 +4769,11 @@ struct bfd
to any input file. */
#define BFD_LINKER_CREATED 0x2000
+ /* This may be set before writing out a BFD to request that it
+ be written using values for UIDs, GIDs, timestamps, etc. that
+ will be consistent from run to run. */
+#define BFD_DETERMINISTIC_OUTPUT 0x4000
+
/* Currently my_archive is tested before adding origin to
anything. I believe that this can become always an add of
origin, with origin set to 0 for non archive files. */
Index: bfd/bfd.c
===================================================================
RCS file: /cvs/src/src/bfd/bfd.c,v
retrieving revision 1.106
diff -u -u -p -r1.106 bfd.c
--- bfd/bfd.c 20 Nov 2008 09:28:06 -0000 1.106
+++ bfd/bfd.c 6 Mar 2009 08:33:11 -0000
@@ -145,6 +145,11 @@ CODE_FRAGMENT
. to any input file. *}
.#define BFD_LINKER_CREATED 0x2000
.
+. {* This may be set before writing out a BFD to request that it
+. be written using values for UIDs, GIDs, timestamps, etc. that
+. will be consistent from run to run. *}
+.#define BFD_DETERMINISTIC_OUTPUT 0x4000
+.
. {* Currently my_archive is tested before adding origin to
. anything. I believe that this can become always an add of
. origin, with origin set to 0 for non archive files. *}
Index: binutils/ar.c
===================================================================
RCS file: /cvs/src/src/binutils/ar.c,v
retrieving revision 1.56
diff -u -u -p -r1.56 ar.c
--- binutils/ar.c 28 Apr 2008 08:30:23 -0000 1.56
+++ binutils/ar.c 6 Mar 2009 08:33:11 -0000
@@ -99,6 +99,11 @@ int newer_only = 0;
if any of the members are object files. */
int write_armap = 0;
+/* Operate in deterministic mode: write zero for timestamps, uids,
+ and gids for archive members and the archive symbol table, and write
+ consistent file modes. */
+int deterministic = 0;
+
/* Nonzero means it's the name of an existing member; position new or moved
files with respect to this one. */
char *posname = NULL;
@@ -240,6 +245,7 @@ usage (int help)
fprintf (s, _(" command specific modifiers:\n"));
fprintf (s, _(" [a] - put file(s) after [member-name]\n"));
fprintf (s, _(" [b] - put file(s) before [member-name] (same as [i])\n"));
+ fprintf (s, _(" [D] - use zero for timestamps and uids/gids\n"));
fprintf (s, _(" [N] - use instance [count] of name\n"));
fprintf (s, _(" [f] - truncate inserted file names\n"));
fprintf (s, _(" [P] - use full path names when matching\n"));
@@ -572,6 +578,9 @@ main (int argc, char **argv)
case 'T':
make_thin_archive = TRUE;
break;
+ case 'D':
+ deterministic = TRUE;
+ break;
default:
/* xgettext:c-format */
non_fatal (_("illegal option -- %c"), c);
@@ -622,6 +631,9 @@ main (int argc, char **argv)
if (newer_only && operation != replace)
fatal (_("`u' is only meaningful with the `r' option."));
+ if (newer_only && deterministic)
+ fatal (_("`u' is not meaningful with the `D' option."));
+
if (postype != pos_default)
posname = argv[arg_index++];
@@ -972,6 +984,9 @@ write_archive (bfd *iarch)
obfd->flags |= BFD_TRADITIONAL_FORMAT;
}
+ if (deterministic)
+ obfd->flags |= BFD_DETERMINISTIC_OUTPUT;
+
if (make_thin_archive || bfd_is_thin_archive (iarch))
bfd_is_thin_archive (obfd) = 1;
Index: binutils/doc/binutils.texi
===================================================================
RCS file: /cvs/src/src/binutils/doc/binutils.texi,v
retrieving revision 1.140
diff -u -u -p -r1.140 binutils.texi
--- binutils/doc/binutils.texi 9 Feb 2009 09:14:15 -0000 1.140
+++ binutils/doc/binutils.texi 6 Mar 2009 08:33:11 -0000
@@ -396,6 +396,15 @@ created if it did not exist, when you re
issued unless you specify in advance that you expect to create it, by
using this modifier.
+@item D
+@cindex deterministic archives
+Operate in @emph{deterministic} mode. When adding files and the archive
+index use zero for UIDs, GIDs, timestamps, and use consistent file modes
+for all files. When this option is used, if @command{ar} is used with
+identical options and identical input files, multiple runs will create
+identical output files regardless of the input files' owners, groups,
+file modes, or modification times.
+
@item f
Truncate names in the archive. @sc{gnu} @command{ar} will normally permit file
names of any length. This will cause it to create archives which are
Index: binutils/testsuite/binutils-all/ar.exp
===================================================================
RCS file: /cvs/src/src/binutils/testsuite/binutils-all/ar.exp,v
retrieving revision 1.11
diff -u -u -p -r1.11 ar.exp
--- binutils/testsuite/binutils-all/ar.exp 28 Mar 2008 06:49:44 -0000 1.11
+++ binutils/testsuite/binutils-all/ar.exp 6 Mar 2009 08:33:11 -0000
@@ -354,6 +354,50 @@ proc argument_parsing { } {
pass $testname
}
+# Test building a deterministic archive.
+
+proc deterministic_archive { } {
+ global AR
+ global AS
+ global NM
+ global srcdir
+ global subdir
+
+ set testname "ar deterministic archive"
+
+ if ![binutils_assemble $srcdir/$subdir/bintest.s tmpdir/bintest.o] {
+ unresolved $testname
+ return
+ }
+
+ if [is_remote host] {
+ set archive artest.a
+ set objfile [remote_download host tmpdir/bintest.o]
+ remote_file host delete $archive
+ } else {
+ set archive tmpdir/artest.a
+ set objfile tmpdir/bintest.o
+ }
+
+ remote_file build delete tmpdir/artest.a
+
+ set got [binutils_run $AR "rcD $archive ${objfile}"]
+ if ![string match "" $got] {
+ fail $testname
+ return
+ }
+
+ set got [binutils_run $AR "tv $archive"]
+ # This only checks the file mode and uid/gid. We can't easily match
+ # date because it's printed with the user's timezone.
+ if ![string match "rw-r--r-- 0/0 *bintest.o*" $got] {
+ fail $testname
+ return
+ }
+
+ pass $testname
+}
+
# Run the tests.
long_filenames
@@ -361,3 +405,4 @@ symbol_table
thin_archive
thin_archive_with_nested
argument_parsing
+deterministic_archive
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [patch RFA] support 'deterministic mode' in ar
2009-03-06 8:43 [patch RFA] support 'deterministic mode' in ar Chris Demetriou
@ 2009-03-10 8:54 ` Nick Clifton
2009-03-11 4:39 ` Chris Demetriou
0 siblings, 1 reply; 3+ messages in thread
From: Nick Clifton @ 2009-03-10 8:54 UTC (permalink / raw)
To: binutils
Hi Chris,
> [bfd/ChangeLog]
> 2009-03-06 Chris Demetriou <cgd@google.com>
>
> * bfd.c (BFD_DETERMINISTIC_OUTPUT): New flag.
> * bfd-in2.h: Regenerate.
> * ar.c (bfd_ar_hdr_from_filesystem): If BFD_DETERMINISTIC_OUTPUT flag
> is set, use 0 for uid, gid, and timestamp, and use 0644 for file mode.
> (bsd_write_armap): Likewise.
> (_bfd_archive_bsd_update_armap_timestamp): If BFD_DETERMINISTIC_OUTPUT
> flag is set, do nothing.
> [binutils/ChangeLog]
> 2009-03-06 Chris Demetriou <cgd@google.com>
>
> * ar.c (deterministic): New global variable.
> (main): Recognize new 'D' option, which enables 'deterministic mode'.
> (usage): Document new 'D' option.
> (write_archive): Set BFD_DETERMINISTIC_OUTPUT in output archive's
> flags if deterministic mode was requested.
> * doc/binutils.texi (ar): Document deterministic mode ('D' option).
>
> [binutils/testsuite/ChangeLog]
> 2009-03-06 Chris Demetriou <cgd@google.com>
>
> * binutils-all/ar.exp (deterministic_archive): New test.
Approved - please apply.
Cheers
Nick
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [patch RFA] support 'deterministic mode' in ar
2009-03-10 8:54 ` Nick Clifton
@ 2009-03-11 4:39 ` Chris Demetriou
0 siblings, 0 replies; 3+ messages in thread
From: Chris Demetriou @ 2009-03-11 4:39 UTC (permalink / raw)
To: Nick Clifton; +Cc: binutils
On Tue, Mar 10, 2009 at 01:54, Nick Clifton <nickc@redhat.com> wrote:
> Hi Chris,
>
>> [bfd/ChangeLog]
>> 2009-03-06 Chris Demetriou <cgd@google.com>
>>
>> * bfd.c (BFD_DETERMINISTIC_OUTPUT): New flag.
>> * bfd-in2.h: Regenerate.
>> * ar.c (bfd_ar_hdr_from_filesystem): If BFD_DETERMINISTIC_OUTPUT
>> flag
>> is set, use 0 for uid, gid, and timestamp, and use 0644 for file
>> mode.
>> (bsd_write_armap): Likewise.c")
>> (_bfd_archive_bsd_update_armap_timestamp): If
>> BFD_DETERMINISTIC_OUTPUT
>> flag is set, do nothing.
>
>> [binutils/ChangeLog]
>> 2009-03-06 Chris Demetriou <cgd@google.com>
>>
>> * ar.c (deterministic): New global variable.
>> (main): Recognize new 'D' option, which enables 'deterministic
>> mode'.
>> (usage): Document new 'D' option.
>> (write_archive): Set BFD_DETERMINISTIC_OUTPUT in output archive's
>> flags if deterministic mode was requested.
>> * doc/binutils.texi (ar): Document deterministic mode ('D' option).
>>
>> [binutils/testsuite/ChangeLog]
>> 2009-03-06 Chris Demetriou <cgd@google.com>
>>
>> * binutils-all/ar.exp (deterministic_archive): New test.
>
> Approved - please apply.
I noticed a bug in the bfd/ChangeLog ("ar.c" should have been
"archive.c"). So, I fixed that, updated to current srcs, retested
(i686-linux build/host/target), and committed.
thanks!
chris
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2009-03-11 4:39 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-03-06 8:43 [patch RFA] support 'deterministic mode' in ar Chris Demetriou
2009-03-10 8:54 ` Nick Clifton
2009-03-11 4:39 ` Chris Demetriou
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).