* Deterministic builds
@ 2016-05-04 13:38 Ken Brown
2016-05-04 14:30 ` Greg Chicares
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Ken Brown @ 2016-05-04 13:38 UTC (permalink / raw)
To: cygwin
Is it possible to build an executable on Cygwin so that subsequent
builds (with no change in source) produce identical results? Currently,
the timestamp embedded in executables prevents this. (I don't know if
that's the only obstacle.)
For example:
$ cat hello.c
#include <stdio.h>
int
main ()
{
printf("Hello, world!\n");
return 0;
}
$ gcc hello.c -o hello1
$ gcc hello.c -o hello2
$ objdump -p hello1.exe | grep Time/Date
Time/Date Wed May 4 09:20:24 2016
$ objdump -p hello2.exe | grep Time/Date
Time/Date Wed May 4 09:20:29 2016
My actual use case is that I'm building a package that produces a large
number of executables. If I make a change in one source file, I'd like
to be able to know which executables change.
Ken
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 13:38 Deterministic builds Ken Brown
@ 2016-05-04 14:30 ` Greg Chicares
2016-05-04 17:11 ` Achim Gratz
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Greg Chicares @ 2016-05-04 14:30 UTC (permalink / raw)
To: cygwin
On 2016-05-04 13:38, Ken Brown wrote:
> Is it possible to build an executable on Cygwin so that subsequent
> builds (with no change in source) produce identical results? Currently,
> the timestamp embedded in executables prevents this. (I don't know if
> that's the only obstacle.)
'-fno-guess-branch-probability' removes one potential obstacle:
https://lists.debian.org/debian-devel/2005/02/msg01154.html
I don't know if there are other obstacles than these.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 13:38 Deterministic builds Ken Brown
2016-05-04 14:30 ` Greg Chicares
@ 2016-05-04 17:11 ` Achim Gratz
2016-05-04 17:21 ` Ismail Donmez
2016-05-05 1:50 ` Andrey Repin
3 siblings, 0 replies; 9+ messages in thread
From: Achim Gratz @ 2016-05-04 17:11 UTC (permalink / raw)
To: cygwin
Ken Brown writes:
> Is it possible to build an executable on Cygwin so that subsequent
> builds (with no change in source) produce identical results?
> Currently, the timestamp embedded in executables prevents this. (I
> don't know if that's the only obstacle.)
I think the basic problems and solutions could be sussed from the work
of Debian folks on reproducible builds:
https://wiki.debian.org/ReproducibleBuilds
> My actual use case is that I'm building a package that produces a
> large number of executables. If I make a change in one source file,
> I'd like to be able to know which executables change.
Just looking at executables, you will probably find that in gcc there
are a few optimizations that do not produce the same bits on each
compile (depending on available memory, CPU speed or some random seed).
For some of these there are switches to stabilize the result at least.
Timestamps in various file formats are another source of chatter,
although you might be able to either fix thetime or just ignore those
timestamps during comparison (but that's generally more costly than just
doing a plain comparison).
Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 13:38 Deterministic builds Ken Brown
2016-05-04 14:30 ` Greg Chicares
2016-05-04 17:11 ` Achim Gratz
@ 2016-05-04 17:21 ` Ismail Donmez
2016-05-04 17:40 ` Ken Brown
2016-05-05 1:50 ` Andrey Repin
3 siblings, 1 reply; 9+ messages in thread
From: Ismail Donmez @ 2016-05-04 17:21 UTC (permalink / raw)
To: cygwin
Hi,
On Wed, May 4, 2016 at 4:38 PM, Ken Brown <kbrown@cornell.edu> wrote:
> Is it possible to build an executable on Cygwin so that subsequent builds
> (with no change in source) produce identical results? Currently, the
> timestamp embedded in executables prevents this. (I don't know if that's
> the only obstacle.)
>
> For example:
>
> $ cat hello.c
> #include <stdio.h>
> int
> main ()
> {
> printf("Hello, world!\n");
> return 0;
> }
>
> $ gcc hello.c -o hello1
>
> $ gcc hello.c -o hello2
>
> $ objdump -p hello1.exe | grep Time/Date
> Time/Date Wed May 4 09:20:24 2016
>
> $ objdump -p hello2.exe | grep Time/Date
> Time/Date Wed May 4 09:20:29 2016
You can easily disable this feature:
latte ~ > gcc -Wl,--no-insert-timestamp hello.c
latte ~ > objdump -p a.exe | grep Time/Date
Time/Date Thu Jan 1 03:31:53 1970
latte ~ > gcc -Wl,--no-insert-timestamp hello.c
latte ~ > objdump -p a.exe | grep Time/Date
Time/Date Thu Jan 1 03:31:53 1970
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 17:21 ` Ismail Donmez
@ 2016-05-04 17:40 ` Ken Brown
2016-05-05 17:59 ` Ken Brown
0 siblings, 1 reply; 9+ messages in thread
From: Ken Brown @ 2016-05-04 17:40 UTC (permalink / raw)
To: cygwin
On 5/4/2016 1:21 PM, Ismail Donmez wrote:
> You can easily disable this feature:
>
> latte ~ > gcc -Wl,--no-insert-timestamp hello.c
> latte ~ > objdump -p a.exe | grep Time/Date
> Time/Date Thu Jan 1 03:31:53 1970
> latte ~ > gcc -Wl,--no-insert-timestamp hello.c
> latte ~ > objdump -p a.exe | grep Time/Date
> Time/Date Thu Jan 1 03:31:53 1970
Thank you! That's exactly what I was looking for.
Ken
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 13:38 Deterministic builds Ken Brown
` (2 preceding siblings ...)
2016-05-04 17:21 ` Ismail Donmez
@ 2016-05-05 1:50 ` Andrey Repin
3 siblings, 0 replies; 9+ messages in thread
From: Andrey Repin @ 2016-05-05 1:50 UTC (permalink / raw)
To: Ken Brown, cygwin
Greetings, Ken Brown!
> Is it possible to build an executable on Cygwin so that subsequent
> builds (with no change in source) produce identical results?
General answer is "no". It is possible to build a consistent object binary,
but executable linked from it will be different on different systems, for
various reasons.
> Currently,
> the timestamp embedded in executables prevents this. (I don't know if
> that's the only obstacle.)
Timestamps are the least of your issues.
Think of things like stack allocation bases.
> My actual use case is that I'm building a package that produces a large
> number of executables. If I make a change in one source file, I'd like
> to be able to know which executables change.
If your interest is purely self-educational, you can limit the noise on a
particular given system. But I wouldn't try to make any universal claims if I
were you.
--
With best regards,
Andrey Repin
Thursday, May 5, 2016 04:32:14
Sorry for my terrible english...
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-04 17:40 ` Ken Brown
@ 2016-05-05 17:59 ` Ken Brown
2016-05-05 20:26 ` Warren Young
0 siblings, 1 reply; 9+ messages in thread
From: Ken Brown @ 2016-05-05 17:59 UTC (permalink / raw)
To: cygwin
On 5/4/2016 1:39 PM, Ken Brown wrote:
> On 5/4/2016 1:21 PM, Ismail Donmez wrote:
>> You can easily disable this feature:
>>
>> latte ~ > gcc -Wl,--no-insert-timestamp hello.c
>> latte ~ > objdump -p a.exe | grep Time/Date
>> Time/Date Thu Jan 1 03:31:53 1970
>> latte ~ > gcc -Wl,--no-insert-timestamp hello.c
>> latte ~ > objdump -p a.exe | grep Time/Date
>> Time/Date Thu Jan 1 03:31:53 1970
>
> Thank you! That's exactly what I was looking for.
Just for the record, in case anyone else finds this useful, Ismail's
suggestion did indeed produce deterministic builds in my setup. I built
a large project with about 150 executables, changed a few source files,
removed the build directory, rebuilt, and found that only the (expected)
few executables changed.
Ken
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-05 17:59 ` Ken Brown
@ 2016-05-05 20:26 ` Warren Young
2016-05-05 21:46 ` Ken Brown
0 siblings, 1 reply; 9+ messages in thread
From: Warren Young @ 2016-05-05 20:26 UTC (permalink / raw)
To: The Cygwin Mailing List
On May 5, 2016, at 11:59 AM, Ken Brown <kbrown@cornell.edu> wrote:
>
> Ismail's suggestion did indeed produce deterministic builds in my setup. I built a large project with about 150 executables, changed a few source files, removed the build directory, rebuilt, and found that only the (expected) few executables changed.
…and does it do the same on a very different system? e.g. Try it on both 64-bit Windows 10 and on 32-bit Windows 7.
Perhaps you don’t need it, but part of the reason for the big push recently for reproducible builds is to be able to verify that binaries from a given source (e.g. Red Hat’s RPM feed) are in fact buildable from the sources distributed from the same source (e.g. Red Hat’s SRPMs).
The usual motivation for that is security: it’s no good receiving an SRPM with a security patch if the binary that yum installs still has the bug.
Therefore, if you get “reproducible” builds on only a single machine, you may not have achieved any useful result.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Deterministic builds
2016-05-05 20:26 ` Warren Young
@ 2016-05-05 21:46 ` Ken Brown
0 siblings, 0 replies; 9+ messages in thread
From: Ken Brown @ 2016-05-05 21:46 UTC (permalink / raw)
To: cygwin
On 5/5/2016 4:26 PM, Warren Young wrote:
> On May 5, 2016, at 11:59 AM, Ken Brown <kbrown@cornell.edu> wrote:
>>
>> Ismail's suggestion did indeed produce deterministic builds in my setup. I built a large project with about 150 executables, changed a few source files, removed the build directory, rebuilt, and found that only the (expected) few executables changed.
>
> â¦and does it do the same on a very different system? e.g. Try it on both 64-bit Windows 10 and on 32-bit Windows 7.
>
> Perhaps you donât need it, but part of the reason for the big push recently for reproducible builds is to be able to verify that binaries from a given source (e.g. Red Hatâs RPM feed) are in fact buildable from the sources distributed from the same source (e.g. Red Hatâs SRPMs).
Yes, that's a much more ambitious goal, and it's not what I was trying
to do.
Ken
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-05-05 21:46 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-04 13:38 Deterministic builds Ken Brown
2016-05-04 14:30 ` Greg Chicares
2016-05-04 17:11 ` Achim Gratz
2016-05-04 17:21 ` Ismail Donmez
2016-05-04 17:40 ` Ken Brown
2016-05-05 17:59 ` Ken Brown
2016-05-05 20:26 ` Warren Young
2016-05-05 21:46 ` Ken Brown
2016-05-05 1:50 ` Andrey Repin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).