gcc binary comparison help

gcc binary comparison help

[Caleb Cittadino]

Hello,



I need to compare two binaries build by gcc to determine if they have
the same contents.  My issue is that I can't build (using the same
build target) two of the same binary.  I believe this is because a
timestamp or something of the kind is being put in.  Is there another
way I can check these contents against each other without the extra
info?



Thanks,

~Caleb

Re: gcc binary comparison help

[Brian Dessent]

Caleb Cittadino wrote:

> I need to compare two binaries build by gcc to determine if they have
> the same contents.  My issue is that I can't build (using the same
> build target) two of the same binary.  I believe this is because a
> timestamp or something of the kind is being put in.  Is there another
> way I can check these contents against each other without the extra
> info?

Well, you didn't mention anything about what platform and object format
you're using so I don't think you're going to get any definitive
answers.

But the gcc bootstrap machinery itself uses "cmp --ignore-initial=16 a
b" to compare the outputs of each stage so that the timestamp in the
object file isn't compared.  (Except for portability it detects whether
to use that form or "cmp a b 16 16" or whether to fall-back to "tail
+16c", see the ACX_PROG_CMP_IGNORE_INITIAL macro in config/acx.m4.)

If that doesn't work, you may have to check the docs for whatever file
format you're using to see where it keeps the timestamp, if any.  And
this assumes that your source code does not contain any macros such as
__TIME__ or __DATE__ or whatever that might embed a unique value in a
string.  If that's the case then I guess you'd have to resort to keeping
the preprocessed source around and only compiling that.

Brian

Re: gcc binary comparison help

[Caleb Cittadino]

> Well, you didn't mention anything about what platform and object format
> you're using so I don't think you're going to get any definitive
> answers.

I'm using MacOSX + gcc4.  What do you mean by object format?

Also, I tried discarding the first 16 bytes as you suggested.  It
looks like a different set of data is changing each time, but the file
size remains the same.  So this method won't work.

Perhaps it is something in addition to (or instead of) the timestamp
that is changing each time?

Any help here is greatly appreciated.

~Caleb

Re: gcc binary comparison help

[Brian Dessent]

Caleb Cittadino wrote:

> I'm using MacOSX + gcc4.  What do you mean by object format?

ELF, COFF, a.out, etc.  I have no idea what OS X uses.

> Also, I tried discarding the first 16 bytes as you suggested.  It
> looks like a different set of data is changing each time, but the file
> size remains the same.  So this method won't work.
> 
> Perhaps it is something in addition to (or instead of) the timestamp
> that is changing each time?

You might need to use -frandom-seed.  And try diffing the preprocessed
source for the two objects that differ to see what exactly is changing.

Brian

Re: gcc binary comparison help

[Andrew Haley]

Caleb Cittadino writes:
 > > Well, you didn't mention anything about what platform and object format
 > > you're using so I don't think you're going to get any definitive
 > > answers.
 > 
 > I'm using MacOSX + gcc4.  What do you mean by object format?
 > 
 > Also, I tried discarding the first 16 bytes as you suggested.  It
 > looks like a different set of data is changing each time, but the file
 > size remains the same.  So this method won't work.
 > 
 > Perhaps it is something in addition to (or instead of) the timestamp
 > that is changing each time?
 > 
 > Any help here is greatly appreciated.

If I were you I'd be comparing assumbly code.

Andrew.

Re: gcc binary comparison help

[Caleb Cittadino]

Strangely, when I use -frandom-seed I get a bigger set of differing
data between the resulting library files.  I checked the documentation
for gcc and it states this options is used to "produce reproducibly
identical object files."  So I assume this will only work on .o files?

Also, I decided to take a comparison of the build directories XCode
makes.  All of the .o files stay the same and I get 3 file differences
overall: pbxindex.header, strings.pbxstrings/control, and
symbols0.pbxsymbols.  Perhaps this is a better way of localizing the
problem.

Any ideas?

~Caleb

On 12/5/06, Perry Smith <pedz@easesoftware.com> wrote:
> The header of object files for xcoff have timestamps in them but I'm not
> sure what file format a Mac uses.  There is also something about the "random
> seed" for gcc.  (Look up -frandom-seed=string in the docs).
>
> I remember this topic came up on either gcc or gcc-help within the past
> year.  You might search the archives to see if you can find some hits.
>
> I just did a trivial test on my Mac and the files compared equal (using g++
> 4.0.1).  But what I was going to suggest is to use cmp -l to get the list of
> places that are different.  It should be just four or eight bytes
>
>
> I would also try comparing the object files and see which ones match and
> which ones don't -- try and break down what is not matching.
>
> Not sure this helps much...
>
> On Dec 5, 2006, at 9:51 AM, Caleb Cittadino wrote:
> Hi Perry,
>
> Here is the start of the thread:
>
> __________________________________
>
> Hello,
>
>
>
> I need to compare two binaries build by gcc to determine if they have
>  the same contents.  My issue is that I can't build (using the same
> build target) two of the same binary.  I believe this is because a
> timestamp or something of the kind is being put in.  Is there another
> way I can check these contents against each other without the extra
> info?
>
>
>
> Thanks,
>
> ~Caleb
> _________________________
>
> On 12/4/06, Perry Smith < pedz@easesoftware.com> wrote:
> >
> >
> >
> > On Dec 4, 2006, at 7:31 PM, Caleb Cittadino wrote:
> >
> >
> >
> > Well, you didn't mention anything about what platform and object format
> > you're using so I don't think you're going to get any definitive
> > answers.
> >
> >
> > I'm using MacOSX + gcc4.  What do you mean by object format?
> >
> >
> > Different machines have different object file formats.  The two I am aware
> of is dwarf and xcoff.  I'm not sure what Mac OSX uses but that should be
> enough for the others on the help list to know.
> >
> >
> > I didn't see the start of this thread.  Are you trying to compare object
> files?
> >
> >
> > Perry Smith ( pedz@easesoftware.com )
> > Ease Software, Inc. (  http://www.easesoftware.com )
> >
> > Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems
> >
> >
> >
>
>
>
> Perry Smith ( pedz@easesoftware.com )
> Ease Software, Inc. ( http://www.easesoftware.com )
>
> Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems
>
>
>

Re: gcc binary comparison help

[Ian Lance Taylor]

"Caleb Cittadino" <caleb.cittadino@gmail.com> writes:

> Strangely, when I use -frandom-seed I get a bigger set of differing
> data between the resulting library files.  I checked the documentation
> for gcc and it states this options is used to "produce reproducibly
> identical object files."  So I assume this will only work on .o files?

Wait: are you comparing .o or .a files?  .a files will always have
lots of differences, because the .a file includes a timestamp.

You should be able to compare the .o files reliably.

Ian

Re: gcc binary comparison help

[Caleb Cittadino]

No I'm actually comparing library files.  Someone had mentioned I
should be comparing .o files first, which I did, but they remain the
same each time.  It's only the build contents I listed and the binary
that change.

~Caleb

On 06 Dec 2006 15:07:18 -0800, Ian Lance Taylor <iant@google.com> wrote:
> "Caleb Cittadino" <caleb.cittadino@gmail.com> writes:
>
> > Strangely, when I use -frandom-seed I get a bigger set of differing
> > data between the resulting library files.  I checked the documentation
> > for gcc and it states this options is used to "produce reproducibly
> > identical object files."  So I assume this will only work on .o files?
>
> Wait: are you comparing .o or .a files?  .a files will always have
> lots of differences, because the .a file includes a timestamp.
>
> You should be able to compare the .o files reliably.
>
> Ian
>

Re: gcc binary comparison help

[Perry Smith]

On Dec 6, 2006, at 5:32 PM, Caleb Cittadino wrote:

> No I'm actually comparing library files.  Someone had mentioned I
> should be comparing .o files first, which I did, but they remain the
> same each time.  It's only the build contents I listed and the binary
> that change.
>
> ~Caleb

Ah... progress.  Don't compare library files.

So, your .o's compare the same each time?  Aside from the library  
files (which we'll
ignore for a moment), I'm not clear what does not compare the same.   
Also, if you do
the cmp -l, how many bytes differ (on each of the things you are  
comparing)?

Perry Smith ( pedz@easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )

Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems

Re: gcc binary comparison help

[Caleb Cittadino]

On 12/6/06, Perry Smith <pedz@easesoftware.com> wrote:
> So, your .o's compare the same each time?  Aside from the library
> files (which we'll
> ignore for a moment), I'm not clear what does not compare the same.

.o files compare fine, it's these files from the build directory that don't:
-bxindex.header
-strings.pbxstrings/control
-symbols0.pbxsymbols.

> Also, if you do
> the cmp -l, how many bytes differ (on each of the things you are
> comparing)?

65 lines, which is 65 bytes I presume?

~Caleb

Re: gcc binary comparison help

[Perry Smith]

On Dec 6, 2006, at 6:16 PM, Caleb Cittadino wrote:

> On 12/6/06, Perry Smith <pedz@easesoftware.com> wrote:
>> So, your .o's compare the same each time?  Aside from the library
>> files (which we'll
>> ignore for a moment), I'm not clear what does not compare the same.
>
> .o files compare fine, it's these files from the build directory  
> that don't:
> -bxindex.header
> -strings.pbxstrings/control
> -symbols0.pbxsymbols.

I don't recognize those files or file types.  Maybe someone else can  
help.

How are they produced?


Perry Smith ( pedz@easesoftware.com )
Ease Software, Inc. ( http://www.easesoftware.com )

Low cost SATA Disk Systems for IBMs p5, pSeries, and RS/6000 AIX systems