Binary built on different platforms with exact same tools different?

Binary built on different platforms with exact same tools different?

[Rick Mann]

Hi. I've built an xscale-elf toolchain from gcc-4.2.2, binutils-2.17,  
newlib-1.15.0. I've used the exact same build procedure on both Mac OS  
X 10.5 (gcc-4.0.1-based) and Cygwin (gcc 3.4-based).

Then I built our code and stripped the resulting binary with objcopy  
on each platform.

Both binaries seem to run the same.

However, cmp shows many differences between the two resulting  
binaries. Here are a few:

$ cmp -l ~/h h | head -10
      77  21  63
      81  32  74
      85  27  71
      97 370 220
      98 162 163
     101 134 364
     105 134 364
     141   4  15
     143  55 240
     144 345 341


Are these differences something to worry about?

TIA,
Rick

More info: Binary built on different platforms with exact same tools different?

[Rick Mann]

On Dec 17, 2007, at 4:43 PM, Rick Mann wrote:

> Hi. I've built an xscale-elf toolchain from gcc-4.2.2,  
> binutils-2.17, newlib-1.15.0. I've used the exact same build  
> procedure on both Mac OS X 10.5 (gcc-4.0.1-based) and Cygwin (gcc  
> 3.4-based).
>
> Then I built our code and stripped the resulting binary with objcopy  
> on each platform.
>
> Both binaries seem to run the same.
>
> However, cmp shows many differences between the two resulting  
> binaries. Here are a few:
>
> $ cmp -l ~/h h | head -10
>     77  21  63
>     81  32  74
>     85  27  71
>     97 370 220
>     98 162 163
>    101 134 364
>    105 134 364
>    141   4  15
>    143  55 240
>    144 345 341
>
>
> Are these differences something to worry about?

I've disassembled the .elf files, and here are some of the differences:

Mac OS X:

8000804c <loop>:
8000804c:	eb00a933 	bl	80032520 <_init>
80008050:	eb000c3c 	bl	8000b148 <main>
80008054:	eb00a939 	bl	80032540 <_fini>
80008058:	eaffffe8 	b	80008000 <//.>
8000805c:	e89da800 	ldmia	sp, {fp, sp, pc}
...
80008060 <bss_start>:
80008060:	80047390 	mulhi	r4, r0, r3

80008064 <bss_end>:
80008064:	803a51f4 	ldrhisht	r5, [sl], -r4


Cygwin:

8000804c <loop>:
8000804c:	eb00a911 	bl	80032498 <_init>
80008050:	eb000c1a 	bl	8000b0c0 <main>
80008054:	eb00a917 	bl	800324b8 <_fini>
80008058:	eaffffe8 	b	80008000 <//.>
8000805c:	e89da800 	ldmia	sp, {fp, sp, pc}
...
80008060 <bss_start>:
80008060:	800472f8 	strhid	r7, [r4], -r8

80008064 <bss_end>:
80008064:	803a515c 	eorhis	r5, sl, ip, asr r1


Some functions are placed in different order, too.

I'm trying to ensure that all our developers build identical binaries.

TIA,
-- 
Rick

Re: More info: Binary built on different platforms with exact same tools different?

[NightStrike]

On 12/17/07, Rick Mann <rmann@latencyzero.com> wrote:
> I'm trying to ensure that all our developers build identical binaries.

Why would binaries on different platforms be identical?

Re: More info: Binary built on different platforms with exact same tools different?

[Rick Mann]

On Dec 17, 2007, at 4:59 PM, NightStrike wrote:

> Why would binaries on different platforms be identical?

The toolchain on each host platform is built with identical steps,  
from the same source distro, targetting an embedded Xscale processor.

The source code does not use any OS APIs, just some things in newlib  
(built as part of the toolchain).

The resulting binary should, in theory, be identical, regardless of  
the host platform.


-- 
Rick

Re: More info: Binary built on different platforms with exact same tools different?

[NightStrike]

On 12/17/07, Rick Mann <rmann@latencyzero.com> wrote:
>
> On Dec 17, 2007, at 4:59 PM, NightStrike wrote:
>
> > Why would binaries on different platforms be identical?
>
> The toolchain on each host platform is built with identical steps,
> from the same source distro, targetting an embedded Xscale processor.
>
> The source code does not use any OS APIs, just some things in newlib
> (built as part of the toolchain).
>
> The resulting binary should, in theory, be identical, regardless of
> the host platform.

Ah, you are cross compiling.  I misunderstood.  So on both platforms,
you are building a cross compiler, gcc 4.2  You are then using that
cross compiler that you build on each platform to build your program,
and the resulting executable is different.

Hmm....  Interesting scenario.

Re: More info: Binary built on different platforms with exact same tools different?

[Rick Mann]

On Dec 17, 2007, at 5:11 PM, NightStrike wrote:

> Ah, you are cross compiling.  I misunderstood.  So on both platforms,
> you are building a cross compiler, gcc 4.2  You are then using that
> cross compiler that you build on each platform to build your program,
> and the resulting executable is different.

Ding! Sorry for being unclear.

> Hmm....  Interesting scenario.

Heh. Yeah.

-- 
Rick

Re: More info: Binary built on different platforms with exact same tools different?

[Kaz Kylheku]

From: "Rick Mann" <rmann@latencyzero.com>
> On Dec 17, 2007, at 4:59 PM, NightStrike wrote:
>
>> Why would binaries on different platforms be identical?
>
> The toolchain on each host platform is built with identical steps,  from 
> the same source distro, targetting an embedded Xscale processor.
>
> The source code does not use any OS APIs, just some things in newlib 
> (built as part of the toolchain).

So you might think.

However, it's easy for code to become polluted by stuff from the build 
machine.

I knows this from experience, having developed a cross-compiled Linux 
distribution from scratch.

In one case I had a failure because some /usr/include header got included, 
and as a result, inline asm's routines for x86 were being mixed into MIPS 
code.

Make sure your GCC is properly sysrooted so it's not picking up local header 
files, etc.

> The resulting binary should, in theory, be identical, regardless of  the 
> host platform.

And of course the input to the compiler after all header file inclusion and 
preprocessing should also be the same.

Here is something to try: compile your code with -E in both environments to 
capture the raw preprocessor output. You don't want any differences there, 
that's for sure.