Same cross-gcc toolchain on different hosts produces different target code?

Same cross-gcc toolchain on different hosts produces different target code?

[McCall, Ronald SIK]

Hi,

I am attempting to move an old gcc powerpc-eabi cross toolchain from an old Solaris SPARC server to a new Linux x86_64 server.  I have built and installed the exact same cross-binutils (2.13.2.1), cross-gcc (3.2.3) and cross-newlib (1.11.0) from source using the exact same target (powerpc-eabi) as I originally built and installed on the Solaris server.

I then built the exact same target source code using the two toolchains and got different target object code (i.e. different instruction sequences).  Should that be possible?

The obvious difference is that the two toolchains are running on different hosts but I wouldn't have expected that to matter (the cross-compiler should have identical functionality).  Another difference is that the two toolchains were compiled with different versions of native gcc.  Native gcc 3.2.3 was originally used on the Solaris server (with Solaris binutils) while native gcc 3.4.6 was used on the Linux server (with GNU binutils).  I wouldn't have expected that to matter either.

Any theories on why the object code is different between the two toolchains?

Ron McCall

Re: Same cross-gcc toolchain on different hosts produces different target code?

[David Daney]

On 03/17/2011 09:59 AM, McCall, Ronald SIK wrote:
> Hi,
>
> I am attempting to move an old gcc powerpc-eabi cross toolchain
> from an old Solaris SPARC server to a new Linux x86_64 server.  I
> have built and installed the exact same cross-binutils (2.13.2.1),
> cross-gcc (3.2.3) and cross-newlib (1.11.0) from source using the
> exact same target (powerpc-eabi) as I originally built and installed
> on the Solaris server.
>
> I then built the exact same target source code using the two
> toolchains and got different target object code (i.e. different
> instruction sequences).  Should that be possible?
>
> The obvious difference is that the two toolchains are running on
> different hosts but I wouldn't have expected that to matter (the
> cross-compiler should have identical functionality).  Another
> difference is that the two toolchains were compiled with different
> versions of native gcc.  Native gcc 3.2.3 was originally used on the
> Solaris server (with Solaris binutils) while native gcc 3.4.6 was
> used on the Linux server (with GNU binutils).  I wouldn't have
> expected that to matter either.
>
> Any theories on why the object code is different between the two
> toolchains?

If you let us in on what exactly the secret differences were, it would 
be easier to opine on this topic.

David Daney

Re: Same cross-gcc toolchain on different hosts produces different target code?

[David Daney]

On 03/17/2011 11:20 AM, McCall, Ronald SIK wrote:
>> If you let us in on what exactly the secret differences were, it would
>> be easier to opine on this topic.
>
> Sure thing!  Here is an instruction sequence from the original Solaris toolchain:
>

Resending to gcc@.  I didn't really want a private message about it.

> fe000230:   54 6a 87 be     rlwinm  r10,r3,16,30,31
> fe000234:   65 49 ff ff     oris    r9,r10,65535
> fe000238:   61 28 ff fc     ori r8,r9,65532
> fe00023c:   6d 07 80 00     xoris   r7,r8,32768
> fe000240:   3c 00 43 30     lis r0,17200
> fe000244:   90 e1 00 0c     stw r7,12(r1)
> fe000248:   90 01 00 08     stw r0,8(r1)
> fe00024c:   3c 80 fe 14     lis r4,-492
> fe000250:   c8 01 00 08     lfd f0,8(r1)
> fe000254:   39 24 82 38     addi    r9,r4,-32200<<<  Instruction sequence #1
> fe000258:   c8 89 00 00     lfd f4,0(r9)<<<  (continued)
> fe00025c:   74 60 00 08     andis.  r0,r3,8
> fe000260:   3c c0 00 01     lis r6,1
> fe000264:   fc 60 20 28     fsub    f3,f0,f4<<<  Instruction sequence #2
> fe000268:   fc 40 18 18     frsp    f2,f3
> fe00026c:   fc 20 10 90     fmr f1,f2
> fe000270:   fc 00 08 1e     fctiwz  f0,f1
> fe000274:   d8 01 00 10     stfd    f0,16(r1)
> fe000278:   80 81 00 14     lwz r4,20(r1)
> fe00027c:   98 86 8f 0d     stb r4,-28915(r6)
> fe000280:   41 82 00 cc     beq-    fe00034c
>
> Here is the same instruction sequence from the newly built Linux toolchain:
>
> fe000230:   54 6a 87 be     rlwinm  r10,r3,16,30,31
> fe000234:   65 49 ff ff     oris    r9,r10,65535
> fe000238:   61 28 ff fc     ori r8,r9,65532
> fe00023c:   6d 07 80 00     xoris   r7,r8,32768
> fe000240:   3c 00 43 30     lis r0,17200
> fe000244:   90 e1 00 0c     stw r7,12(r1)
> fe000248:   90 01 00 08     stw r0,8(r1)
> fe00024c:   3c 80 fe 14     lis r4,-492
> fe000250:   c8 01 00 08     lfd f0,8(r1)
> fe000254:   c9 a4 87 b0     lfd f13,-30800(r4)<<<  Instruction sequence #1
> fe000258:   fc 60 68 28     fsub    f3,f0,f13<<<  Instruction sequence #2
> fe00025c:   74 60 00 08     andis.  r0,r3,8
> fe000260:   3c 80 00 01     lis r4,1
> fe000264:   fc 40 18 18     frsp    f2,f3
> fe000268:   fc 20 10 90     fmr f1,f2
> fe00026c:   fc 00 08 1e     fctiwz  f0,f1
> fe000270:   d8 01 00 10     stfd    f0,16(r1)
> fe000274:   80 e1 00 14     lwz r7,20(r1)
> fe000278:   98 e4 8f 0d     stb r7,-28915(r4)
> fe00027c:   41 82 00 c8     beq-    fe000344
>
> Instruction sequence #1 has been combined into a single equivalent instruction.  Instruction sequence #2 moved.  Register usage is also different but equivalent.
>
> Ron McCall

Re: Same cross-gcc toolchain on different hosts produces different target code?

[Jeff Law]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/17/11 12:25, David Daney wrote:
> On 03/17/2011 11:20 AM, McCall, Ronald SIK wrote:
>>> If you let us in on what exactly the secret differences were, it would
>>> be easier to opine on this topic.
>>
>> Sure thing!  Here is an instruction sequence from the original Solaris
>> toolchain:
>>
> 
> Resending to gcc@.  I didn't really want a private message about it.
> 
>> fe000230:   54 6a 87 be     rlwinm  r10,r3,16,30,31
>> fe000234:   65 49 ff ff     oris    r9,r10,65535
>> fe000238:   61 28 ff fc     ori r8,r9,65532
>> fe00023c:   6d 07 80 00     xoris   r7,r8,32768
>> fe000240:   3c 00 43 30     lis r0,17200
>> fe000244:   90 e1 00 0c     stw r7,12(r1)
>> fe000248:   90 01 00 08     stw r0,8(r1)
>> fe00024c:   3c 80 fe 14     lis r4,-492
>> fe000250:   c8 01 00 08     lfd f0,8(r1)
>> fe000254:   39 24 82 38     addi    r9,r4,-32200<<<  Instruction
>> sequence #1
>> fe000258:   c8 89 00 00     lfd f4,0(r9)<<<  (continued)
>> fe00025c:   74 60 00 08     andis.  r0,r3,8
>> fe000260:   3c c0 00 01     lis r6,1
>> fe000264:   fc 60 20 28     fsub    f3,f0,f4<<<  Instruction sequence #2
>> fe000268:   fc 40 18 18     frsp    f2,f3
>> fe00026c:   fc 20 10 90     fmr f1,f2
>> fe000270:   fc 00 08 1e     fctiwz  f0,f1
>> fe000274:   d8 01 00 10     stfd    f0,16(r1)
>> fe000278:   80 81 00 14     lwz r4,20(r1)
>> fe00027c:   98 86 8f 0d     stb r4,-28915(r6)
>> fe000280:   41 82 00 cc     beq-    fe00034c
>>
>> Here is the same instruction sequence from the newly built Linux
>> toolchain:
>>
>> fe000230:   54 6a 87 be     rlwinm  r10,r3,16,30,31
>> fe000234:   65 49 ff ff     oris    r9,r10,65535
>> fe000238:   61 28 ff fc     ori r8,r9,65532
>> fe00023c:   6d 07 80 00     xoris   r7,r8,32768
>> fe000240:   3c 00 43 30     lis r0,17200
>> fe000244:   90 e1 00 0c     stw r7,12(r1)
>> fe000248:   90 01 00 08     stw r0,8(r1)
>> fe00024c:   3c 80 fe 14     lis r4,-492
>> fe000250:   c8 01 00 08     lfd f0,8(r1)
>> fe000254:   c9 a4 87 b0     lfd f13,-30800(r4)<<<  Instruction
>> sequence #1
>> fe000258:   fc 60 68 28     fsub    f3,f0,f13<<<  Instruction sequence #2
>> fe00025c:   74 60 00 08     andis.  r0,r3,8
>> fe000260:   3c 80 00 01     lis r4,1
>> fe000264:   fc 40 18 18     frsp    f2,f3
>> fe000268:   fc 20 10 90     fmr f1,f2
>> fe00026c:   fc 00 08 1e     fctiwz  f0,f1
>> fe000270:   d8 01 00 10     stfd    f0,16(r1)
>> fe000274:   80 e1 00 14     lwz r7,20(r1)
>> fe000278:   98 e4 8f 0d     stb r7,-28915(r4)
>> fe00027c:   41 82 00 c8     beq-    fe000344
>>
>> Instruction sequence #1 has been combined into a single equivalent
>> instruction.  Instruction sequence #2 moved.  Register usage is also
>> different but equivalent.
In my experience, there's all kinds of reasons why this can happen.

Use of floating point within the compiler (we generally avoid it),
unstable qsort (again, we generally try to avoid it), hashing issues,
particularly with pointers, etc etc.

For those reasons and many more, we don't try to guarantee that a
compiler running on different hosts will generate 100% identical target
code.

jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJNglL7AAoJEBRtltQi2kC7kgsH/R1jVnxVlHimB09T56wJmU82
T3XtZRusl7j1vLNhzLDvxS+QpHz8bwL0mJFzDhJOtpPEm3gwTPQ9DSh32RGDbZ92
7ZgtsmHYTuSfZD2isUxLKBTi0PPs/Lv8bcmcwhhi11/QjppaLrQCycbzoALei4tF
BB1feKLwoYQFWK6PuOcdB1TCJZd0frAjcu93qHDbq04CJj3/EGPqBzi/TcEkAL0S
WQkWdmWtxKLtrT0dR+PiqO5PKdJ21Zl7qyIeOdFYepwYxQ7FjpAVMe1p4Wr7uc0J
DVoApYBWmD+Nkv/pP4kDqKxWnp48Jtd95qKh9frgPpoaY+fCIKtSENFxKqTZt9c=
=vHxO
-----END PGP SIGNATURE-----

RE: Same cross-gcc toolchain on different hosts produces different target code?

[McCall, Ronald SIK]

> In my experience, there's all kinds of reasons why this can happen.
>
> Use of floating point within the compiler (we generally avoid it),
> unstable qsort (again, we generally try to avoid it), hashing issues,
> particularly with pointers, etc etc.
>
> For those reasons and many more, we don't try to guarantee that a
> compiler running on different hosts will generate 100% identical target
> code.

OK, that certainly explains the issue.  Thank you very much!

Ron McCall

Re: Same cross-gcc toolchain on different hosts produces different target code?

[Dave Korn]

On 17/03/2011 18:29, Jeff Law wrote:
> On 03/17/11 12:25, David Daney wrote:
>>> Instruction sequence #1 has been combined into a single equivalent
>>> instruction.  Instruction sequence #2 moved.  Register usage is also
>>> different but equivalent.
> In my experience, there's all kinds of reasons why this can happen.
> 
> Use of floating point within the compiler (we generally avoid it),
> unstable qsort (again, we generally try to avoid it), hashing issues,
> particularly with pointers, etc etc.

  In particular, isn't it a known issue that 32-bit hosts and 64-bit hosts can
generate different constant-loading sequences owing to the differing size of
HOST_WIDE_INT?

On 17/03/2011 16:59, McCall, Ronald SIK wrote:

> I am attempting to move an old gcc powerpc-eabi cross toolchain from an old
> Solaris SPARC server to a new Linux x86_64 server.

  Was that 32-bit Solaris?

    cheers,
      DaveK

Re: Same cross-gcc toolchain on different hosts produces different target code?

[Joern Rennecke]

Quoting Jeff Law <law@redhat.com>:

> Use of floating point within the compiler (we generally avoid it),
> unstable qsort (again, we generally try to avoid it), hashing issues,
> particularly with pointers, etc etc.

The floating point issues also mean that the compiler behaviour is
not defined by the standard; the others are even worse, potentially
or manifestly causing the compiler to have random output (in particular
when address space randomization is in effect, and/or when shared
libraries are moving around).  These things can make debugging the compiler,
checking pre-compiled headers, and/or marginal performance comparisons so
much harder that it is easiest to fix the unpredictability issue first.
Moreover, when a hashing issue is about generating different hashes for
things that compare equal, if the issue doesn't cause wrong code, it is
likely to cause missed optimizations (with rare instances when hash
collisions make things work).

So, I would say that inconsistent / unpredictable compilation results are
generally worth fixing, assuming you have or can make a suitable patch.

> For those reasons and many more, we don't try to guarantee that a
> compiler running on different hosts will generate 100% identical target
> code.

Of course, guarantees are rare for open-source projects (and for most kinds
of software, for that matter), and also, on a pragmatic level, the impact
from inconsistent compiler output on end-users tends to be low, so these
issues tend to only get addressed when they impact compiler development.

As GCC 3.2.3 is fairly old, if the original poster really wants it fixed,
he'll have to hack the compiler himself, or hire somebody to do it for him.

RE: Same cross-gcc toolchain on different hosts produces different target code?

[McCall, Ronald SIK]

> In particular, isn't it a known issue that 32-bit hosts and 64-bit hosts can
> generate different constant-loading sequences owing to the differing size of
> HOST_WIDE_INT?
>
> Was that 32-bit Solaris?
>
> cheers,
> DaveK

The system is 64-bit Solaris 9 but the native gcc is sparc-sun-solaris2.9 which I believe is 32-bit (sparcv9-sun-solaris2.9 is the 64-bit target if memory serves).  

RE: Same cross-gcc toolchain on different hosts produces different target code?

[McCall, Ronald SIK]

> In particular, isn't it a known issue that 32-bit hosts and 64-bit hosts can
> generate different constant-loading sequences owing to the differing size of
> HOST_WIDE_INT?
>
> Was that 32-bit Solaris?
>
> cheers,
> DaveK

As a followup, I built a 32-bit version of the entire toolchain and it produces identical object code as compared with the Solaris hosted toolchain.  So, a 64-bit host versus a 32-bit host does indeed make a difference in this case.

Thanks for all the help!

Ron McCall