[Bug lto/41526] New: gimple bytecode streams are not portable between different hosts

[Bug lto/41526] New: gimple bytecode streams are not portable between different hosts

[dnovillo at gcc dot gnu dot org]

>From http://gcc.gnu.org/ml/gcc-patches/2009-09/msg02148.html

I can see some issues here with output portability that should be 
addressed for objects with LTO information to be portable between hosts.

1. There are some structures lto_*header that include 16-bit or 32-bit 
integer fields (plus an enum).  It appears that the byte sequences for 
these headers get written out verbatim eithout regard for endianness 
issues, and reading in a file written on a host with the other endianness 
will result in an assertion failure.  Regarding the appropriateness of an 
assertion failure here see my comments on error handling, but it should 
not be hard to write out each field explicitly with defined endianness 
that does not depend on the endianness of the host.  There are other 
places writing out individual integers like this (at least 
write_global_references, lto_output_decl_state_refs, write_symbol_vec) 
that will also need fixing.

2. The storing of values in bitpacks may depend on the number of bits in 
various types.  I haven't looked at whether how the integers of size 
BITS_PER_BITPACK_WORD = HOST_BITS_PER_WIDE_INT are then written out to see 
whether the host endianness is also relevant there; if it is, that should 
be addressed.

  (a) The cases where HOST_BITS_PER_WIDE_INT bits are stored in a bitpack 
  would most simply be addressed by making LTO imply 64-bit (exactly) 
  HOST_WIDE_INT for all hosts and targets.  (I'd be happy for 
  HOST_WIDE_INT to be made 64-bit everywhere whether or not LTO is 
  enabled, in the interests of avoiding host-dependency in code 
  generation, but making it so for LTO may be less controversial.)

  (b) We can reasonably assume HOST_BITS_PER_INT to be 32, and 
  HOST_BITS_PER_SHORT to be 16, for all hosts, but a static assertion that 
  these are so if building LTO would ensure no problems arise with it 
  building but producing bad objects on unusual hosts.  Likewise for the 
  case using sizeof (unsigned) * 8.

  (c) REAL_CSTs, writing out HOST_BITS_PER_LONG bits, may take more care.  
  The obvious way is to encode them in target format before writing out, 
  then decode when reading in.


-- 
           Summary: gimple bytecode streams are not portable between
                    different hosts
           Product: gcc
           Version: lto
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: lto
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: dnovillo at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[dnovillo at gcc dot gnu dot org]

------- Comment #1 from dnovillo at gcc dot gnu dot org  2009-09-30 19:50 -------
Other portability concerns:

http://gcc.gnu.org/ml/gcc-patches/2009-09/msg02157.html

> The host portability issue with __attribute__ ((visibility ("hidden"))) 
> has already been noted.  I suggest conditioning the attributes on defined 
> (__GNUC__) && defined (__ELF__) - GCC should support the attribute on ELF 
> hosts, but LTO should also work for ELF targets with non-ELF hosts.

and

http://gcc.gnu.org/ml/gcc-patches/2009-09/msg02190.html

> > The host portability issue with __attribute__ ((visibility ("hidden"))) 
> > has already been noted.  I suggest conditioning the attributes on defined 
> > (__GNUC__) && defined (__ELF__) - GCC should support the attribute on ELF 
> > hosts, but LTO should also work for ELF targets with non-ELF hosts.
>
> Unfortunately, neither is true: older versions of GCC on Solaris 2 didn't
> support hidden with the vendor linker.  This was only fixed in GCC 4.4.0
> for Solaris 9 and up.  And IRIX 6 doesn't support hidden at all: the vendor
> linker knows nothing about it; maybe it can be made to work by using GNU
> ld, but I'm unsure if rld (runtime linker) support is needed as well.
> 
> Cf. PR bootstrap/39020 and bootstrap/39021.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[dnovillo at gcc dot gnu dot org]

------- Comment #2 from dnovillo at gcc dot gnu dot org  2009-09-30 20:17 -------
More portability concerns from

http://gcc.gnu.org/ml/gcc-patches/2009-09/msg02157.html

> +/* This needs to be included after config.h.  Otherwise, _GNU_SOURCE will not
> +   be defined in time to set __USE_GNU in the system headers, and strsignal
> +   will not be declared.  */
> +#include <sys/mman.h>

mmap and <sys/mman.h> are also not portable to all supported hosts (e.g. 
MinGW).  There should be configure tests and a suitable fallback.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[jsm28 at gcc dot gnu dot org]

------- Comment #3 from jsm28 at gcc dot gnu dot org  2010-09-01 19:29 -------
The handling of target options (the patch for bug 45475) is another
host-dependency in the bytecode stream.


-- 

jsm28 at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|0                           |1
   Last reconfirmed|0000-00-00 00:00:00         |2010-09-01 19:29:58
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[richard at weickelt dot de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

--- Comment #11 from Richard Weickelt <richard at weickelt dot de> ---
I would prefer a warning over an error. Cross-compiling for arm-none-eabi does
work unless floating point arithmetic is involved as reported in bug 105641,
but I understand that this might be a special case.

My use-case is that vendor A provides a binary blob of a library to vendor B.
Both A and B ensure that they use the same toolchain version, but the
developers involved use different host operating systems. LTO is a valuable
feature for us because it shrinks our binaries by 25%.

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|2010-09-01 19:29:58         |2022-7-4

--- Comment #10 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #9)
> *** Bug 106177 has been marked as a duplicate of this bug. ***

The float encoding issue was re-confirmed here.

I'm not sure if we meanwhile have a per-file LTO header but it might be
nice to put enough info there to give a reasonable error.  As said
endianess is a hard issue to fix, likewise int/long/pointer size differences.
The float encoding part could be fixed in theory.

target specific issues with host dependences might also exist (host
dependent flags might be streamed?).

Regression testing is another obvious problem here.

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |clyon at gcc dot gnu.org

--- Comment #9 from Richard Biener <rguenth at gcc dot gnu.org> ---
*** Bug 106177 has been marked as a duplicate of this bug. ***

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |richard at weickelt dot de

--- Comment #8 from Richard Biener <rguenth at gcc dot gnu.org> ---
*** Bug 105641 has been marked as a duplicate of this bug. ***

[Bug lto/41526] gimple bytecode streams are not portable between different hosts

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41526

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |daniel.f.starke at freenet dot de

--- Comment #7 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
*** Bug 69394 has been marked as a duplicate of this bug. ***