sysroot issue when cross-compiling a native compiler (gcc 7.3.0)

sysroot issue when cross-compiling a native compiler (gcc 7.3.0)

[Stefan Ring]

I want to build a native compiler for a very dated ARM machine
(SheevaPlug) running Fedora 18 and have to use a cross-compiler due to
the limited amount of memory available (and also 24h builds not being
fun). The cross compiler is gcc 7.3.0 as well and is running on
x86_64. I configure like this:

/tmp/gcc-7.3.0/configure --build= --host=arm-unknown-linux-gnueabi
--target=arm-unknown-linux-gnueabi --prefix=/home/sr/gcc7
--with-build-sysroot=$HOME/xtools-arm5/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot
--with-float=soft --with-abi=aapcs-linux --with-arch=armv5te
--with-mode=arm --enable-languages=c,c++ --enable-__cxa_atexit
--disable-nls --disable-multilib

The build and the resulting compiler does work if I also add
'--with-sysroot=/'. The only problem with this is that I have to build
binutils myself on the target because the system ld complains about
not supporting sysroots. Therefore I'd like to leave it out. This
works mostly fine except for the value of the 'target_header_dir'
variable in gcc/configure which does not get set up to what I consider
the correct value. When I observe the value there, I notice that (this
part of) the script is run twice. The first time the result is
'/usr/local/arm-unknown-linux-gnueabi/sys-include', which does not
exist, and the second time '/usr/include'. I don't quite understand
why with_build_sysroot does not seem to evaluate to the correct value
(specified on the command line).

[snippet from gcc/configure]
if test x$host != x$target || test "x$TARGET_SYSTEM_ROOT" != x; then
  if test "x$with_headers" != x && test "x$with_headers" != xyes; then
    target_header_dir=$with_headers
  elif test "x$with_sysroot" = x; then
    target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-include"
  elif test "x$with_build_sysroot" != "x"; then
    target_header_dir="${with_build_sysroot}${native_system_header_dir}"
  elif test "x$with_sysroot" = xyes; then
    target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-root${native_system_header_dir}"
  else
    target_header_dir="${with_sysroot}${native_system_header_dir}"
  fi
else
  target_header_dir=${native_system_header_dir}
fi
#target_header_dir=/home/sr/xtools-arm5/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include
echo Final target_header_dir $target_header_dir

I noticed the problem because it would break like this late in the build:

In file included from /tmp/gcc-7.3.0/libgcc/config/arm/unwind-arm.c:143:0:
/tmp/gcc-7.3.0/libgcc/unwind-arm-common.inc:30:10: fatal error:
sys/sdt.h: No such file or directory
 #include <sys/sdt.h>
          ^~~~~~~~~~~
compilation terminated.
make[2]: *** [unwind-arm.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory
`/home/sr/temp/gcbuild-testcross/arm-unknown-linux-gnueabi/libgcc'
make[1]: *** [all-target-libgcc] Error 2
make[1]: Leaving directory `/home/sr/temp/gcbuild-testcross'
make: *** [all] Error 2

Because the configure script looked at the system include directory to
determine the presence of 'sys/sdt.h'. I hard-coded the correct path
for 'target_header_dir' into the configure script (commented out
above), and everything worked perfectly. Maybe it did not use the
correct header files after all during the build, but at least this
does not seem to have an adverse effect.

Re: sysroot issue when cross-compiling a native compiler (gcc 7.3.0)

[Stefan Ring]

On Sat, Dec 8, 2018 at 12:09 AM Stefan Ring <stefanrin@gmail.com> wrote:
> In file included from /tmp/gcc-7.3.0/libgcc/config/arm/unwind-arm.c:143:0:
> /tmp/gcc-7.3.0/libgcc/unwind-arm-common.inc:30:10: fatal error:
> sys/sdt.h: No such file or directory
>  #include <sys/sdt.h>
>           ^~~~~~~~~~~
> compilation terminated.
> make[2]: *** [unwind-arm.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> make[2]: Leaving directory
> `/home/sr/temp/gcbuild-testcross/arm-unknown-linux-gnueabi/libgcc'
> make[1]: *** [all-target-libgcc] Error 2
> make[1]: Leaving directory `/home/sr/temp/gcbuild-testcross'
> make: *** [all] Error 2

It thought, I'd retry the same procedure with gcc 8.2.0 today. This
target header issue is indeed gone -- the build just works. Alas, the
resulting compiler does not :(.

$ ~/gcc8/bin/g++ -x c++ x.c
during IPA pass: inline
x.c:13:1: internal compiler error: in inline_small_functions, at
ipa-inline.c:1865
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.

For this program:

#include <stdio.h>

#include <iostream>

using std::cout;
using std::endl;

int main()
{
    printf("hello!\n");
    cout << "hello2!" << endl;
    return 0;
}

Re: sysroot issue when cross-compiling a native compiler (gcc 7.3.0)

[Stefan Ring]

On Sat, Dec 8, 2018 at 11:10 AM Stefan Ring <stefanrin@gmail.com> wrote:
> On Sat, Dec 8, 2018 at 12:09 AM Stefan Ring <stefanrin@gmail.com> wrote:
> It thought, I'd retry the same procedure with gcc 8.2.0 today. This
> target header issue is indeed gone -- the build just works. Alas, the
> resulting compiler does not :(.
>
> $ ~/gcc8/bin/g++ -x c++ x.c
> during IPA pass: inline
> x.c:13:1: internal compiler error: in inline_small_functions, at
> ipa-inline.c:1865
>  }

This is only partially the compiler's fault. I noticed that it worked
fine on an arm64 machine chrooted into the armv5 environment and
remembered that /proc/cpu/alignment (used to?) default to 0 on Fedora
ARM, causing misaligned accesses to produce garbage. It works after
activating mis-alignment fixup (setting the value to 2). But this
means that gcc version 8 has introduced some unaligned accesses
lurking somewhere in its code.

Re: sysroot issue when cross-compiling a native compiler (gcc 7.3.0)

[Stefan Ring]

On Sat, Dec 8, 2018 at 11:26 AM Stefan Ring <stefanrin@gmail.com> wrote:
> On Sat, Dec 8, 2018 at 11:10 AM Stefan Ring <stefanrin@gmail.com> wrote:
> > On Sat, Dec 8, 2018 at 12:09 AM Stefan Ring <stefanrin@gmail.com> wrote:
> > $ ~/gcc8/bin/g++ -x c++ x.c
> > during IPA pass: inline
> > x.c:13:1: internal compiler error: in inline_small_functions, at
> > ipa-inline.c:1865
> >  }
>
> This is only partially the compiler's fault. I noticed that it worked
> fine on an arm64 machine chrooted into the armv5 environment and
> remembered that /proc/cpu/alignment (used to?) default to 0 on Fedora
> ARM, causing misaligned accesses to produce garbage. It works after
> activating mis-alignment fixup (setting the value to 2). But this
> means that gcc version 8 has introduced some unaligned accesses
> lurking somewhere in its code.

This seems to be a stack alignment issue. The last ldrd instruction traps.

Dump of assembler code for function
cgraph_node::create_edge(cgraph_node*, gcall*, profile_count):
   0x00398e64 <+0>:     sub     sp, sp, #8
   0x00398e68 <+4>:     push    {r4, r5, r6, r7, r8, lr}
   0x00398e6c <+8>:     sub     sp, sp, #24
   0x00398e70 <+12>:    str     r3, [sp, #3280068]      ; 0x34
   0x00398e74 <+16>:    ldrd    r6, [sp, #52]   ; 0x34

All of the sp manipulations leading up to the ldrd instruction work on
multiples of 8, which 52 is not. So the function will only work if the
stack is _not_ aligned to 8 bytes at function entry, which is rather
strange. This is not just a consequence of my exotic cross build
setup, but happens in the exact same way with a native build in the
Fedora armv5 chroot with only harmless configure arguments:
--build=armv5tel-unknown-linux-gnueabi --prefix=$HOME/gcc8
--enable-languages=c,c++ --with-arch=armv5te --with-mode=arm
--disable-nls. I think it's almost time for opening a bug report. I
will check how armv7 hardfp behaves before doing that.