RFC: Extending --with-advance-toolchain to aarch64

RFC: Extending --with-advance-toolchain to aarch64

[Steve Ellcey]

I have a question about building a toolchain that uses (at run time) a
dynamic linker and system libraries and headers that are in a non-standard
place.

I just noticed the IBM --with-advance-toolchain option and I would
like to replicate it for aarch64.

Let me first describe what I do now:

configure/build BINUTILS with --prefix=${X} --with-sysroot=${X}
configure/build an initial GCC (all-gcc all-target-libgcc) with
	--prefix=${X} --with-sysroot=${X}
configure/build GLIBC, using that GCC, with --prefix=/usr,
	followed by install with DESTDIR=${X}
configure/build final GCC with --prefix=${X} --with-sysroot=${X}

This all works, but if I want my executables to find the shared libraries and
dynamic linker from ${X} when they are running, I need to compile things with:

   -Wl,--rpath=${X}/lib64 -Wl,--dynamic-linker=${X}/lib/ld-linux-aarch64.so.1

I would like these used by default so I took some ideas from
--with-advance-toolchain and used that to automatically add these options
to LINK_SPEC (see attached patch).  I can compile and link a program with
this setup, but when I run the program I get:

% ./x
Inconsistency detected by ld.so: get-dynamic-info.h: 147: elf_get_dynamic_info: 
Assertion `info[DT_RPATH] == NULL' failed!

I am not sure why this doesn't work.  Can anyone help me understand
why this doesn't work or help me figure out how else I might be able to
get the functionality I want. That is: to use shared libraries and a dynamic
linker (at run time) that are in a non-standard location without needing
to compile or link with special flags.

Steve Ellcey
sellcey@marvell.com


Here is the patch I am trying, I use the --with-advance-toolchain option as
an absolute pathname instead of relative to /opt like IBM does and I set it
to ${X} in a build that otherwise looks like what I describe above.  Everything
works until I start the final GCC build which is when I get the assertion.


diff --git a/gcc/config.gcc b/gcc/config.gcc
index 481bc9586a7..0532139b0b1 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -3879,7 +3879,7 @@ fi
 supported_defaults=
 case "${target}" in
        aarch64*-*-*)
-               supported_defaults="abi cpu arch"
+               supported_defaults="abi cpu arch advance_toolchain"
                for which in cpu arch; do
 
                        eval "val=\$with_$which"
@@ -3981,6 +3981,23 @@ case "${target}" in
                          exit 1
                        fi
                done
+               if test "x$with_advance_toolchain" != x; then
+                   at=$with_advance_toolchain
+                   if test -d "$at/." -a -d "$at/include/."; then
+                       tm_file="$tm_file ./advance-toolchain.h"
+                       (
+                        echo "/* Use Advance Toolchain $at */"
+                        echo "#undef  LINK_ADVANCE_SPEC"
+                        echo "#define LINK_ADVANCE_SPEC" \
+                          "\"--rpath=$at/lib%{mabi=ilp32:ilp32}%{mabi=lp64:64} \
+                          "--rpath=$at/usr/lib%{mabi=ilp32:ilp32}%{mabi=lp64:64} \
+                          "--dynamic-linker=$at/lib/ld-linux-aarch64%{mbig-endian:_be}%{mabi=ilp32:_ilp32}.so.1\""
+                       ) > advance-toolchain.h
+                   else
+                       echo "Unknown advance-toolchain $at"
+                       exit 1
+                   fi
+               fi
                ;;
 
        alpha*-*-*)
diff --git a/gcc/config/aarch64/aarch64-linux.h b/gcc/config/aarch64/aarch64-linux.h
index 6ff2163b633..d76fa56c73e 100644
--- a/gcc/config/aarch64/aarch64-linux.h
+++ b/gcc/config/aarch64/aarch64-linux.h
@@ -47,7 +47,10 @@
    -maarch64linux%{mabi=ilp32:32}%{mbig-endian:b}"
 
 
-#define LINK_SPEC LINUX_TARGET_LINK_SPEC AARCH64_ERRATA_LINK_SPEC
+#ifndef LINK_ADVANCE_SPEC
+#define LINK_ADVANCE_SPEC
+#endif
+#define LINK_SPEC LINUX_TARGET_LINK_SPEC AARCH64_ERRATA_LINK_SPEC LINK_ADVANCE_SPEC
 
 #define GNU_USER_TARGET_MATHFILE_SPEC \
   "%{Ofast|ffast-math|funsafe-math-optimizations:crtfastmath.o%s}"

Re: RFC: Extending --with-advance-toolchain to aarch64

[Alan Modra]

On Wed, Oct 09, 2019 at 10:29:48PM +0000, Steve Ellcey wrote:
> I have a question about building a toolchain that uses (at run time) a
> dynamic linker and system libraries and headers that are in a non-standard
> place.

I had scripts a long time ago to build a complete toolchain including
glibc that could be installed in a non-standard location and co-exist
with other system libraries.  I worked around..

> Inconsistency detected by ld.so: get-dynamic-info.h: 147: elf_get_dynamic_info: 
> Assertion `info[DT_RPATH] == NULL' failed!

..this by patching glibc.

-- 
Alan Modra
Australia Development Lab, IBM

Re: RFC: Extending --with-advance-toolchain to aarch64

[Steve Ellcey]

On Thu, 2019-10-10 at 10:49 +1030, Alan Modra wrote:
> On Wed, Oct 09, 2019 at 10:29:48PM +0000, Steve Ellcey wrote:
> > I have a question about building a toolchain that uses (at run
> > time) a
> > dynamic linker and system libraries and headers that are in a non-
> > standard
> > place.
> 
> I had scripts a long time ago to build a complete toolchain including
> glibc that could be installed in a non-standard location and co-exist
> with other system libraries.  I worked around..
> 
> > Inconsistency detected by ld.so: get-dynamic-info.h: 147:
> > elf_get_dynamic_info: 
> > Assertion `info[DT_RPATH] == NULL' failed!
> 
> ..this by patching glibc.

Yes, I have something working by patching glibc (and gcc) but when I
saw the IBM --with-advance-toolchain option I was hoping I might be
able to come up with a build process that worked and did not need any
patching.

Steve Ellcey
sellcey@marvell.com

Re: RFC: Extending --with-advance-toolchain to aarch64

[Florian Weimer]

* Steve Ellcey:

> I would like these used by default so I took some ideas from
> --with-advance-toolchain and used that to automatically add these options
> to LINK_SPEC (see attached patch).  I can compile and link a program with
> this setup, but when I run the program I get:
>
> % ./x
> Inconsistency detected by ld.so: get-dynamic-info.h: 147: elf_get_dynamic_info: 
> Assertion `info[DT_RPATH] == NULL' failed!
>
> I am not sure why this doesn't work.  Can anyone help me understand
> why this doesn't work or help me figure out how else I might be able to
> get the functionality I want. That is: to use shared libraries and a dynamic
> linker (at run time) that are in a non-standard location without needing
> to compile or link with special flags.

An argument could be made that if ld.so has DT_RPATH set,
LD_LIBRARY_PATH would stop working, which would be a bug.  Hence the
assert.  It's probably less an issue for DT_RUNPATH.

The real fix would be to make sure that ld.so isn't built with those
dynamic tags.  If ld.so wants to use an alternative search path, that
should be baked into the loader itself, explicitly.

Do you know where those dynamic tags originate?  Is there some wrapper
script involved that sets them unconditionally?

Thanks,
Florian

Re: RFC: Extending --with-advance-toolchain to aarch64

[Steve Ellcey]

On Thu, 2019-10-10 at 18:41 +0200, Florian Weimer wrote:
> 
> * Steve Ellcey:
> 
> > I would like these used by default so I took some ideas from
> > --with-advance-toolchain and used that to automatically add these options
> > to LINK_SPEC (see attached patch).  I can compile and link a program with
> > this setup, but when I run the program I get:
> > 
> > % ./x
> > Inconsistency detected by ld.so: get-dynamic-info.h: 147: elf_get_dynamic_info: 
> > Assertion `info[DT_RPATH] == NULL' failed!
> > 
> > I am not sure why this doesn't work.  Can anyone help me understand
> > why this doesn't work or help me figure out how else I might be able to
> > get the functionality I want. That is: to use shared libraries and a dynamic
> > linker (at run time) that are in a non-standard location without needing
> > to compile or link with special flags.
> 
> An argument could be made that if ld.so has DT_RPATH set,
> LD_LIBRARY_PATH would stop working, which would be a bug.  Hence the
> assert.  It's probably less an issue for DT_RUNPATH.
> 
> The real fix would be to make sure that ld.so isn't built with those
> dynamic tags.  If ld.so wants to use an alternative search path, that
> should be baked into the loader itself, explicitly.
> 
> Do you know where those dynamic tags originate?  Is there some wrapper
> script involved that sets them unconditionally?

I am not sure, but my guess is that it is because I am building
binutils (including ld) using --with-sysroot.  I build both GCC and
binutils with the sysroot directory where I put the glibc that I am
building.  Maybe I should try building GCC with --with-sysroot but
build binutils without it.

Steve Ellcey
sellcey@marvell.com

Re: RFC: Extending --with-advance-toolchain to aarch64

[Tulio Magno Quites Machado Filho]

Steve Ellcey <sellcey@marvell.com> writes:

> I have a question about building a toolchain that uses (at run time) a
> dynamic linker and system libraries and headers that are in a non-standard
> place.
>
> I just noticed the IBM --with-advance-toolchain option and I would
> like to replicate it for aarch64.
>
> Let me first describe what I do now:
>
> configure/build BINUTILS with --prefix=${X} --with-sysroot=${X}
> configure/build an initial GCC (all-gcc all-target-libgcc) with
> 	--prefix=${X} --with-sysroot=${X}
> configure/build GLIBC, using that GCC, with --prefix=/usr,
> 	followed by install with DESTDIR=${X}

Can you use --prefix=${X}?

> configure/build final GCC with --prefix=${X} --with-sysroot=${X}
>
> This all works, but if I want my executables to find the shared libraries and
> dynamic linker from ${X} when they are running, I need to compile things with:
>
>    -Wl,--rpath=${X}/lib64 -Wl,--dynamic-linker=${X}/lib/ld-linux-aarch64.so.1

If you set glibc's --prefix to ${X}, I don't think you need to set --rpath too.
I have plans to work on a patch for the Advance Toolchain in order to avoid
this.

> I would like these used by default so I took some ideas from
> --with-advance-toolchain and used that to automatically add these options
> to LINK_SPEC (see attached patch).  I can compile and link a program with
> this setup, but when I run the program I get:
>
> % ./x
> Inconsistency detected by ld.so: get-dynamic-info.h: 147: elf_get_dynamic_info: 
> Assertion `info[DT_RPATH] == NULL' failed!

Florian already explained why glibc has this test.
But the Advance Toolchain carries the following patch:
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=9ca2648e2aa7094e022d5150281b2575f866259f

-- 
Tulio Magno

Re: RFC: Extending --with-advance-toolchain to aarch64

[Steve Ellcey]

On Thu, 2019-10-10 at 15:38 -0300, Tulio Magno Quites Machado Filho wrote:
> 
> > Let me first describe what I do now:
> > 
> > configure/build BINUTILS with --prefix=${X} --with-sysroot=${X}
> > configure/build an initial GCC (all-gcc all-target-libgcc) with
> > 	--prefix=${X} --with-sysroot=${X}
> > configure/build GLIBC, using that GCC, with --prefix=/usr,
> > 	followed by install with DESTDIR=${X}
> 
> Can you use --prefix=${X}?

I can.  I would rather not, because when you don't have prefix set to
/usr you get a different glibc build.  For example, on aarch64 building
with --prefix=/usr means that libraries are put in lib64 (or libilp32)
instead of just lib.  The glibc folks are always preaching against 
building with a prefix of anything other than /usr.

> 
> Florian already explained why glibc has this test.
> But the Advance Toolchain carries the following patch:
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceware.org_git_-3Fp-3Dglibc.git-3Ba-3Dcommitdiff-3Bh-3D9ca2648e2aa7094e022d5150281b2575f866259f&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=Kj0CuWu6MgrNHos80CzrFt4fiXgwrFhMWDTO9Ue_lRU&m=zJmKExSapjGitHa0CdqSuR7k0QkL_7nNpzI76Y8XSLs&s=oE8dt9sjEr5MEtYG4c_pIgGtWYh2ZH3CG1jPypnGAdg&e=
> 

Ah, I see.  I was hoping that using --with-advance-toolchain would give
me a way to build a toolchain without needing any local/non-standard
patches.

Steve Ellcey
sellcey@marvell.com

Re: RFC: Extending --with-advance-toolchain to aarch64

[Florian Weimer]

* Steve Ellcey:

> I am not sure, but my guess is that it is because I am building
> binutils (including ld) using --with-sysroot.

That could be the case.  I guess there are two conflicting use cases for
sysroots, one where you want to mangle run-time paths to confine things
to the sysroot, and one where the run-time paths are different from the
sysroot paths (so it's more like a cross-compiler, even if the target is
native).

Thanks,
Florian