* Fwd: [PATCH 0/4] Darwin: Replace environment runpath with embedded [PR88590].
[not found] <20211117210319.92514-1-iain@sandoe.co.uk>
@ 2021-11-17 21:24 ` Iain Sandoe
0 siblings, 0 replies; only message in thread
From: Iain Sandoe @ 2021-11-17 21:24 UTC (permalink / raw)
To: libstdc++
It seems git send-mail drops CC: headers when there’s a —cc= on the command line,
which is a bit sad when one wants to CC different folks for different patches in a series.
> Begin forwarded message:
>
> From: Iain Sandoe <iains.gcc@gmail.com>
> Subject: [PATCH 0/4] Darwin: Replace environment runpath with embedded [PR88590].
> Date: 17 November 2021 at 21:03:15 GMT
> To: gcc-patches@gcc.gnu.org
> Cc: joseph@codesourcery.com
> Reply-To: iain@sandoe.co.uk
>
> This is a fairly long explanation of the problems being addressed by
> the patch set. Most of the changes are Darwin-specific - a change to
> the libtool component allowing for this @rpath and some minor additions
> to makefiles where libtool is not in use. At present, this seems pretty
> specific to the GCC build; since we depend on accessing newly-built
> components during the bootstrap.
>
> There are additional details relevant to each patch in its own commit
> message.
>
> =====
>
> Darwin builds shared libraries with information on the runpath as part
> of the library name. For example, /installation/path/for/libfoo.dylib.
>
> That is regarded as two components; the 'runpath' /installation/path/for/
> and the library name libfoo.dylib.
>
> This means that (at runtime) two libraries with the same name can be
> disambiguated by their runpaths, and potentially used by the same exe.
>
> ===== Problem #1
>
> That is fine and works well; until we disturb the assumptions by
> overriding the library runpath with an environment one: DYLD_LIBRARY_PATH.
>
> Now the library runpath(s) can be discarded and the libraries are first
> searched on the basis of that provided by the environment; two libraries
> with the same name are no longer distinct (if a library with that name is
> found in the environment path).
>
> This causes problems in configuring, building and testing GCC because we
> set the runpath environment at a very high level so that it applies to
> stage1+ target configures and stage2+ host configures. This is needed so
> that executables built during those configures get the newly-built libgcc_s
> when the target defaults to using a shared libgcc.
>
> However, it also means that every tool that is used during the configure
> has its libgcc_s (or any of the newly-built bootstrapped libs) overriden
> to use the new one(s) - which might be buggy.
>
> In the testsuite it is more serious - since more target libs come into
> play - especially libstdc++. Several system tools on Darwin use(d) libc++
> and that has caused wrong or crashed test output. In principle,
> LD_LIBRARY_PATH on Linux has the same issue - although perhaps there is
> less tendency to default to use of shared dependent libs.
>
> Ideally, one would have several environment paths, and some way to use
> the appropriate one at the appropriate time. I experimented with this
> as a solution to both this and the following problem, but it proved
> unrealistic - since the process would have to be applied to all relevant
> OSS projects using auto-tools to be safe - and mostly the uninstalled
> use of libraries is a GCC build-time issue.
>
> ===== Problem #2
>
> A change in security policy for Darwin means that DYLD_LIBRARY_PATH is
> now removed from the environment for all system tools (e.g. /usr/sh, env
> etc). This means that for all realistic build steps that use any system
> utility (like sh) will no longer see the the environment runpath and the
> only ones available will be those in the libraries.
>
> This breaks GCC's configuration since the steps mentioned above are now
> not seeing the newly-built shared libraries, but actually much olders ones
> installed on the system. It means that for all Darwin15+ we misconfigure
> libstdc++.
>
> /bin/sh is hardwired into autoconf, one cannot use CONFIG_SHELL to work
> around this - because /bin/sh is invoked first, and then passes control to
> CONFIG_SHELL.
>
> A second problem is that we cannot bump the SO name for libgcc_s (which
> I need to do to solve an EH problem) - since the new SO name is not
> available on the system, and therefore none of the stage1+ target configures
> will succeed. This is because the eventual install path is correctly
> encoded into the built library, but it is not present at the install
> position (and, in general, cannot be installed - since that might not even
> be a suitable path on the build system).
>
> This has also meant that we could not do in-tree testing without first
> installing the target libraries (which is mostly inconvenient rather than
> a show-stopper, but still).
>
> ===== Tested solution.
>
> Darwin has the ability to make the runpaths install-position independent.
>
> One sets the library runpath to @rpath/ which essentially means "use the
> runpath available at the time we want to load this".
>
> One can then add (potentially multiple) runpaths to the executable, the
> built library can be put anywhere convenient - providing we can put that
> path into the exe.
>
> For GCC's build, test and install process this means that we need at each
> stage to build exes with the runpaths that are relevant to the positions
> of the dependent libraries.
>
> To do this, we add an rpath for each of the startfile paths. While we are
> building/testing GCC these correspond to (for example gcc/ or
> <target>/libstdc++/src/.libs etc) and then, after the compiler is installed
> at its intended install path - these become /compiler/installation/path/lib
> etc.
>
> I have tested this widely on i686, powerpc, x86_64 and aarch64 Darwin over
> more than a year.
>
> So patch 1 : provides a spec that expands to -rpath xxx for each xxx in the
> startfiles (which means that this works for anything added as -B and thus
> the testsuite also works without installing the target libs again).
>
> The remaining patches implement the Darwin changes to build the target
> libraries with @rpath runpaths - this is the largest part of the patch
> in bytes - because the libtool change expands.
>
> I would like to get this in early in stage 3 so that it has time to be
> kicked around before we branch.
>
> OK for master?
> (ideally, we'd also backport to open branches because the breakage is
> quite significant)
>
> thanks
> Iain
>
>
> Iain Sandoe (4):
> Driver : Provide a spec to insert rpaths for compiler lib dirs.
> Darwin : Handle rpaths given on the command line.
> Darwin : Allow for configuring Darwin to use embedded runpath.
> Darwin, Ada : Add loader path as a default rpath.
>
> configure | 5 +
> configure.ac | 5 +
> fixincludes/config.h.in | 204 ---------------------------
> fixincludes/configure | 2 +-
> gcc/aclocal.m4 | 50 +++++++
> gcc/ada/gcc-interface/Makefile.in | 2 +
> gcc/config/darwin-driver.c | 18 +++
> gcc/config/darwin.h | 11 +-
> gcc/config/darwin.opt | 4 +
> gcc/configure | 157 +++++++++++++++++++--
> gcc/gcc.c | 18 +++
> libatomic/Makefile.am | 6 +-
> libatomic/Makefile.in | 5 +-
> libatomic/configure | 82 ++++++++++-
> libatomic/testsuite/Makefile.in | 1 +
> libbacktrace/configure | 80 ++++++++++-
> libcc1/Makefile.am | 9 ++
> libcc1/Makefile.in | 12 +-
> libcc1/configure | 157 +++++++++++++++++++--
> libffi/Makefile.am | 7 +-
> libffi/Makefile.in | 6 +-
> libffi/configure | 157 +++++++++++++++++++--
> libgcc/config.host | 22 ++-
> libgcc/config/t-darwin-rpath | 5 +
> libgcc/config/t-slibgcc-darwin | 13 +-
> libgfortran/Makefile.am | 3 +
> libgfortran/Makefile.in | 30 ++--
> libgfortran/configure | 155 ++++++++++++++++++--
> libgfortran/configure.ac | 4 +-
> libgomp/Makefile.am | 6 +-
> libgomp/Makefile.in | 3 +-
> libgomp/configure | 151 ++++++++++++++++++--
> libitm/Makefile.am | 5 +-
> libitm/Makefile.in | 3 +-
> libitm/configure | 171 ++++++++++++++++++++---
> libobjc/configure | 96 +++++++++++--
> libobjc/configure.ac | 14 +-
> liboffloadmic/configure | 199 +++++++++++++++++++++-----
> liboffloadmic/plugin/Makefile.in | 2 +-
> liboffloadmic/plugin/aclocal.m4 | 2 +-
> liboffloadmic/plugin/configure | 199 +++++++++++++++++++++-----
> libphobos/configure | 151 ++++++++++++++++++--
> libphobos/libdruntime/Makefile.am | 5 +-
> libphobos/libdruntime/Makefile.in | 3 +-
> libphobos/src/Makefile.am | 5 +-
> libphobos/src/Makefile.in | 3 +-
> libquadmath/Makefile.am | 6 +-
> libquadmath/Makefile.in | 4 +-
> libquadmath/configure | 225 +++++++++++++++++++++++++++++-
> libquadmath/configure.ac | 2 +
> libsanitizer/asan/Makefile.am | 6 +-
> libsanitizer/asan/Makefile.in | 5 +-
> libsanitizer/configure | 157 +++++++++++++++++++--
> libsanitizer/hwasan/Makefile.am | 6 +-
> libsanitizer/hwasan/Makefile.in | 5 +-
> libsanitizer/lsan/Makefile.am | 7 +-
> libsanitizer/lsan/Makefile.in | 6 +-
> libsanitizer/tsan/Makefile.am | 6 +-
> libsanitizer/tsan/Makefile.in | 5 +-
> libsanitizer/ubsan/Makefile.am | 6 +-
> libsanitizer/ubsan/Makefile.in | 5 +-
> libssp/Makefile.am | 5 +-
> libssp/Makefile.in | 3 +-
> libssp/configure | 82 ++++++++++-
> libstdc++-v3/configure | 169 +++++++++++++++++++---
> libstdc++-v3/src/Makefile.am | 6 +-
> libstdc++-v3/src/Makefile.in | 3 +-
> libtool.m4 | 64 ++++++++-
> libvtv/configure | 171 ++++++++++++++++++++---
> lto-plugin/Makefile.in | 1 -
> lto-plugin/configure | 80 ++++++++++-
> zlib/configure | 82 ++++++++++-
> 72 files changed, 2832 insertions(+), 533 deletions(-)
> create mode 100644 libgcc/config/t-darwin-rpath
>
> --
> 2.24.3 (Apple Git-128)
>
^ permalink raw reply [flat|nested] only message in thread