From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by sourceware.org (Postfix) with ESMTPS id A82E13858402 for ; Wed, 17 Nov 2021 21:24:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A82E13858402 Received: by mail-wm1-x32b.google.com with SMTP id p3-20020a05600c1d8300b003334fab53afso5924938wms.3 for ; Wed, 17 Nov 2021 13:24:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:mime-version:subject:message-id:references :to:date; bh=+uafbR8EDN7TfB9dP9OTlQAHlMllTGFOPtpqYIpB9kU=; b=TvG/w9KTi2TUDsLqoY3uFH02MIMf6M4DhpKYnEvAC/15k1rGxP+JOv1ttYkdh/Dqcd STuWJ99d+EOag/VG6+ufnGlvFxbqG3kth10gRUsndwyrvPWj9ZKQYgKsxF7k581+G5h7 QbKgblCKaF0LfHVEvRMohr9Q9H8B5d0JhEBwvRIaWKM1BGte/U37Vqa2AXF7SYBtCOAF QGpfUMnom1lYNg3d4nc/jRFKzYOmYzYTGpdsocEJ+iqCPLrgmhanC/NC1MF4ia/YZeNa 7Qmd4B/Vs0+bCqbQjy8OFEWcV5tvv0Fj31yckkSxgpcGdmOWw8ig0Xq946qiy93GIshE uHOA== X-Gm-Message-State: AOAM531cj14fgvBb9aGm/66TeM39x077Gp8GNC7bRwul2napqh2qXNr6 HyKi8x4qa/6FtP7AQFVQWJVCD/oTj+0= X-Google-Smtp-Source: ABdhPJzkNkX82BvCdyaJMdnpyR4Sus4a3gm+tihckL3SA5+3GBsXL+UhBHYXCzeX1+VFON1Zd6hzew== X-Received: by 2002:a7b:cbc3:: with SMTP id n3mr3530456wmi.90.1637184271696; Wed, 17 Nov 2021 13:24:31 -0800 (PST) Received: from [192.168.1.214] (host81-138-1-83.in-addr.btopenworld.com. [81.138.1.83]) by smtp.gmail.com with ESMTPSA id r8sm1397552wrz.43.2021.11.17.13.24.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Nov 2021 13:24:31 -0800 (PST) From: Iain Sandoe Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.21\)) Subject: Fwd: [PATCH 0/4] Darwin: Replace environment runpath with embedded [PR88590]. Message-Id: References: <20211117210319.92514-1-iain@sandoe.co.uk> To: libstdc++ Date: Wed, 17 Nov 2021 21:24:30 +0000 X-Mailer: Apple Mail (2.3445.104.21) X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: libstdc++@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libstdc++ mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Nov 2021 21:24:36 -0000 It seems git send-mail drops CC: headers when there=E2=80=99s a =E2=80=94c= c=3D on the command line, which is a bit sad when one wants to CC different folks for different = patches in a series. > Begin forwarded message: >=20 > From: Iain Sandoe > Subject: [PATCH 0/4] Darwin: Replace environment runpath with embedded = [PR88590]. > Date: 17 November 2021 at 21:03:15 GMT > To: gcc-patches@gcc.gnu.org > Cc: joseph@codesourcery.com > Reply-To: iain@sandoe.co.uk >=20 > This is a fairly long explanation of the problems being addressed by > the patch set. Most of the changes are Darwin-specific - a change to > the libtool component allowing for this @rpath and some minor = additions > to makefiles where libtool is not in use. At present, this seems = pretty > specific to the GCC build; since we depend on accessing newly-built > components during the bootstrap. >=20 > There are additional details relevant to each patch in its own commit > message. >=20 > =3D=3D=3D=3D=3D >=20 > Darwin builds shared libraries with information on the runpath as part > of the library name. For example, = /installation/path/for/libfoo.dylib. >=20 > That is regarded as two components; the 'runpath' = /installation/path/for/ > and the library name libfoo.dylib. >=20 > This means that (at runtime) two libraries with the same name can be > disambiguated by their runpaths, and potentially used by the same exe. >=20 > =3D=3D=3D=3D=3D Problem #1 >=20 > That is fine and works well; until we disturb the assumptions by > overriding the library runpath with an environment one: = DYLD_LIBRARY_PATH. >=20 > Now the library runpath(s) can be discarded and the libraries are = first > searched on the basis of that provided by the environment; two = libraries > with the same name are no longer distinct (if a library with that name = is > found in the environment path). >=20 > This causes problems in configuring, building and testing GCC because = we > set the runpath environment at a very high level so that it applies to > stage1+ target configures and stage2+ host configures. This is needed = so > that executables built during those configures get the newly-built = libgcc_s > when the target defaults to using a shared libgcc. >=20 > However, it also means that every tool that is used during the = configure > has its libgcc_s (or any of the newly-built bootstrapped libs) = overriden > to use the new one(s) - which might be buggy. >=20 > In the testsuite it is more serious - since more target libs come into > play - especially libstdc++. Several system tools on Darwin use(d) = libc++ > and that has caused wrong or crashed test output. In principle, > LD_LIBRARY_PATH on Linux has the same issue - although perhaps there = is > less tendency to default to use of shared dependent libs. >=20 > Ideally, one would have several environment paths, and some way to use > the appropriate one at the appropriate time. I experimented with this > as a solution to both this and the following problem, but it proved > unrealistic - since the process would have to be applied to all = relevant > OSS projects using auto-tools to be safe - and mostly the uninstalled > use of libraries is a GCC build-time issue. >=20 > =3D=3D=3D=3D=3D Problem #2 >=20 > A change in security policy for Darwin means that DYLD_LIBRARY_PATH is > now removed from the environment for all system tools (e.g. /usr/sh, = env > etc). This means that for all realistic build steps that use any = system > utility (like sh) will no longer see the the environment runpath and = the > only ones available will be those in the libraries. >=20 > This breaks GCC's configuration since the steps mentioned above are = now > not seeing the newly-built shared libraries, but actually much olders = ones > installed on the system. It means that for all Darwin15+ we = misconfigure > libstdc++. >=20 > /bin/sh is hardwired into autoconf, one cannot use CONFIG_SHELL to = work > around this - because /bin/sh is invoked first, and then passes = control to > CONFIG_SHELL. >=20 > A second problem is that we cannot bump the SO name for libgcc_s = (which > I need to do to solve an EH problem) - since the new SO name is not > available on the system, and therefore none of the stage1+ target = configures > will succeed. This is because the eventual install path is correctly > encoded into the built library, but it is not present at the install > position (and, in general, cannot be installed - since that might not = even > be a suitable path on the build system). >=20 > This has also meant that we could not do in-tree testing without first > installing the target libraries (which is mostly inconvenient rather = than > a show-stopper, but still). >=20 > =3D=3D=3D=3D=3D Tested solution. >=20 > Darwin has the ability to make the runpaths install-position = independent. >=20 > One sets the library runpath to @rpath/ which essentially means "use = the > runpath available at the time we want to load this". >=20 > One can then add (potentially multiple) runpaths to the executable, = the > built library can be put anywhere convenient - providing we can put = that > path into the exe. >=20 > For GCC's build, test and install process this means that we need at = each > stage to build exes with the runpaths that are relevant to the = positions > of the dependent libraries. >=20 > To do this, we add an rpath for each of the startfile paths. While we = are > building/testing GCC these correspond to (for example gcc/ or=20 > /libstdc++/src/.libs etc) and then, after the compiler is = installed > at its intended install path - these become = /compiler/installation/path/lib > etc. >=20 > I have tested this widely on i686, powerpc, x86_64 and aarch64 Darwin = over > more than a year. >=20 > So patch 1 : provides a spec that expands to -rpath xxx for each xxx = in the > startfiles (which means that this works for anything added as -B and = thus > the testsuite also works without installing the target libs again). >=20 > The remaining patches implement the Darwin changes to build the target > libraries with @rpath runpaths - this is the largest part of the patch > in bytes - because the libtool change expands. >=20 > I would like to get this in early in stage 3 so that it has time to be > kicked around before we branch. >=20 > OK for master? > (ideally, we'd also backport to open branches because the breakage is > quite significant) >=20 > thanks > Iain >=20 >=20 > Iain Sandoe (4): > Driver : Provide a spec to insert rpaths for compiler lib dirs. > Darwin : Handle rpaths given on the command line. > Darwin : Allow for configuring Darwin to use embedded runpath. > Darwin, Ada : Add loader path as a default rpath. >=20 > configure | 5 + > configure.ac | 5 + > fixincludes/config.h.in | 204 --------------------------- > fixincludes/configure | 2 +- > gcc/aclocal.m4 | 50 +++++++ > gcc/ada/gcc-interface/Makefile.in | 2 + > gcc/config/darwin-driver.c | 18 +++ > gcc/config/darwin.h | 11 +- > gcc/config/darwin.opt | 4 + > gcc/configure | 157 +++++++++++++++++++-- > gcc/gcc.c | 18 +++ > libatomic/Makefile.am | 6 +- > libatomic/Makefile.in | 5 +- > libatomic/configure | 82 ++++++++++- > libatomic/testsuite/Makefile.in | 1 + > libbacktrace/configure | 80 ++++++++++- > libcc1/Makefile.am | 9 ++ > libcc1/Makefile.in | 12 +- > libcc1/configure | 157 +++++++++++++++++++-- > libffi/Makefile.am | 7 +- > libffi/Makefile.in | 6 +- > libffi/configure | 157 +++++++++++++++++++-- > libgcc/config.host | 22 ++- > libgcc/config/t-darwin-rpath | 5 + > libgcc/config/t-slibgcc-darwin | 13 +- > libgfortran/Makefile.am | 3 + > libgfortran/Makefile.in | 30 ++-- > libgfortran/configure | 155 ++++++++++++++++++-- > libgfortran/configure.ac | 4 +- > libgomp/Makefile.am | 6 +- > libgomp/Makefile.in | 3 +- > libgomp/configure | 151 ++++++++++++++++++-- > libitm/Makefile.am | 5 +- > libitm/Makefile.in | 3 +- > libitm/configure | 171 ++++++++++++++++++++--- > libobjc/configure | 96 +++++++++++-- > libobjc/configure.ac | 14 +- > liboffloadmic/configure | 199 +++++++++++++++++++++----- > liboffloadmic/plugin/Makefile.in | 2 +- > liboffloadmic/plugin/aclocal.m4 | 2 +- > liboffloadmic/plugin/configure | 199 +++++++++++++++++++++----- > libphobos/configure | 151 ++++++++++++++++++-- > libphobos/libdruntime/Makefile.am | 5 +- > libphobos/libdruntime/Makefile.in | 3 +- > libphobos/src/Makefile.am | 5 +- > libphobos/src/Makefile.in | 3 +- > libquadmath/Makefile.am | 6 +- > libquadmath/Makefile.in | 4 +- > libquadmath/configure | 225 +++++++++++++++++++++++++++++- > libquadmath/configure.ac | 2 + > libsanitizer/asan/Makefile.am | 6 +- > libsanitizer/asan/Makefile.in | 5 +- > libsanitizer/configure | 157 +++++++++++++++++++-- > libsanitizer/hwasan/Makefile.am | 6 +- > libsanitizer/hwasan/Makefile.in | 5 +- > libsanitizer/lsan/Makefile.am | 7 +- > libsanitizer/lsan/Makefile.in | 6 +- > libsanitizer/tsan/Makefile.am | 6 +- > libsanitizer/tsan/Makefile.in | 5 +- > libsanitizer/ubsan/Makefile.am | 6 +- > libsanitizer/ubsan/Makefile.in | 5 +- > libssp/Makefile.am | 5 +- > libssp/Makefile.in | 3 +- > libssp/configure | 82 ++++++++++- > libstdc++-v3/configure | 169 +++++++++++++++++++--- > libstdc++-v3/src/Makefile.am | 6 +- > libstdc++-v3/src/Makefile.in | 3 +- > libtool.m4 | 64 ++++++++- > libvtv/configure | 171 ++++++++++++++++++++--- > lto-plugin/Makefile.in | 1 - > lto-plugin/configure | 80 ++++++++++- > zlib/configure | 82 ++++++++++- > 72 files changed, 2832 insertions(+), 533 deletions(-) > create mode 100644 libgcc/config/t-darwin-rpath >=20 > --=20 > 2.24.3 (Apple Git-128) >=20