* [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] @ 2024-02-07 1:54 Jason Merrill 2024-02-07 20:36 ` Ian Lance Taylor 2024-02-08 5:36 ` Alexandre Oliva 0 siblings, 2 replies; 31+ messages in thread From: Jason Merrill @ 2024-02-07 1:54 UTC (permalink / raw) To: gcc-patches; +Cc: Alexandre Oliva, Paolo Bonzini, Ian Lance Taylor Tested x86_64-pc-linux-gnu. Any thoughts? -- 8< -- The patch for PR22340 (r104978) moved the adding of TARGET_LIB_PATH to RPATH_ENVVAR from POSTSTAGE1_HOST_EXPORTS to HOST_EXPORTS, but didn't mention that in the ChangeLog; it also wasn't part of the patch that was sent to gcc-patches. I suspect it was included accidentally? It also causes PR105688 when rebuilding stage1: once the stage1 libstdc++ has been built, if calling the system gcc to build host code involves invoking any tool that links against libstdc++.so (gold, ccache) they get the just-built library instead of the system library they expect. Reverting that hunk of the change fixed my problem with bubblestrapping GCC 12 with ccache on a host with a newer system libstdc++. But I believe that adding TARGET_LIB_PATH to RPATH_ENVVAR is not needed for post-stage1 either, at this point. Including TARGET_LIB_PATH goes back to r37545, with the stated rationale of getting other C++ library configury to succeed, but it looks to me like that is no longer necessary. So I propose to stop adding target libraries to LD_LIBRARY_PATH; see https://gcc.gnu.org/legacy-ml/gcc/2012-06/msg00325.html for a previous proposal by Ian to make this change. I have tried and failed to test this on a system without system libstdc++; bootstrap on cfarm220 and cfarm240 failed for unrelated reasons. PR bootstrap/105688 ChangeLog: * Makefile.tpl (HOST_EXPORTS): Don't add TARGET_LIB_PATH to RPATH_ENVVAR. * Makefile.in: Regenerate. --- Makefile.in | 3 --- Makefile.tpl | 3 --- 2 files changed, 6 deletions(-) diff --git a/Makefile.in b/Makefile.in index edb0c8a9a42..c2843d5df89 100644 --- a/Makefile.in +++ b/Makefile.in @@ -242,9 +242,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ diff --git a/Makefile.tpl b/Makefile.tpl index adbcbdd1d57..cb39fbd0434 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -245,9 +245,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ base-commit: c5d34912ad576be1ef19be92f7eabde54b9089eb -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-07 1:54 [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] Jason Merrill @ 2024-02-07 20:36 ` Ian Lance Taylor 2024-02-08 5:36 ` Alexandre Oliva 1 sibling, 0 replies; 31+ messages in thread From: Ian Lance Taylor @ 2024-02-07 20:36 UTC (permalink / raw) To: Jason Merrill; +Cc: gcc-patches, Alexandre Oliva, Paolo Bonzini On Tue, Feb 6, 2024 at 6:08 PM Jason Merrill <jason@redhat.com> wrote: > > Tested x86_64-pc-linux-gnu. Any thoughts? It still makes sense to me, for what that's worth. Ian > -- 8< -- > > The patch for PR22340 (r104978) moved the adding of TARGET_LIB_PATH to > RPATH_ENVVAR from POSTSTAGE1_HOST_EXPORTS to HOST_EXPORTS, but didn't > mention that in the ChangeLog; it also wasn't part of the patch that was > sent to gcc-patches. I suspect it was included accidentally? > > It also causes PR105688 when rebuilding stage1: once the stage1 libstdc++ > has been built, if calling the system gcc to build host code involves > invoking any tool that links against libstdc++.so (gold, ccache) they get > the just-built library instead of the system library they expect. > > Reverting that hunk of the change fixed my problem with bubblestrapping GCC > 12 with ccache on a host with a newer system libstdc++. > > But I believe that adding TARGET_LIB_PATH to RPATH_ENVVAR is not needed for > post-stage1 either, at this point. Including TARGET_LIB_PATH goes back to > r37545, with the stated rationale of getting other C++ library configury to > succeed, but it looks to me like that is no longer necessary. > > So I propose to stop adding target libraries to LD_LIBRARY_PATH; see > https://gcc.gnu.org/legacy-ml/gcc/2012-06/msg00325.html for a previous > proposal by Ian to make this change. > > I have tried and failed to test this on a system without system libstdc++; > bootstrap on cfarm220 and cfarm240 failed for unrelated reasons. > > PR bootstrap/105688 > > ChangeLog: > > * Makefile.tpl (HOST_EXPORTS): Don't add TARGET_LIB_PATH to > RPATH_ENVVAR. > * Makefile.in: Regenerate. > --- > Makefile.in | 3 --- > Makefile.tpl | 3 --- > 2 files changed, 6 deletions(-) > > diff --git a/Makefile.in b/Makefile.in > index edb0c8a9a42..c2843d5df89 100644 > --- a/Makefile.in > +++ b/Makefile.in > @@ -242,9 +242,6 @@ HOST_EXPORTS = \ > ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ > ISLINC="$(HOST_ISLINC)"; export ISLINC; \ > XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ > -@if gcc-bootstrap > - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ > -@endif gcc-bootstrap > $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); > > POSTSTAGE1_CXX_EXPORT = \ > diff --git a/Makefile.tpl b/Makefile.tpl > index adbcbdd1d57..cb39fbd0434 100644 > --- a/Makefile.tpl > +++ b/Makefile.tpl > @@ -245,9 +245,6 @@ HOST_EXPORTS = \ > ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ > ISLINC="$(HOST_ISLINC)"; export ISLINC; \ > XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ > -@if gcc-bootstrap > - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ > -@endif gcc-bootstrap > $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); > > POSTSTAGE1_CXX_EXPORT = \ > > base-commit: c5d34912ad576be1ef19be92f7eabde54b9089eb > -- > 2.43.0 > ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-07 1:54 [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] Jason Merrill 2024-02-07 20:36 ` Ian Lance Taylor @ 2024-02-08 5:36 ` Alexandre Oliva 2024-02-08 15:04 ` Iain Sandoe 1 sibling, 1 reply; 31+ messages in thread From: Alexandre Oliva @ 2024-02-08 5:36 UTC (permalink / raw) To: Jason Merrill; +Cc: gcc-patches, Paolo Bonzini, Ian Lance Taylor On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: > Reverting that hunk of the change fixed my problem with bubblestrapping GCC > 12 with ccache on a host with a newer system libstdc++. Did you have libcc1, gnattools and gotools enabled in your testing? Those non-bootstrapped tools are most likely to be problem spots. I have a vague recollection that, in native scenarios, when bootstrapping but not when not boostrapping, they are (or used to be?) built using the just-built (last-stage) compiler and target libraries, rather than system tools or previous stage (which I believe would fail bootstrap-lean). Just by looking at the current code I can't bring back to mind all the involved moving parts :-/ but my guess is that the need for TARGET_LIB_PATH comes about for this combination of circumstances, in which we would need to use the just-built native-target libstdc++ to link or to run these host tools using the just-built tools. Having a newer system libstdc++ would, I suspect, unfortunately mask the problem that those who had an older or no system libstdc++ would likely run into. OTOH, if the system linker requires the newer system library, and ld.so and ld overload LD_LIBRARY_PATH for both the libraries loaded for ld to run, the libraries ld searches to link executables, and the libraries loaded for so-linked programs to run, we seem to need some means to distinguish between these three circumstances so as to be able to set LD_LIBRARY_PATH correctly for each one, whether the libstdc++ we're building is newer or older than the system one. And there's an added complication that any single ld invocation involves two different and potentially conflicting uses of LD_LIBRARY_PATH. /me runs screaming :-) Perhaps the best we can do out of these conflicting requirements is to somehow detect a system libstdc++ newer than the one we're about to build, and fail early if we find that some of the tools depend on the newer libstdc++, or disable this env var setting for this case only and hope the newer system library is compatible enough. Holy nightmare, Batman! :-) :-( Perhaps the way to go is an explicit configure setting, recommended by the fail early, to disable the env var setting? -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer More tolerance and less prejudice are key for inclusion and diversity Excluding neuro-others for not behaving ""normal"" is *not* inclusive ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 5:36 ` Alexandre Oliva @ 2024-02-08 15:04 ` Iain Sandoe 2024-02-08 17:12 ` Jason Merrill 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-08 15:04 UTC (permalink / raw) To: Jason Merrill Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva Hi Jason, I have tested this on modern Darwin (with libc++ as the system library) and on older Darwin, where we do see the issue - because the system linker is written in C++ and links with libstdc++ (so sometimes we get a crash, or worse unpredictable beahviour). ----- For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, but for older Darwin…. > On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: > > On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: > >> Reverting that hunk of the change fixed my problem with bubblestrapping GCC >> 12 with ccache on a host with a newer system libstdc++. > > Did you have libcc1, gnattools and gotools enabled in your testing? … I have done all but “go” since that’s not supported on Darwin. The patch breaks bootstrap on older Darwin because: Ada uses exceptions. gnat1 pulls in system libraries that link with the system unwinder - so we have to link gnat1 “-shared-libgcc” - which means we need to be able to find the just-built one when building the target libs. - that means we need to find it as a host library, which we were getting away with because the host list was appended to the target list. To recover bootstrap I think the following addition is needed: diff --git a/Makefile.tpl b/Makefile.tpl index cb39fbd0434..65621bf6882 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -663,7 +663,7 @@ TARGET_LIB_PATH_[+module+] = $$r/$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: # so that programs built for the host machine work. HOST_LIB_PATH = [+ FOR host_modules +][+ IF lib_path +]$(HOST_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ - ENDFOR host_modules +] + ENDFOR host_modules +]$(HOST_LIB_PATH_gcc) # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc It’s not clear to me why we consider prev-gcc as relevant to host libgcc (we must have built libgcc in order to have linked it to any host deps .. but perhaps I’m missing a case). I agree with the sentiment below that this is a delicate area…. > Those non-bootstrapped tools are most likely to be problem spots. > > I have a vague recollection that, in native scenarios, when > bootstrapping but not when not boostrapping, they are (or used to be?) > built using the just-built (last-stage) compiler and target libraries, > rather than system tools or previous stage (which I believe would fail > bootstrap-lean). > > Just by looking at the current code I can't bring back to mind all the > involved moving parts :-/ but my guess is that the need for > TARGET_LIB_PATH comes about for this combination of circumstances, in > which we would need to use the just-built native-target libstdc++ to > link or to run these host tools using the just-built tools. > > Having a newer system libstdc++ would, I suspect, unfortunately mask the > problem that those who had an older or no system libstdc++ would likely > run into. > > > OTOH, if the system linker requires the newer system library, and ld.so > and ld overload LD_LIBRARY_PATH for both the libraries loaded for ld to > run, the libraries ld searches to link executables, and the libraries > loaded for so-linked programs to run, we seem to need some means to > distinguish between these three circumstances so as to be able to set > LD_LIBRARY_PATH correctly for each one, whether the libstdc++ we're > building is newer or older than the system one. And there's an added > complication that any single ld invocation involves two different and > potentially conflicting uses of LD_LIBRARY_PATH. > > /me runs screaming :-) > > > Perhaps the best we can do out of these conflicting requirements is to > somehow detect a system libstdc++ newer than the one we're about to > build, and fail early if we find that some of the tools depend on the > newer libstdc++, or disable this env var setting for this case only and > hope the newer system library is compatible enough. > > Holy nightmare, Batman! :-) :-( > > > Perhaps the way to go is an explicit configure setting, recommended by > the fail early, to disable the env var setting? > > -- > Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ > Free Software Activist GNU Toolchain Engineer > More tolerance and less prejudice are key for inclusion and diversity > Excluding neuro-others for not behaving ""normal"" is *not* inclusive ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 15:04 ` Iain Sandoe @ 2024-02-08 17:12 ` Jason Merrill 2024-02-08 17:16 ` Jason Merrill 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-08 17:12 UTC (permalink / raw) To: Iain Sandoe; +Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva [-- Attachment #1: Type: text/plain, Size: 1706 bytes --] On 2/8/24 10:04, Iain Sandoe wrote: > Hi Jason, > > I have tested this on modern Darwin (with libc++ as the system library) and on > older Darwin, where we do see the issue - because the system linker is written > in C++ and links with libstdc++ (so sometimes we get a crash, or worse unpredictable > beahviour). Thank you! > ----- > > For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, but for older Darwin…. > >> On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: >> >> On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: >> >>> Reverting that hunk of the change fixed my problem with bubblestrapping GCC >>> 12 with ccache on a host with a newer system libstdc++. >> >> Did you have libcc1, gnattools and gotools enabled in your testing? > > … I have done all but “go” since that’s not supported on Darwin. > > The patch breaks bootstrap on older Darwin because: > > Ada uses exceptions. > gnat1 pulls in system libraries that link with the system unwinder > - so we have to link gnat1 “-shared-libgcc” > - which means we need to be able to find the just-built one when building the target libs. Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. In neither case do we want it to find the libgcc from the current stage. So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. Does this make sense? Does it work on Darwin? [-- Attachment #2: 0001-build-add-prev-target-libs-to-rpath.patch --] [-- Type: text/x-patch, Size: 5627 bytes --] From 1115afdce16742003030599aab327daaf9db5073 Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Thu, 8 Feb 2024 11:58:11 -0500 Subject: [PATCH] build: add prev target libs to rpath To: gcc-patches@gcc.gnu.org ChangeLog: * Makefile.tpl (POSTSTAGE1_HOST_EXPORTS): Add TARGET_PREV_LIB_PATH to RPATH_ENVVAR. * Makefile.in: Regenerate. --- Makefile.in | 14 +++++++++++++- Makefile.tpl | 10 +++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/Makefile.in b/Makefile.in index c2843d5df89..31737cba37f 100644 --- a/Makefile.in +++ b/Makefile.in @@ -266,6 +266,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(TARGET_PREV_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -727,41 +728,51 @@ all: # This is the list of directories that may be needed in RPATH_ENVVAR # so that programs built for the target machine work. TARGET_LIB_PATH = $(TARGET_LIB_PATH_libstdc++-v3)$(TARGET_LIB_PATH_libsanitizer)$(TARGET_LIB_PATH_libvtv)$(TARGET_LIB_PATH_libssp)$(TARGET_LIB_PATH_libphobos)$(TARGET_LIB_PATH_libgm2)$(TARGET_LIB_PATH_libgomp)$(TARGET_LIB_PATH_libitm)$(TARGET_LIB_PATH_libatomic)$(HOST_LIB_PATH_gcc) +TARGET_PREV_LIB_PATH = $(TARGET_PREV_LIB_PATH_libstdc++-v3)$(TARGET_PREV_LIB_PATH_libsanitizer)$(TARGET_PREV_LIB_PATH_libvtv)$(TARGET_PREV_LIB_PATH_libssp)$(TARGET_PREV_LIB_PATH_libphobos)$(TARGET_PREV_LIB_PATH_libgm2)$(TARGET_PREV_LIB_PATH_libgomp)$(TARGET_PREV_LIB_PATH_libitm)$(TARGET_PREV_LIB_PATH_libatomic)$(HOST_PREV_LIB_PATH_gcc) @if target-libstdc++-v3 TARGET_LIB_PATH_libstdc++-v3 = $$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs: +TARGET_PREV_LIB_PATH_libstdc++-v3 = $$r/prev-$(TARGET_SUBDIR)/libstdc++-v3/src/.libs: @endif target-libstdc++-v3 @if target-libsanitizer TARGET_LIB_PATH_libsanitizer = $$r/$(TARGET_SUBDIR)/libsanitizer/.libs: +TARGET_PREV_LIB_PATH_libsanitizer = $$r/prev-$(TARGET_SUBDIR)/libsanitizer/.libs: @endif target-libsanitizer @if target-libvtv TARGET_LIB_PATH_libvtv = $$r/$(TARGET_SUBDIR)/libvtv/.libs: +TARGET_PREV_LIB_PATH_libvtv = $$r/prev-$(TARGET_SUBDIR)/libvtv/.libs: @endif target-libvtv @if target-libssp TARGET_LIB_PATH_libssp = $$r/$(TARGET_SUBDIR)/libssp/.libs: +TARGET_PREV_LIB_PATH_libssp = $$r/prev-$(TARGET_SUBDIR)/libssp/.libs: @endif target-libssp @if target-libphobos TARGET_LIB_PATH_libphobos = $$r/$(TARGET_SUBDIR)/libphobos/src/.libs: +TARGET_PREV_LIB_PATH_libphobos = $$r/prev-$(TARGET_SUBDIR)/libphobos/src/.libs: @endif target-libphobos @if target-libgm2 TARGET_LIB_PATH_libgm2 = $$r/$(TARGET_SUBDIR)/libgm2/.libs: +TARGET_PREV_LIB_PATH_libgm2 = $$r/prev-$(TARGET_SUBDIR)/libgm2/.libs: @endif target-libgm2 @if target-libgomp TARGET_LIB_PATH_libgomp = $$r/$(TARGET_SUBDIR)/libgomp/.libs: +TARGET_PREV_LIB_PATH_libgomp = $$r/prev-$(TARGET_SUBDIR)/libgomp/.libs: @endif target-libgomp @if target-libitm TARGET_LIB_PATH_libitm = $$r/$(TARGET_SUBDIR)/libitm/.libs: +TARGET_PREV_LIB_PATH_libitm = $$r/prev-$(TARGET_SUBDIR)/libitm/.libs: @endif target-libitm @if target-libatomic TARGET_LIB_PATH_libatomic = $$r/$(TARGET_SUBDIR)/libatomic/.libs: +TARGET_PREV_LIB_PATH_libatomic = $$r/prev-$(TARGET_SUBDIR)/libatomic/.libs: @endif target-libatomic @@ -772,7 +783,8 @@ HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(H # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR) + @endif gcc diff --git a/Makefile.tpl b/Makefile.tpl index cb39fbd0434..cbb07c93a34 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -269,6 +269,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(TARGET_PREV_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -652,9 +653,13 @@ all: TARGET_LIB_PATH = [+ FOR target_modules +][+ IF lib_path +]$(TARGET_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ ENDFOR target_modules +]$(HOST_LIB_PATH_gcc) +TARGET_PREV_LIB_PATH = [+ FOR target_modules +][+ + IF lib_path +]$(TARGET_PREV_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ + ENDFOR target_modules +]$(HOST_PREV_LIB_PATH_gcc) [+ FOR target_modules +][+ IF lib_path +] @if target-[+module+] TARGET_LIB_PATH_[+module+] = $$r/$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: +TARGET_PREV_LIB_PATH_[+module+] = $$r/prev-$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: @endif target-[+module+] [+ ENDIF lib_path +][+ ENDFOR target_modules +] @@ -667,7 +672,10 @@ HOST_LIB_PATH = [+ FOR host_modules +][+ # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR) +[+ IF bootstrap +] +HOST_PREV_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +[+ ENDIF bootstrap +] @endif gcc [+ FOR host_modules +][+ IF lib_path +] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 17:12 ` Jason Merrill @ 2024-02-08 17:16 ` Jason Merrill 2024-02-08 17:51 ` Iain Sandoe 2024-02-08 17:55 ` Paolo Bonzini 0 siblings, 2 replies; 31+ messages in thread From: Jason Merrill @ 2024-02-08 17:16 UTC (permalink / raw) To: Iain Sandoe; +Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva [-- Attachment #1: Type: text/plain, Size: 1899 bytes --] On 2/8/24 12:12, Jason Merrill wrote: > On 2/8/24 10:04, Iain Sandoe wrote: >> Hi Jason, >> >> I have tested this on modern Darwin (with libc++ as the system >> library) and on >> older Darwin, where we do see the issue - because the system linker is >> written >> in C++ and links with libstdc++ (so sometimes we get a crash, or worse >> unpredictable >> beahviour). > > Thank you! > >> ----- >> >> For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, >> but for older Darwin…. >> >>> On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: >>> >>> On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: >>> >>>> Reverting that hunk of the change fixed my problem with >>>> bubblestrapping GCC >>>> 12 with ccache on a host with a newer system libstdc++. >>> >>> Did you have libcc1, gnattools and gotools enabled in your testing? >> >> … I have done all but “go” since that’s not supported on Darwin. >> >> The patch breaks bootstrap on older Darwin because: >> >> Ada uses exceptions. >> gnat1 pulls in system libraries that link with the system unwinder >> - so we have to link gnat1 “-shared-libgcc” >> - which means we need to be able to find the just-built one when >> building the target libs. > > Hmm. In stage 1, when we build with the system gcc, I'd think we want > the just-built gnat1 to find the system libgcc. > > In stage 2, when we build with the stage 1 gcc, we want the just-built > gnat1 to find the stage 1 libgcc. > > In neither case do we want it to find the libgcc from the current stage. > > So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to > include the TARGET_LIB_PATH from the previous stage. Something like the > below, on top of the earlier patch. > > Does this make sense? Does it work on Darwin? Oops, that was broken, please consider this one instead: [-- Attachment #2: 0001-build-add-prev-target-libs-to-rpath.patch --] [-- Type: text/x-patch, Size: 5651 bytes --] From 4c9558e7f82f95fb7583892c18401bc2ba98ec4d Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Thu, 8 Feb 2024 11:58:11 -0500 Subject: [PATCH] build: add prev target libs to rpath To: gcc-patches@gcc.gnu.org ChangeLog: * Makefile.tpl (POSTSTAGE1_HOST_EXPORTS): Add TARGET_PREV_LIB_PATH to RPATH_ENVVAR. * Makefile.in: Regenerate. --- Makefile.in | 14 +++++++++++++- Makefile.tpl | 8 +++++++- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/Makefile.in b/Makefile.in index c2843d5df89..4b168cb2087 100644 --- a/Makefile.in +++ b/Makefile.in @@ -266,6 +266,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(TARGET_PREV_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -727,41 +728,51 @@ all: # This is the list of directories that may be needed in RPATH_ENVVAR # so that programs built for the target machine work. TARGET_LIB_PATH = $(TARGET_LIB_PATH_libstdc++-v3)$(TARGET_LIB_PATH_libsanitizer)$(TARGET_LIB_PATH_libvtv)$(TARGET_LIB_PATH_libssp)$(TARGET_LIB_PATH_libphobos)$(TARGET_LIB_PATH_libgm2)$(TARGET_LIB_PATH_libgomp)$(TARGET_LIB_PATH_libitm)$(TARGET_LIB_PATH_libatomic)$(HOST_LIB_PATH_gcc) +TARGET_PREV_LIB_PATH = $(TARGET_PREV_LIB_PATH_libstdc++-v3)$(TARGET_PREV_LIB_PATH_libsanitizer)$(TARGET_PREV_LIB_PATH_libvtv)$(TARGET_PREV_LIB_PATH_libssp)$(TARGET_PREV_LIB_PATH_libphobos)$(TARGET_PREV_LIB_PATH_libgm2)$(TARGET_PREV_LIB_PATH_libgomp)$(TARGET_PREV_LIB_PATH_libitm)$(TARGET_PREV_LIB_PATH_libatomic)$(HOST_PREV_LIB_PATH_gcc) @if target-libstdc++-v3 TARGET_LIB_PATH_libstdc++-v3 = $$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs: +TARGET_PREV_LIB_PATH_libstdc++-v3 = $$r/prev-$(TARGET_SUBDIR)/libstdc++-v3/src/.libs: @endif target-libstdc++-v3 @if target-libsanitizer TARGET_LIB_PATH_libsanitizer = $$r/$(TARGET_SUBDIR)/libsanitizer/.libs: +TARGET_PREV_LIB_PATH_libsanitizer = $$r/prev-$(TARGET_SUBDIR)/libsanitizer/.libs: @endif target-libsanitizer @if target-libvtv TARGET_LIB_PATH_libvtv = $$r/$(TARGET_SUBDIR)/libvtv/.libs: +TARGET_PREV_LIB_PATH_libvtv = $$r/prev-$(TARGET_SUBDIR)/libvtv/.libs: @endif target-libvtv @if target-libssp TARGET_LIB_PATH_libssp = $$r/$(TARGET_SUBDIR)/libssp/.libs: +TARGET_PREV_LIB_PATH_libssp = $$r/prev-$(TARGET_SUBDIR)/libssp/.libs: @endif target-libssp @if target-libphobos TARGET_LIB_PATH_libphobos = $$r/$(TARGET_SUBDIR)/libphobos/src/.libs: +TARGET_PREV_LIB_PATH_libphobos = $$r/prev-$(TARGET_SUBDIR)/libphobos/src/.libs: @endif target-libphobos @if target-libgm2 TARGET_LIB_PATH_libgm2 = $$r/$(TARGET_SUBDIR)/libgm2/.libs: +TARGET_PREV_LIB_PATH_libgm2 = $$r/prev-$(TARGET_SUBDIR)/libgm2/.libs: @endif target-libgm2 @if target-libgomp TARGET_LIB_PATH_libgomp = $$r/$(TARGET_SUBDIR)/libgomp/.libs: +TARGET_PREV_LIB_PATH_libgomp = $$r/prev-$(TARGET_SUBDIR)/libgomp/.libs: @endif target-libgomp @if target-libitm TARGET_LIB_PATH_libitm = $$r/$(TARGET_SUBDIR)/libitm/.libs: +TARGET_PREV_LIB_PATH_libitm = $$r/prev-$(TARGET_SUBDIR)/libitm/.libs: @endif target-libitm @if target-libatomic TARGET_LIB_PATH_libatomic = $$r/$(TARGET_SUBDIR)/libatomic/.libs: +TARGET_PREV_LIB_PATH_libatomic = $$r/prev-$(TARGET_SUBDIR)/libatomic/.libs: @endif target-libatomic @@ -772,7 +783,8 @@ HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(H # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR) +HOST_PREV_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR) @endif gcc diff --git a/Makefile.tpl b/Makefile.tpl index cb39fbd0434..193cea6d803 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -269,6 +269,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(TARGET_PREV_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -652,9 +653,13 @@ all: TARGET_LIB_PATH = [+ FOR target_modules +][+ IF lib_path +]$(TARGET_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ ENDFOR target_modules +]$(HOST_LIB_PATH_gcc) +TARGET_PREV_LIB_PATH = [+ FOR target_modules +][+ + IF lib_path +]$(TARGET_PREV_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ + ENDFOR target_modules +]$(HOST_PREV_LIB_PATH_gcc) [+ FOR target_modules +][+ IF lib_path +] @if target-[+module+] TARGET_LIB_PATH_[+module+] = $$r/$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: +TARGET_PREV_LIB_PATH_[+module+] = $$r/prev-$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: @endif target-[+module+] [+ ENDIF lib_path +][+ ENDFOR target_modules +] @@ -667,7 +672,8 @@ HOST_LIB_PATH = [+ FOR host_modules +][+ # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR) +HOST_PREV_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR) @endif gcc [+ FOR host_modules +][+ IF lib_path +] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 17:16 ` Jason Merrill @ 2024-02-08 17:51 ` Iain Sandoe 2024-02-08 19:25 ` Jason Merrill 2024-02-08 17:55 ` Paolo Bonzini 1 sibling, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-08 17:51 UTC (permalink / raw) To: Jason Merrill Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva > On 8 Feb 2024, at 17:16, Jason Merrill <jason@redhat.com> wrote: > > On 2/8/24 12:12, Jason Merrill wrote: >> On 2/8/24 10:04, Iain Sandoe wrote: >>> Hi Jason, >>> >>> I have tested this on modern Darwin (with libc++ as the system library) and on >>> older Darwin, where we do see the issue - because the system linker is written >>> in C++ and links with libstdc++ (so sometimes we get a crash, or worse unpredictable >>> beahviour). >> Thank you! >>> ----- >>> >>> For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, but for older Darwin…. >>> >>>> On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: >>>> >>>> On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: >>>> >>>>> Reverting that hunk of the change fixed my problem with bubblestrapping GCC >>>>> 12 with ccache on a host with a newer system libstdc++. >>>> >>>> Did you have libcc1, gnattools and gotools enabled in your testing? >>> >>> … I have done all but “go” since that’s not supported on Darwin. >>> >>> The patch breaks bootstrap on older Darwin because: >>> >>> Ada uses exceptions. >>> gnat1 pulls in system libraries that link with the system unwinder >>> - so we have to link gnat1 “-shared-libgcc” >>> - which means we need to be able to find the just-built one when building the target libs. >> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >> In neither case do we want it to find the libgcc from the current stage. >> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >> Does this make sense? Yes in terms of me mixing up “just built” and “prev-*“ However, IIUC this still means we’re adding LD_LIBRARY_PATHs for each target library, including libstdc++ (so I’m not sure if it solves the problem that I’d seen before). I can see we need to add any paths to shared libraries used by the host tools (so that, perhaps, we’ll need to add libgrust if the rust FE uses that) but, otherwise AFAIK, the only [shared] target library used by the host tools is libgcc_s (when gnat1 needs it) - otherwise we default to static linking of libstdc++ and libgcc (and the gnat* tools link libgnat statically too) ? >> Does it work on Darwin? I’ll kick off a run shortly - know later this evening. Iain > > Oops, that was broken, please consider this one instead: > > <0001-build-add-prev-target-libs-to-rpath.patch> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 17:51 ` Iain Sandoe @ 2024-02-08 19:25 ` Jason Merrill 2024-02-08 19:52 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-08 19:25 UTC (permalink / raw) To: Iain Sandoe; +Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva On 2/8/24 12:51, Iain Sandoe wrote: > > >> On 8 Feb 2024, at 17:16, Jason Merrill <jason@redhat.com> wrote: >> >> On 2/8/24 12:12, Jason Merrill wrote: >>> On 2/8/24 10:04, Iain Sandoe wrote: >>>> Hi Jason, >>>> >>>> I have tested this on modern Darwin (with libc++ as the system library) and on >>>> older Darwin, where we do see the issue - because the system linker is written >>>> in C++ and links with libstdc++ (so sometimes we get a crash, or worse unpredictable >>>> beahviour). >>> Thank you! >>>> ----- >>>> >>>> For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, but for older Darwin…. >>>> >>>>> On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: >>>>> >>>>> On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: >>>>> >>>>>> Reverting that hunk of the change fixed my problem with bubblestrapping GCC >>>>>> 12 with ccache on a host with a newer system libstdc++. >>>>> >>>>> Did you have libcc1, gnattools and gotools enabled in your testing? >>>> >>>> … I have done all but “go” since that’s not supported on Darwin. >>>> >>>> The patch breaks bootstrap on older Darwin because: >>>> >>>> Ada uses exceptions. >>>> gnat1 pulls in system libraries that link with the system unwinder >>>> - so we have to link gnat1 “-shared-libgcc” >>>> - which means we need to be able to find the just-built one when building the target libs. >>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>> In neither case do we want it to find the libgcc from the current stage. >>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>> Does this make sense? > > Yes in terms of me mixing up “just built” and “prev-*“ I've been unclear about that myself. > However, IIUC this still means we’re adding LD_LIBRARY_PATHs for each target library, including libstdc++ (so I’m not sure if it solves the problem that I’d seen before). Yes, after stage 1. You're right that we won't need all of the stage1 target libraries to run the stage2 compiler. But if your gnat1 is linked with shared libgcc, isn't also linked with shared libstdc++, so it would need to find the stage1 libstdc++? In general, though, it would make sense to decide whether to include both libgcc and libstdc++ based on with_static_shared_libraries. And likewise the other language runtimes (libada, libgfortran, ...)? We probably don't need libgomp/libatomic/libitm? libsanitizer/vtv/ssp should only be needed for a bootstrap using those hardening options. If your stage2 gnat1 does need the stage1 libstdc++ and you're also using the system ld that needs the system libstdc++, that seems to call for another approach discussed on the PR: have exec-tool.in splice out the build directory from LD_LIBRARY_PATH when calling a tool from outside the build directory. Jason ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 19:25 ` Jason Merrill @ 2024-02-08 19:52 ` Iain Sandoe 2024-02-22 18:06 ` Gaius Mulley 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-08 19:52 UTC (permalink / raw) To: Jason Merrill Cc: GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva, Gaius Mulley > On 8 Feb 2024, at 19:25, Jason Merrill <jason@redhat.com> wrote: > > On 2/8/24 12:51, Iain Sandoe wrote: >>> On 8 Feb 2024, at 17:16, Jason Merrill <jason@redhat.com> wrote: >>> >>> On 2/8/24 12:12, Jason Merrill wrote: >>>> On 2/8/24 10:04, Iain Sandoe wrote: >>>>> Hi Jason, >>>>> >>>>> I have tested this on modern Darwin (with libc++ as the system library) and on >>>>> older Darwin, where we do see the issue - because the system linker is written >>>>> in C++ and links with libstdc++ (so sometimes we get a crash, or worse unpredictable >>>>> beahviour). >>>> Thank you! >>>>> ----- >>>>> >>>>> For modern Darwin [ > macOS 10.11] , there’s no issue seen so far, but for older Darwin…. >>>>> >>>>>> On 8 Feb 2024, at 05:36, Alexandre Oliva <oliva@adacore.com> wrote: >>>>>> >>>>>> On Feb 6, 2024, Jason Merrill <jason@redhat.com> wrote: >>>>>> >>>>>>> Reverting that hunk of the change fixed my problem with bubblestrapping GCC >>>>>>> 12 with ccache on a host with a newer system libstdc++. >>>>>> >>>>>> Did you have libcc1, gnattools and gotools enabled in your testing? >>>>> >>>>> … I have done all but “go” since that’s not supported on Darwin. >>>>> >>>>> The patch breaks bootstrap on older Darwin because: >>>>> >>>>> Ada uses exceptions. >>>>> gnat1 pulls in system libraries that link with the system unwinder >>>>> - so we have to link gnat1 “-shared-libgcc” >>>>> - which means we need to be able to find the just-built one when building the target libs. >>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>> In neither case do we want it to find the libgcc from the current stage. >>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>> Does this make sense? >> Yes in terms of me mixing up “just built” and “prev-*“ > > I've been unclear about that myself. > >> However, IIUC this still means we’re adding LD_LIBRARY_PATHs for each target library, including libstdc++ (so I’m not sure if it solves the problem that I’d seen before). > > Yes, after stage 1. > > You're right that we won't need all of the stage1 target libraries to run the stage2 compiler. But if your gnat1 is linked with shared libgcc, isn't also linked with shared libstdc++, so it would need to find the stage1 libstdc++? No, we continue to link with -static-libstdc++ (the only reason we use shared libgcc is to avoid two unwinders in the same process, which never ends well - usually gnat1 hangs) > In general, though, it would make sense to decide whether to include both libgcc and libstdc++ based on with_static_shared_libraries. And likewise the other language runtimes (libada, libgfortran, ...)? Right now, AFAIK the only target runtimes used by host tools are libstdc++, libgcc and libgnat. I agree that might change with rust - since the rust folks are talking about using one of the runtimes in the FE, I am not aware of other language FEs requiring their targte runtimes to be available to the host tools (adding Gaius in case I missed something with m2 - which is quite complex inthe bootstrapping). linking these shared would seem to me to be placing additional constraints on the build that are likely to take some shaking down - personally, I much prefer a toolchain with no shared deps - since then one knows what was tested is not going to change in reponse to some DSO update. > We probably don't need libgomp/libatomic/libitm? unless/until we have multiple threads in the compiler(s), it seems unlikely, indeed > libsanitizer/vtv/ssp should only be needed for a bootstrap using those hardening options. Ah, now in that case we probably _do_ need to force shared libstdc++ (since the sanitizers link against c++ lib) - an alternative is to force all of libstdc++.a to be included in the host tools, and use that to provide the symbols, > If your stage2 gnat1 does need the stage1 libstdc++ and you're also using the system ld that needs the system libstdc++, that seems to call for another approach discussed on the PR: have exec-tool.in splice out the build directory from LD_LIBRARY_PATH when calling a tool from outside the build directory. I’ve done all sorts of dances with this - e.g. linking the ld with a specially-named libstdc++ (or forcing the linker to be statically linked - although that creates a secondary problem since the Darwin libLTO (for clang) is also linked against c++) Iain > > Jason ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 19:52 ` Iain Sandoe @ 2024-02-22 18:06 ` Gaius Mulley 2024-02-22 20:30 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Gaius Mulley @ 2024-02-22 18:06 UTC (permalink / raw) To: Iain Sandoe Cc: Jason Merrill, GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva Iain Sandoe <idsandoe@googlemail.com> writes: > Right now, AFAIK the only target runtimes used by host tools are > libstdc++, libgcc and libgnat. I agree that might change with rust - > since the rust folks are talking about using one of the runtimes in > the FE, I am not aware of other language FEs requiring their targte > runtimes to be available to the host tools (adding Gaius in case I > missed something with m2 - which is quite complex inthe > bootstrapping). Hi Iain, the m2 infrastructure translates and builds gcc/m2/gm2-libs along with gcc/m2/gm2-compiler and uses these objects for cc1gm2, pge, mc etc - rather than the library archives generated from /libgm2 regards, Gaius ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-22 18:06 ` Gaius Mulley @ 2024-02-22 20:30 ` Iain Sandoe 2024-02-24 10:23 ` Gaius Mulley 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-22 20:30 UTC (permalink / raw) To: Gaius Mulley Cc: Jason Merrill, GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva Hi Gaius, > On 22 Feb 2024, at 18:06, Gaius Mulley <gaiusmod2@gmail.com> wrote: > > Iain Sandoe <idsandoe@googlemail.com> writes: > >> Right now, AFAIK the only target runtimes used by host tools are >> libstdc++, libgcc and libgnat. I agree that might change with rust - >> since the rust folks are talking about using one of the runtimes in >> the FE, I am not aware of other language FEs requiring their targte >> runtimes to be available to the host tools (adding Gaius in case I >> missed something with m2 - which is quite complex inthe >> bootstrapping). > the m2 infrastructure translates and builds gcc/m2/gm2-libs along with > gcc/m2/gm2-compiler and uses these objects for cc1gm2, pge, mc etc - > rather than the library archives generated from /libgm2 If I understand this (and my builds of the m2 stuff) correctly, this is done locally to the builds of the host-side components; in particular not controlled by the top level Makefile.{tpl,def}? (so that we do not see builds of libgm2 in stage1/2-<target> but only in the stage3-target builds? in which case, this should be outside the scope of the patch here. Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-22 20:30 ` Iain Sandoe @ 2024-02-24 10:23 ` Gaius Mulley 0 siblings, 0 replies; 31+ messages in thread From: Gaius Mulley @ 2024-02-24 10:23 UTC (permalink / raw) To: Iain Sandoe Cc: Jason Merrill, GCC Patches, Paolo Bonzini, Ian Lance Taylor, Alexandre Oliva Iain Sandoe <idsandoe@googlemail.com> writes: > Hi Gaius, > >> On 22 Feb 2024, at 18:06, Gaius Mulley <gaiusmod2@gmail.com> wrote: >> >> Iain Sandoe <idsandoe@googlemail.com> writes: >> >>> Right now, AFAIK the only target runtimes used by host tools are >>> libstdc++, libgcc and libgnat. I agree that might change with rust - >>> since the rust folks are talking about using one of the runtimes in >>> the FE, I am not aware of other language FEs requiring their targte >>> runtimes to be available to the host tools (adding Gaius in case I >>> missed something with m2 - which is quite complex inthe >>> bootstrapping). > >> the m2 infrastructure translates and builds gcc/m2/gm2-libs along with >> gcc/m2/gm2-compiler and uses these objects for cc1gm2, pge, mc etc - >> rather than the library archives generated from /libgm2 > > If I understand this (and my builds of the m2 stuff) correctly, this is done > locally to the builds of the host-side components; in particular not controlled > by the top level Makefile.{tpl,def}? Hi Iain, yes indeed, > (so that we do not see builds of libgm2 in stage1/2-<target> but only in the > stage3-target builds? > > in which case, this should be outside the scope of the patch here. regards, Gaius ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 17:16 ` Jason Merrill 2024-02-08 17:51 ` Iain Sandoe @ 2024-02-08 17:55 ` Paolo Bonzini 2024-02-08 21:44 ` Jason Merrill 1 sibling, 1 reply; 31+ messages in thread From: Paolo Bonzini @ 2024-02-08 17:55 UTC (permalink / raw) To: Jason Merrill, Iain Sandoe; +Cc: GCC Patches, Ian Lance Taylor, Alexandre Oliva On 2/8/24 18:16, Jason Merrill wrote: >>> >> >> Hmm. In stage 1, when we build with the system gcc, I'd think we want >> the just-built gnat1 to find the system libgcc. >> >> In stage 2, when we build with the stage 1 gcc, we want the just-built >> gnat1 to find the stage 1 libgcc. >> >> In neither case do we want it to find the libgcc from the current stage. >> >> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to >> include the TARGET_LIB_PATH from the previous stage. Something like >> the below, on top of the earlier patch. >> >> Does this make sense? Does it work on Darwin? > > Oops, that was broken, please consider this one instead: Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). Paolo ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 17:55 ` Paolo Bonzini @ 2024-02-08 21:44 ` Jason Merrill 2024-02-09 10:56 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-08 21:44 UTC (permalink / raw) To: Paolo Bonzini, Iain Sandoe; +Cc: GCC Patches, Ian Lance Taylor, Alexandre Oliva [-- Attachment #1: Type: text/plain, Size: 1077 bytes --] On 2/8/24 12:55, Paolo Bonzini wrote: > On 2/8/24 18:16, Jason Merrill wrote: >>>> >>> >>> Hmm. In stage 1, when we build with the system gcc, I'd think we >>> want the just-built gnat1 to find the system libgcc. >>> >>> In stage 2, when we build with the stage 1 gcc, we want the >>> just-built gnat1 to find the stage 1 libgcc. >>> >>> In neither case do we want it to find the libgcc from the current stage. >>> >>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to >>> include the TARGET_LIB_PATH from the previous stage. Something like >>> the below, on top of the earlier patch. >>> >>> Does this make sense? Does it work on Darwin? >> >> Oops, that was broken, please consider this one instead: > > Yes, this one makes sense (and the current code would not work since it > lacks the prev- prefix on TARGET_LIB_PATH). Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. So, here's another patch that just includes that for post-stage1: [-- Attachment #2: 0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch --] [-- Type: text/x-patch, Size: 5000 bytes --] From 3709ba23e4e3f5f6b2b6af709179ea5e471e2a55 Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Wed, 24 Jan 2024 07:47:26 -0500 Subject: [PATCH] build: drop target libs from LD_LIBRARY_PATH [PR105688] To: gcc-patches@gcc.gnu.org The patch for PR22340 (r104978) moved the adding of TARGET_LIB_PATH to RPATH_ENVVAR from POSTSTAGE1_HOST_EXPORTS to HOST_EXPORTS, but didn't mention that in the ChangeLog; it also wasn't part of the patch that was sent to gcc-patches. I suspect it was included accidentally? It also causes PR105688 when rebuilding stage1: once the stage1 libstdc++ has been built, if calling the system gcc to build host code involves invoking any tool that links against libstdc++.so (gold, ccache) they get the just-built library instead of the system library they expect. Reverting that hunk of the change fixed my problem with bubblestrapping GCC 12 with ccache on a host with a newer system libstdc++. But I believe that adding TARGET_LIB_PATH to RPATH_ENVVAR is not needed for post-stage1 either. Including TARGET_LIB_PATH goes back to r37545, with the stated rationale of getting other C++ library configury to succeed, but that seems like a rationale for having it in TARGET_EXPORTS, not HOST_. And when building stage2+, looking for current stage target libraries isn't what we want anyway; we would have wanted the previous stage target libraries, and we haven't been looking for them, except for the previous stage libgcc that we get from HOST_LIB_PATH_gcc. So, for stage2+, let's add just prev-gcc. PR bootstrap/105688 ChangeLog: * Makefile.tpl (HOST_EXPORTS): Don't add TARGET_LIB_PATH to RPATH_ENVVAR. (POSTSTAGE1_HOST_EXPORTS): Add PREV_HOST_LIB_PATH_gcc. (PREV_HOST_LIB_PATH_gcc): Split out from HOST_LIB_PATH_gcc. * Makefile.in: Regenerate. --- Makefile.in | 7 +++---- Makefile.tpl | 7 +++---- 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/Makefile.in b/Makefile.in index edb0c8a9a42..c248003ee36 100644 --- a/Makefile.in +++ b/Makefile.in @@ -242,9 +242,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -269,6 +266,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(HOST_PREV_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -775,7 +773,8 @@ HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(H # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR): +HOST_PREV_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): @endif gcc diff --git a/Makefile.tpl b/Makefile.tpl index adbcbdd1d57..844b9e8f63a 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -245,9 +245,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -272,6 +269,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(PREV_HOST_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -670,7 +668,8 @@ HOST_LIB_PATH = [+ FOR host_modules +][+ # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR): +PREV_HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): @endif gcc [+ FOR host_modules +][+ IF lib_path +] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-08 21:44 ` Jason Merrill @ 2024-02-09 10:56 ` Iain Sandoe 2024-02-09 23:21 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-09 10:56 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: > > On 2/8/24 12:55, Paolo Bonzini wrote: >> On 2/8/24 18:16, Jason Merrill wrote: >>>>> >>>> >>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>> >>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>> >>>> In neither case do we want it to find the libgcc from the current stage. >>>> >>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>> >>>> Does this make sense? Does it work on Darwin? >>> >>> Oops, that was broken, please consider this one instead: >> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). > > Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. > > So, here's another patch that just includes that for post-stage1: > <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> Hmm this still fails for me with gnat1 being unable to find libgcc_s. It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what exactly is failing. Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-09 10:56 ` Iain Sandoe @ 2024-02-09 23:21 ` Iain Sandoe 2024-02-10 10:46 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-09 23:21 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >> >> On 2/8/24 12:55, Paolo Bonzini wrote: >>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>> >>>>> >>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>> >>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>> >>>>> In neither case do we want it to find the libgcc from the current stage. >>>>> >>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>> >>>>> Does this make sense? Does it work on Darwin? >>>> >>>> Oops, that was broken, please consider this one instead: >>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >> >> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >> >> So, here's another patch that just includes that for post-stage1: >> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> > > Hmm this still fails for me with gnat1 being unable to find libgcc_s. > It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, > presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what > exactly is failing. The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it requires this host path to be added to the target exports since it’s a host library used during target builds (or do folks expect the host exports to be made for target lib builds as well?) Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this because HOST_LIB_PATH is appended to the TARGET_EXPORTS (we do not seem to make those depend on the stage). Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-09 23:21 ` Iain Sandoe @ 2024-02-10 10:46 ` Iain Sandoe 2024-02-10 12:07 ` Jason Merrill 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-10 10:46 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: > > > >> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>> >>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>> >>>>>> >>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>> >>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>> >>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>> >>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>> >>>>>> Does this make sense? Does it work on Darwin? >>>>> >>>>> Oops, that was broken, please consider this one instead: >>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>> >>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>> >>> So, here's another patch that just includes that for post-stage1: >>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >> >> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >> exactly is failing. > > The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it > requires this host path to be added to the target exports since it’s a host library used during target > builds (or do folks expect the host exports to be made for target lib builds as well?) > > Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this Hmm this is still not right, in this case, I think it should actually be the “just built” directory; - if we have a tool that depends on host libraries (that happen to be also target ones), then those libraries have to be built before the tool so that they can be linked to it. (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be linked to the just-built host libraries (which will also be installed). So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). It would be nice to declare the libraries that are used for both host and target as separate modules in the top level template (since we might want to build them with different configure options; e.g. there’s no real point in bootstrapping the host libstdc++ as a shared lib - pulling in its dependent libs - if we are only going to link the .a). Iain (will check a cross-compiler build with the changes above). > because HOST_LIB_PATH is > appended to the TARGET_EXPORTS (we do not seem to make those depend on the stage). > > Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-10 10:46 ` Iain Sandoe @ 2024-02-10 12:07 ` Jason Merrill 2024-02-10 12:30 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-10 12:07 UTC (permalink / raw) To: Iain Sandoe; +Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva On 2/10/24 05:46, Iain Sandoe wrote: > > >> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >> >> >> >>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>> >>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>> >>>>>>> >>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>> >>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>> >>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>> >>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>> >>>>>>> Does this make sense? Does it work on Darwin? >>>>>> >>>>>> Oops, that was broken, please consider this one instead: >>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>> >>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>> >>>> So, here's another patch that just includes that for post-stage1: >>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>> >>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>> exactly is failing. >> >> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >> requires this host path to be added to the target exports since it’s a host library used during target >> builds (or do folks expect the host exports to be made for target lib builds as well?) >> >> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this > > Hmm this is still not right, in this case, I think it should actually be the “just built” directory; > > - if we have a tool that depends on host libraries (that happen to be also target ones), > then those libraries have to be built before the tool so that they can be linked to it. > (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) > > - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be > linked to the just-built host libraries (which will also be installed). > > So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH > and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. What am I missing? Jason ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-10 12:07 ` Jason Merrill @ 2024-02-10 12:30 ` Iain Sandoe 2024-02-12 19:59 ` Jason Merrill 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-10 12:30 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: > > On 2/10/24 05:46, Iain Sandoe wrote: >>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>> >>> >>> >>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>> >>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>> >>>>>>>> >>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>> >>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>> >>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>> >>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>> >>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>> >>>>>>> Oops, that was broken, please consider this one instead: >>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>> >>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>> >>>>> So, here's another patch that just includes that for post-stage1: >>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>> >>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>> exactly is failing. >>> >>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>> requires this host path to be added to the target exports since it’s a host library used during target >>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>> >>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >> - if we have a tool that depends on host libraries (that happen to be also target ones), >> then those libraries have to be built before the tool so that they can be linked to it. >> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >> linked to the just-built host libraries (which will also be installed). >> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). > > I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. > > In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. > > So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. > > What am I missing? nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. Iain > Jason ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-10 12:30 ` Iain Sandoe @ 2024-02-12 19:59 ` Jason Merrill 2024-02-14 22:59 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-12 19:59 UTC (permalink / raw) To: Iain Sandoe; +Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva [-- Attachment #1: Type: text/plain, Size: 4277 bytes --] On 2/10/24 07:30, Iain Sandoe wrote: > > >> On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: >> >> On 2/10/24 05:46, Iain Sandoe wrote: >>>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>> >>>> >>>> >>>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>>> >>>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>>> >>>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>>> >>>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>>> >>>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>>> >>>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>>> >>>>>>>> Oops, that was broken, please consider this one instead: >>>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>>> >>>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>>> >>>>>> So, here's another patch that just includes that for post-stage1: >>>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>>> >>>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>>> exactly is failing. >>>> >>>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>>> requires this host path to be added to the target exports since it’s a host library used during target >>>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>>> >>>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >>> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >>> - if we have a tool that depends on host libraries (that happen to be also target ones), >>> then those libraries have to be built before the tool so that they can be linked to it. >>> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >>> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >>> linked to the just-built host libraries (which will also be installed). >>> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >>> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). >> >> I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. >> >> In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. >> >> So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. >> >> What am I missing? > > nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. > > However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. Ah, I see: When building target libraries in stage2, we run the stage2 compiler that needs the stage1 libgcc_s, but we don't have the HOST_EXPORTS because we're building target code, so we also need to get the libgcc path into TARGET_EXPORTS. Since TARGET_LIB_PATH is only added when gcc-bootstrap, I guess the previous libgcc is the only piece needed in TARGET_EXPORTS as well. So, how about this version of the patch? Jason [-- Attachment #2: 0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch --] [-- Type: text/x-patch, Size: 6361 bytes --] From 843303e635ec55f512cfb97650032dc921f0c117 Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Wed, 24 Jan 2024 07:47:26 -0500 Subject: [PATCH] build: drop target libs from LD_LIBRARY_PATH [PR105688] To: gcc-patches@gcc.gnu.org The patch for PR22340 (r104978) moved the adding of TARGET_LIB_PATH to RPATH_ENVVAR from POSTSTAGE1_HOST_EXPORTS to HOST_EXPORTS, but didn't mention that in the ChangeLog; it also wasn't part of the patch that was sent to gcc-patches. I suspect it was included accidentally? It also causes PR105688 when rebuilding stage1: once the stage1 libstdc++ has been built, if calling the system gcc to build host code involves invoking any tool that links against libstdc++.so (gold, ccache) they get the just-built library instead of the system library they expect. Reverting that hunk of the change fixed my problem with bubblestrapping GCC 12 with ccache on a host with a newer system libstdc++. But I believe that adding TARGET_LIB_PATH to RPATH_ENVVAR is not needed for post-stage1 either. Including TARGET_LIB_PATH goes back to r37545, with the stated rationale of getting other C++ library configury to succeed, but that seems like a rationale for having it in TARGET_EXPORTS, not HOST_. And when building stage2+, looking for current stage target libraries isn't what we want anyway; we would have wanted the previous stage target libraries, and we haven't been looking for them, except for the previous stage libgcc that we get from HOST_LIB_PATH_gcc. So, for stage2+, let's add just prev-gcc. PR bootstrap/105688 ChangeLog: * Makefile.tpl (HOST_EXPORTS): Don't add TARGET_LIB_PATH to RPATH_ENVVAR. (POSTSTAGE1_HOST_EXPORTS): Add PREV_HOST_LIB_PATH_gcc. (BASE_TARGET_EXPORTS): Add PREV_HOST_LIB_PATH_gcc instead of TARGET_LIB_PATH. (PREV_HOST_LIB_PATH_gcc): Split out from HOST_LIB_PATH_gcc. * Makefile.in: Regenerate. --- Makefile.in | 9 ++++----- Makefile.tpl | 9 ++++----- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/Makefile.in b/Makefile.in index edb0c8a9a42..5ff2de9d590 100644 --- a/Makefile.in +++ b/Makefile.in @@ -242,9 +242,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -269,6 +266,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(PREV_HOST_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -326,7 +324,7 @@ BASE_TARGET_EXPORTS = \ WINDRES="$(WINDRES_FOR_TARGET)"; export WINDRES; \ WINDMC="$(WINDMC_FOR_TARGET)"; export WINDMC; \ @if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ + $(RPATH_ENVVAR)=`echo "$(PREV_HOST_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ @endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ TARGET_CONFIGDIRS="$(TARGET_CONFIGDIRS)"; export TARGET_CONFIGDIRS; @@ -775,7 +773,8 @@ HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(H # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR): +PREV_HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): @endif gcc diff --git a/Makefile.tpl b/Makefile.tpl index adbcbdd1d57..6f774336b8d 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -245,9 +245,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -272,6 +269,7 @@ POSTSTAGE1_CXX_EXPORT = \ # Similar, for later GCC stages. POSTSTAGE1_HOST_EXPORTS = \ $(HOST_EXPORTS) \ + $(RPATH_ENVVAR)=`echo "$(PREV_HOST_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ CC="$(STAGE_CC_WRAPPER) $$r/$(HOST_SUBDIR)/prev-gcc/xgcc$(exeext) \ -B$$r/$(HOST_SUBDIR)/prev-gcc/ -B$(build_tooldir)/bin/ \ $(XGCC_FLAGS_FOR_TARGET) $$TFLAGS"; export CC; \ @@ -329,7 +327,7 @@ BASE_TARGET_EXPORTS = \ WINDRES="$(WINDRES_FOR_TARGET)"; export WINDRES; \ WINDMC="$(WINDMC_FOR_TARGET)"; export WINDMC; \ @if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ + $(RPATH_ENVVAR)=`echo "$(PREV_HOST_LIB_PATH_gcc)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ @endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ TARGET_CONFIGDIRS="$(TARGET_CONFIGDIRS)"; export TARGET_CONFIGDIRS; @@ -670,7 +668,8 @@ HOST_LIB_PATH = [+ FOR host_modules +][+ # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR): +PREV_HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): @endif gcc [+ FOR host_modules +][+ IF lib_path +] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-12 19:59 ` Jason Merrill @ 2024-02-14 22:59 ` Iain Sandoe 2024-02-14 23:33 ` Iain Sandoe 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-14 22:59 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 12 Feb 2024, at 19:59, Jason Merrill <jason@redhat.com> wrote: > > On 2/10/24 07:30, Iain Sandoe wrote: >>> On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: >>> >>> On 2/10/24 05:46, Iain Sandoe wrote: >>>>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>> >>>>> >>>>> >>>>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>>>> >>>>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>>>> >>>>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>>>> >>>>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>>>> >>>>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>>>> >>>>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>>>> >>>>>>>>> Oops, that was broken, please consider this one instead: >>>>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>>>> >>>>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>>>> >>>>>>> So, here's another patch that just includes that for post-stage1: >>>>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>>>> >>>>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>>>> exactly is failing. >>>>> >>>>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>>>> requires this host path to be added to the target exports since it’s a host library used during target >>>>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>>>> >>>>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >>>> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >>>> - if we have a tool that depends on host libraries (that happen to be also target ones), >>>> then those libraries have to be built before the tool so that they can be linked to it. >>>> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >>>> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >>>> linked to the just-built host libraries (which will also be installed). >>>> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >>>> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). >>> >>> I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. >>> >>> In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. >>> >>> So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. >>> >>> What am I missing? >> nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. >> However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. > > Ah, I see: When building target libraries in stage2, we run the stage2 compiler that needs the stage1 libgcc_s, but we don't have the HOST_EXPORTS because we're building target code, so we also need to get the libgcc path into TARGET_EXPORTS. > > Since TARGET_LIB_PATH is only added when gcc-bootstrap, I guess the previous libgcc is the only piece needed in TARGET_EXPORTS as well. So, how about this version of the patch? I tested this one on an affected platform version with and without —enable-host-shared and for all languages (less go which is not yet supported). It works for me, thanks, Iain > > Jason<0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-14 22:59 ` Iain Sandoe @ 2024-02-14 23:33 ` Iain Sandoe 2024-02-16 21:05 ` Jason Merrill 0 siblings, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-14 23:33 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 14 Feb 2024, at 22:59, Iain Sandoe <idsandoe@googlemail.com> wrote: >> On 12 Feb 2024, at 19:59, Jason Merrill <jason@redhat.com> wrote: >> >> On 2/10/24 07:30, Iain Sandoe wrote: >>>> On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: >>>> >>>> On 2/10/24 05:46, Iain Sandoe wrote: >>>>>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>>>>> >>>>>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>>>>> >>>>>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>>>>> >>>>>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>>>>> >>>>>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>>>>> >>>>>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>>>>> >>>>>>>>>> Oops, that was broken, please consider this one instead: >>>>>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>>>>> >>>>>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>>>>> >>>>>>>> So, here's another patch that just includes that for post-stage1: >>>>>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>>>>> >>>>>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>>>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>>>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>>>>> exactly is failing. >>>>>> >>>>>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>>>>> requires this host path to be added to the target exports since it’s a host library used during target >>>>>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>>>>> >>>>>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >>>>> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >>>>> - if we have a tool that depends on host libraries (that happen to be also target ones), >>>>> then those libraries have to be built before the tool so that they can be linked to it. >>>>> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >>>>> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >>>>> linked to the just-built host libraries (which will also be installed). >>>>> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >>>>> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). >>>> >>>> I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. >>>> >>>> In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. >>>> >>>> So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. >>>> >>>> What am I missing? >>> nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. >>> However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. >> >> Ah, I see: When building target libraries in stage2, we run the stage2 compiler that needs the stage1 libgcc_s, but we don't have the HOST_EXPORTS because we're building target code, so we also need to get the libgcc path into TARGET_EXPORTS. >> >> Since TARGET_LIB_PATH is only added when gcc-bootstrap, I guess the previous libgcc is the only piece needed in TARGET_EXPORTS as well. So, how about this version of the patch? > > I tested this one on an affected platform version with and without —enable-host-shared and for all languages (less go which is not yet supported). It works for me, thanks, > Iain Incidentally, during my investigations I was looking into various parts of this and it seems that actually TARGET_LIB_PATH might well be effectively dead code now. Outside of the top level Makefile, it is referred to by libgm2 and libgrust top level Makefile.am (as a passed-on flag), however AFAICT it is not exported from the top level Makefile (so, I assume that those references are also unused). Iain > > >> >> Jason<0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-14 23:33 ` Iain Sandoe @ 2024-02-16 21:05 ` Jason Merrill 2024-02-19 20:08 ` Iain Sandoe 2024-02-20 5:45 ` Alexandre Oliva 0 siblings, 2 replies; 31+ messages in thread From: Jason Merrill @ 2024-02-16 21:05 UTC (permalink / raw) To: Iain Sandoe; +Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva [-- Attachment #1: Type: text/plain, Size: 5047 bytes --] On 2/14/24 18:33, Iain Sandoe wrote: > >> On 14 Feb 2024, at 22:59, Iain Sandoe <idsandoe@googlemail.com> wrote: > >>> On 12 Feb 2024, at 19:59, Jason Merrill <jason@redhat.com> wrote: >>> >>> On 2/10/24 07:30, Iain Sandoe wrote: >>>>> On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: >>>>> >>>>> On 2/10/24 05:46, Iain Sandoe wrote: >>>>>>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>>>>>> >>>>>>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>>>>>> >>>>>>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>>>>>> >>>>>>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>>>>>> >>>>>>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>>>>>> >>>>>>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>>>>>> >>>>>>>>>>> Oops, that was broken, please consider this one instead: >>>>>>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>>>>>> >>>>>>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>>>>>> >>>>>>>>> So, here's another patch that just includes that for post-stage1: >>>>>>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>>>>>> >>>>>>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>>>>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>>>>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>>>>>> exactly is failing. >>>>>>> >>>>>>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>>>>>> requires this host path to be added to the target exports since it’s a host library used during target >>>>>>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>>>>>> >>>>>>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >>>>>> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >>>>>> - if we have a tool that depends on host libraries (that happen to be also target ones), >>>>>> then those libraries have to be built before the tool so that they can be linked to it. >>>>>> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >>>>>> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >>>>>> linked to the just-built host libraries (which will also be installed). >>>>>> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >>>>>> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). >>>>> >>>>> I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. >>>>> >>>>> In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. >>>>> >>>>> So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. >>>>> >>>>> What am I missing? >>>> nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. >>>> However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. >>> >>> Ah, I see: When building target libraries in stage2, we run the stage2 compiler that needs the stage1 libgcc_s, but we don't have the HOST_EXPORTS because we're building target code, so we also need to get the libgcc path into TARGET_EXPORTS. >>> >>> Since TARGET_LIB_PATH is only added when gcc-bootstrap, I guess the previous libgcc is the only piece needed in TARGET_EXPORTS as well. So, how about this version of the patch? >> >> I tested this one on an affected platform version with and without —enable-host-shared and for all languages (less go which is not yet supported). It works for me, thanks, >> Iain > > Incidentally, during my investigations I was looking into various parts of this and it seems that actually TARGET_LIB_PATH might well be effectively dead code now. Good point. Could you test this version as well? [-- Attachment #2: 0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch --] [-- Type: text/x-patch, Size: 8838 bytes --] From 6cd188de23ba5b7ac38a7902580fd861af03d3aa Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Wed, 24 Jan 2024 07:47:26 -0500 Subject: [PATCH] build: drop target libs from LD_LIBRARY_PATH [PR105688] To: gcc-patches@gcc.gnu.org The patch for PR22340 (r104978) moved the adding of TARGET_LIB_PATH to RPATH_ENVVAR from POSTSTAGE1_HOST_EXPORTS to HOST_EXPORTS, but didn't mention that in the ChangeLog; it also wasn't part of the patch that was sent to gcc-patches. I suspect it was included accidentally? It also causes PR105688 when rebuilding stage1: once the stage1 libstdc++ has been built, if calling the system gcc to build host code involves invoking any tool that links against libstdc++.so (gold, ccache) they get the just-built library instead of the system library they expect. Reverting that hunk of the change fixed my problem with bubblestrapping GCC 12 with ccache on a host with a newer system libstdc++. But I believe that adding TARGET_LIB_PATH to RPATH_ENVVAR is not needed for post-stage1 either. Including TARGET_LIB_PATH goes back to r37545, with the stated rationale of getting other C++ library configury to succeed, but that seems like a rationale for having it in TARGET_EXPORTS, not HOST_. And when building stage2+, looking for current stage target libraries isn't what we want anyway; we would have wanted the previous stage target libraries, and we haven't been looking for them, except for the previous stage libgcc that we get from HOST_LIB_PATH_gcc. We do want them for running testcases, but the testsuite handles that in target-libpath.exp. So, for stage2+, let's add just prev- libgcc. We do this in both HOST_ and TARGET_EXPORTS; when building stage2 host tools with the stage1 compiler, if the stage1 compiler was built with the system gcc, it will load the stage1 libgcc instead of the system libgcc--but that also happens without this change. If that turns out to be a problem, we could arrange to only add it to HOST_EXPORTS for stage3+ in a later patch. With this change nothing is using TARGET_LIB_PATH, so let's remove it. Many thanks to Iain Sandoe, who tested this on Darwin systems both with and without system libstdc++, and reports it working. PR bootstrap/105688 ChangeLog: * Makefile.tpl (HOST_EXPORTS, BASE_TARGET_EXPORTS): Don't add TARGET_LIB_PATH to RPATH_ENVVAR. (TARGET_LIB_PATH): Remove. (HOST_LIB_PATH): Add HOST_LIB_PATH_gcc. (HOST_LIB_PATH_gcc): In prev-gcc, and only when bootstrapping. * Makefile.in: Regenerate. --- Makefile.in | 59 ++++++---------------------------------------------- Makefile.tpl | 29 ++++++-------------------- 2 files changed, 12 insertions(+), 76 deletions(-) diff --git a/Makefile.in b/Makefile.in index edb0c8a9a42..9d801464251 100644 --- a/Makefile.in +++ b/Makefile.in @@ -242,9 +242,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -325,9 +322,6 @@ BASE_TARGET_EXPORTS = \ SYSROOT_CFLAGS_FOR_TARGET="$(SYSROOT_CFLAGS_FOR_TARGET)"; export SYSROOT_CFLAGS_FOR_TARGET; \ WINDRES="$(WINDRES_FOR_TARGET)"; export WINDRES; \ WINDMC="$(WINDMC_FOR_TARGET)"; export WINDMC; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ TARGET_CONFIGDIRS="$(TARGET_CONFIGDIRS)"; export TARGET_CONFIGDIRS; @@ -727,56 +721,15 @@ all: @host_makefile_frag@ ### -# This is the list of directories that may be needed in RPATH_ENVVAR -# so that programs built for the target machine work. -TARGET_LIB_PATH = $(TARGET_LIB_PATH_libstdc++-v3)$(TARGET_LIB_PATH_libsanitizer)$(TARGET_LIB_PATH_libvtv)$(TARGET_LIB_PATH_libssp)$(TARGET_LIB_PATH_libphobos)$(TARGET_LIB_PATH_libgm2)$(TARGET_LIB_PATH_libgomp)$(TARGET_LIB_PATH_libitm)$(TARGET_LIB_PATH_libatomic)$(HOST_LIB_PATH_gcc) - -@if target-libstdc++-v3 -TARGET_LIB_PATH_libstdc++-v3 = $$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs: -@endif target-libstdc++-v3 - -@if target-libsanitizer -TARGET_LIB_PATH_libsanitizer = $$r/$(TARGET_SUBDIR)/libsanitizer/.libs: -@endif target-libsanitizer - -@if target-libvtv -TARGET_LIB_PATH_libvtv = $$r/$(TARGET_SUBDIR)/libvtv/.libs: -@endif target-libvtv - -@if target-libssp -TARGET_LIB_PATH_libssp = $$r/$(TARGET_SUBDIR)/libssp/.libs: -@endif target-libssp - -@if target-libphobos -TARGET_LIB_PATH_libphobos = $$r/$(TARGET_SUBDIR)/libphobos/src/.libs: -@endif target-libphobos - -@if target-libgm2 -TARGET_LIB_PATH_libgm2 = $$r/$(TARGET_SUBDIR)/libgm2/.libs: -@endif target-libgm2 - -@if target-libgomp -TARGET_LIB_PATH_libgomp = $$r/$(TARGET_SUBDIR)/libgomp/.libs: -@endif target-libgomp - -@if target-libitm -TARGET_LIB_PATH_libitm = $$r/$(TARGET_SUBDIR)/libitm/.libs: -@endif target-libitm - -@if target-libatomic -TARGET_LIB_PATH_libatomic = $$r/$(TARGET_SUBDIR)/libatomic/.libs: -@endif target-libatomic - - - # This is the list of directories that may be needed in RPATH_ENVVAR # so that programs built for the host machine work. -HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl) +HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl)$(HOST_LIB_PATH_gcc) -# Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch -@if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): -@endif gcc +# libgcc is a target library in the current stage and a host library +# in the previous stage (when bootstrapping). +@if gcc-bootstrap +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +@endif gcc-bootstrap @if gmp diff --git a/Makefile.tpl b/Makefile.tpl index adbcbdd1d57..e3de2a7ba3b 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -245,9 +245,6 @@ HOST_EXPORTS = \ ISLLIBS="$(HOST_ISLLIBS)"; export ISLLIBS; \ ISLINC="$(HOST_ISLINC)"; export ISLINC; \ XGCC_FLAGS_FOR_TARGET="$(XGCC_FLAGS_FOR_TARGET)"; export XGCC_FLAGS_FOR_TARGET; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); POSTSTAGE1_CXX_EXPORT = \ @@ -328,9 +325,6 @@ BASE_TARGET_EXPORTS = \ SYSROOT_CFLAGS_FOR_TARGET="$(SYSROOT_CFLAGS_FOR_TARGET)"; export SYSROOT_CFLAGS_FOR_TARGET; \ WINDRES="$(WINDRES_FOR_TARGET)"; export WINDRES; \ WINDMC="$(WINDMC_FOR_TARGET)"; export WINDMC; \ -@if gcc-bootstrap - $(RPATH_ENVVAR)=`echo "$(TARGET_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ -@endif gcc-bootstrap $(RPATH_ENVVAR)=`echo "$(HOST_LIB_PATH)$$$(RPATH_ENVVAR)" | sed 's,::*,:,g;s,^:*,,;s,:*$$,,'`; export $(RPATH_ENVVAR); \ TARGET_CONFIGDIRS="$(TARGET_CONFIGDIRS)"; export TARGET_CONFIGDIRS; @@ -650,28 +644,17 @@ all: @host_makefile_frag@ ### -# This is the list of directories that may be needed in RPATH_ENVVAR -# so that programs built for the target machine work. -TARGET_LIB_PATH = [+ FOR target_modules +][+ - IF lib_path +]$(TARGET_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ - ENDFOR target_modules +]$(HOST_LIB_PATH_gcc) -[+ FOR target_modules +][+ IF lib_path +] -@if target-[+module+] -TARGET_LIB_PATH_[+module+] = $$r/$(TARGET_SUBDIR)/[+module+]/[+lib_path+]: -@endif target-[+module+] -[+ ENDIF lib_path +][+ ENDFOR target_modules +] - - # This is the list of directories that may be needed in RPATH_ENVVAR # so that programs built for the host machine work. HOST_LIB_PATH = [+ FOR host_modules +][+ IF lib_path +]$(HOST_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ - ENDFOR host_modules +] + ENDFOR host_modules +]$(HOST_LIB_PATH_gcc) -# Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch -@if gcc -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): -@endif gcc +# libgcc is a target library in the current stage and a host library +# in the previous stage (when bootstrapping). +@if gcc-bootstrap +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +@endif gcc-bootstrap [+ FOR host_modules +][+ IF lib_path +] @if [+module+] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-16 21:05 ` Jason Merrill @ 2024-02-19 20:08 ` Iain Sandoe 2024-02-20 5:45 ` Alexandre Oliva 1 sibling, 0 replies; 31+ messages in thread From: Iain Sandoe @ 2024-02-19 20:08 UTC (permalink / raw) To: Jason Merrill Cc: Paolo Bonzini, GCC Patches, Ian Lance Taylor, Alexandre Oliva > On 16 Feb 2024, at 21:05, Jason Merrill <jason@redhat.com> wrote: > > On 2/14/24 18:33, Iain Sandoe wrote: >>> On 14 Feb 2024, at 22:59, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>> On 12 Feb 2024, at 19:59, Jason Merrill <jason@redhat.com> wrote: >>>> >>>> On 2/10/24 07:30, Iain Sandoe wrote: >>>>>> On 10 Feb 2024, at 12:07, Jason Merrill <jason@redhat.com> wrote: >>>>>> >>>>>> On 2/10/24 05:46, Iain Sandoe wrote: >>>>>>>> On 9 Feb 2024, at 23:21, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 9 Feb 2024, at 10:56, Iain Sandoe <idsandoe@googlemail.com> wrote: >>>>>>>>>> On 8 Feb 2024, at 21:44, Jason Merrill <jason@redhat.com> wrote: >>>>>>>>>> >>>>>>>>>> On 2/8/24 12:55, Paolo Bonzini wrote: >>>>>>>>>>> On 2/8/24 18:16, Jason Merrill wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hmm. In stage 1, when we build with the system gcc, I'd think we want the just-built gnat1 to find the system libgcc. >>>>>>>>>>>>> >>>>>>>>>>>>> In stage 2, when we build with the stage 1 gcc, we want the just-built gnat1 to find the stage 1 libgcc. >>>>>>>>>>>>> >>>>>>>>>>>>> In neither case do we want it to find the libgcc from the current stage. >>>>>>>>>>>>> >>>>>>>>>>>>> So it seems to me that what we want is for stage2+ LD_LIBRARY_PATH to include the TARGET_LIB_PATH from the previous stage. Something like the below, on top of the earlier patch. >>>>>>>>>>>>> >>>>>>>>>>>>> Does this make sense? Does it work on Darwin? >>>>>>>>>>>> >>>>>>>>>>>> Oops, that was broken, please consider this one instead: >>>>>>>>>>> Yes, this one makes sense (and the current code would not work since it lacks the prev- prefix on TARGET_LIB_PATH). >>>>>>>>>> >>>>>>>>>> Indeed, that seems like evidence that the only element of TARGET_LIB_PATH that has been useful in HOST_EXPORTS is the prev- part of HOST_LIB_PATH_gcc. >>>>>>>>>> >>>>>>>>>> So, here's another patch that just includes that for post-stage1: >>>>>>>>>> <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> >>>>>>>>> >>>>>>>>> Hmm this still fails for me with gnat1 being unable to find libgcc_s. >>>>>>>>> It seems I have to add the PREV_HOST_LIB_PATH_gcc to HOST_LIB_PATH for it to succeed so, >>>>>>>>> presumably, the post stage1 exports are not being forwarded to that build. I’ll try to analyze what >>>>>>>>> exactly is failing. >>>>>>>> >>>>>>>> The fail is occurring in the target libada build; so, I suppose, one might say it’s reasonable that it >>>>>>>> requires this host path to be added to the target exports since it’s a host library used during target >>>>>>>> builds (or do folks expect the host exports to be made for target lib builds as well?) >>>>>>>> >>>>>>>> Appending the prev-gcc dirctory to the HOST_LIB_PATH fixes this >>>>>>> Hmm this is still not right, in this case, I think it should actually be the “just built” directory; >>>>>>> - if we have a tool that depends on host libraries (that happen to be also target ones), >>>>>>> then those libraries have to be built before the tool so that they can be linked to it. >>>>>>> (we specially copy libgcc* and the CRTs to gcc/ to allow for this case) >>>>>>> - there is no prev-gcc in cross and —disable-bootstrap builds, but the tool will still be >>>>>>> linked to the just-built host libraries (which will also be installed). >>>>>>> So, I think we have to add HOST_LIB_PATH_gcc to HOST_LIB_PATH >>>>>>> and HOST_PREV_LIB_PATH_gcc to POSTSTAGE1_HOST_EXPORTS (as per this patch). >>>>>> >>>>>> I don't follow. In a cross build, host libraries are a different architecture from target libraries, and certainly can't be linked into host binaries. >>>>>> >>>>>> In a disable-bootstrap build, even before my change TARGET_LIB_PATH isn't added to RPATH_ENVVAR, since that has been guarded with @if gcc-bootstrap. >>>>>> >>>>>> So in a bootstrap build, it shouldn't be needed for stage1 either. And for stage2, the one we need is from stage1, that matches the compiler we're building host tools with. >>>>>> >>>>>> What am I missing? >>>>> nothing, I was off on a tangent about the cross/non-bootstrap, sorry about that. >>>>> However, when doing target builds (the previous point) it seems we do have to make provision for gnat1 to find libgcc_s, and, at present, it seems that only the target exports are active. >>>> >>>> Ah, I see: When building target libraries in stage2, we run the stage2 compiler that needs the stage1 libgcc_s, but we don't have the HOST_EXPORTS because we're building target code, so we also need to get the libgcc path into TARGET_EXPORTS. >>>> >>>> Since TARGET_LIB_PATH is only added when gcc-bootstrap, I guess the previous libgcc is the only piece needed in TARGET_EXPORTS as well. So, how about this version of the patch? >>> >>> I tested this one on an affected platform version with and without —enable-host-shared and for all languages (less go which is not yet supported). It works for me, thanks, >>> Iain >> Incidentally, during my investigations I was looking into various parts of this and it seems that actually TARGET_LIB_PATH might well be effectively dead code now. > > Good point. Could you test this version as well? > > > <0001-build-drop-target-libs-from-LD_LIBRARY_PATH-PR105688.patch> This also works for me on a number of darwin versions (including one that actually needs libgcc_s to bootstrap), thanks Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-16 21:05 ` Jason Merrill 2024-02-19 20:08 ` Iain Sandoe @ 2024-02-20 5:45 ` Alexandre Oliva 2024-02-21 23:06 ` Jason Merrill 1 sibling, 1 reply; 31+ messages in thread From: Alexandre Oliva @ 2024-02-20 5:45 UTC (permalink / raw) To: Jason Merrill; +Cc: Iain Sandoe, Paolo Bonzini, GCC Patches, Ian Lance Taylor On Feb 16, 2024, Jason Merrill <jason@redhat.com> wrote: > So, for stage2+, let's add just prev- libgcc. I'm pretty sure this will break bootstrap-lean where libgcc_s isn't a system library, and we're building post-bootstrap host tools :-( We need the current stage lib after the prev stage is removed. I also doubt that TARGET_LIB_PATH was defined and used for no reason. My hunch is that bootstrap options and/or targets that don't have these libraries as system libraries will break in some obscure way without it. But I don't have the bandwidth to track down the history behind their inclusion. I insist that the entire approach of choosing the same set of target library directories regardless of the freshness relationship between e.g. a system libstdc++ and the one we're building can't possibly be an overall improvement, it's only trading problems in some scenarios (where we're building an older libstdc++) for problems in other scenarios (where we're building a newer libstdc++). The latter is unfortunately far more likely, which is reason enough for the current arrangement, but libstdc++ problems will likely only hit if the gap between system and being-built libraries is large enough (say, new symbols in the newer libstdc++ used by the compiler, but not available in the system library). I'm really uncomfortable with this change, especially at this stage. I'd much rather have a relatively obscure workaround for this relatively obscure problem, while keeping the defaults that have accumulated lots of testing on lots of configurations. An idea that occurred to me is to have some configure option or just a make variable that would be prepended to RPATH_ENVVAR, so that it would preempt TARGET_LIB_PATH. That would be a far more conservative change, that I think we could make even at this stage. WDYT? -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer More tolerance and less prejudice are key for inclusion and diversity Excluding neuro-others for not behaving ""normal"" is *not* inclusive ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-20 5:45 ` Alexandre Oliva @ 2024-02-21 23:06 ` Jason Merrill 2024-02-21 23:36 ` Iain Sandoe 2024-02-23 11:23 ` Alexandre Oliva 0 siblings, 2 replies; 31+ messages in thread From: Jason Merrill @ 2024-02-21 23:06 UTC (permalink / raw) To: Alexandre Oliva; +Cc: Iain Sandoe, Paolo Bonzini, GCC Patches, Ian Lance Taylor [-- Attachment #1: Type: text/plain, Size: 3445 bytes --] On 2/20/24 00:45, Alexandre Oliva wrote: > On Feb 16, 2024, Jason Merrill <jason@redhat.com> wrote: > >> So, for stage2+, let's add just prev- libgcc. > > I'm pretty sure this will break bootstrap-lean where libgcc_s isn't a > system library, and we're building post-bootstrap host tools :-( > We need the current stage lib after the prev stage is removed. That's a good point, we should make sure it doesn't break. It looks to me like stage3-bubble removes stage1 after we're done building stage3, which should be fine, but compare removes the stage2 libgcc that we might still need to run stage3. So indeed I guess we still want both prev and current libgcc directories in RPATH to handle the case where we've removed the previous stage, as below. > I also doubt that TARGET_LIB_PATH was defined and used for no reason. > My hunch is that bootstrap options and/or targets that don't have these > libraries as system libraries will break in some obscure way without it. > But I don't have the bandwidth to track down the history behind their > inclusion. That has not seemed to be the case in Iain's testing on a system without these libraries as system libraries. I can't think of why we would need to depend on the current stage target libraries, and we already weren't depending on the previous stage target libraries. I believe the only target code we run is tests, and if the tests need the target libraries in RPATH that should happen in the testsuite. It's arguable that we should pass TARGET_LIB_PATH down to make it easier for the testsuites to find them, in case they are currently relying on them being part of RPATH. My impression from Iain's testing is that this isn't actually needed. I wouldn't mind keeping TARGET_LIB_PATH unused, but I'm not sure why that would be better than bringing it back if we turn out to need it. > I insist that the entire approach of choosing the same set of target > library directories regardless of the freshness relationship between > e.g. a system libstdc++ and the one we're building can't possibly be an > overall improvement, it's only trading problems in some scenarios (where > we're building an older libstdc++) for problems in other scenarios > (where we're building a newer libstdc++). The latter is unfortunately > far more likely, which is reason enough for the current arrangement, but > libstdc++ problems will likely only hit if the gap between system and > being-built libraries is large enough (say, new symbols in the newer > libstdc++ used by the compiler, but not available in the system > library). If bootstrap doesn't actually need the target libraries, as seems to be the case, then I think removing them from RPATH trades the former problem for no problem. > I'm really uncomfortable with this change, especially at this stage. > I'd much rather have a relatively obscure workaround for this relatively > obscure problem, while keeping the defaults that have accumulated lots > of testing on lots of configurations. I'm happy to defer this change to GCC 15 stage 1. > An idea that occurred to me is to have some configure option or just a > make variable that would be prepended to RPATH_ENVVAR, so that it would > preempt TARGET_LIB_PATH. That would be a far more conservative change, > that I think we could make even at this stage. WDYT? I'm hoping for a fix that doesn't require individual users to know about a workaround. Jason [-- Attachment #2: 0001-lean.patch --] [-- Type: text/x-patch, Size: 1271 bytes --] From 5312d534f1724c106feabebc9fdfa99e3717f859 Mon Sep 17 00:00:00 2001 From: Jason Merrill <jason@redhat.com> Date: Wed, 21 Feb 2024 23:03:36 +0000 Subject: [PATCH] lean To: gcc-patches@gcc.gnu.org ChangeLog: * Makefile.tpl: --- Makefile.tpl | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/Makefile.tpl b/Makefile.tpl index e3de2a7ba3b..674a6d7c711 100644 --- a/Makefile.tpl +++ b/Makefile.tpl @@ -650,10 +650,12 @@ HOST_LIB_PATH = [+ FOR host_modules +][+ IF lib_path +]$(HOST_LIB_PATH_[+module+])[+ ENDIF lib_path +][+ ENDFOR host_modules +]$(HOST_LIB_PATH_gcc) -# libgcc is a target library in the current stage and a host library -# in the previous stage (when bootstrapping). +# When bootstrapping, the previous stage libgcc is a host library. But with +# bootstrap-lean we might remove the previous libgcc and still want to run the +# current gcc that depends on it, so let's also include the current stage +# libgcc to accommodate that situation. @if gcc-bootstrap -HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR): +HOST_LIB_PATH_gcc = $$r/$(HOST_SUBDIR)/prev-gcc$(GCC_SHLIB_SUBDIR):$$r/$(HOST_SUBDIR)/gcc$(GCC_SHLIB_SUBDIR): @endif gcc-bootstrap [+ FOR host_modules +][+ IF lib_path +] -- 2.43.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-21 23:06 ` Jason Merrill @ 2024-02-21 23:36 ` Iain Sandoe 2024-02-23 9:09 ` Iain Sandoe 2024-02-23 11:23 ` Alexandre Oliva 1 sibling, 1 reply; 31+ messages in thread From: Iain Sandoe @ 2024-02-21 23:36 UTC (permalink / raw) To: Jason Merrill Cc: Alexandre Oliva, Paolo Bonzini, GCC Patches, Ian Lance Taylor > On 21 Feb 2024, at 23:06, Jason Merrill <jason@redhat.com> wrote: > > On 2/20/24 00:45, Alexandre Oliva wrote: >> On Feb 16, 2024, Jason Merrill <jason@redhat.com> wrote: >>> So, for stage2+, let's add just prev- libgcc. >> I'm pretty sure this will break bootstrap-lean where libgcc_s isn't a >> system library, and we're building post-bootstrap host tools :-( >> We need the current stage lib after the prev stage is removed. > > That's a good point, we should make sure it doesn't break. It looks to me like stage3-bubble removes stage1 after we're done building stage3, which should be fine, but compare removes the stage2 libgcc that we might still need to run stage3. So indeed I guess we still want both prev and current libgcc directories in RPATH to handle the case where we've removed the previous stage, as below. I’ll try that on darwin and aarch64 linux (I quite often need to use bootstrap-lean on the latter becuase of low disk space). >> I also doubt that TARGET_LIB_PATH was defined and used for no reason. >> My hunch is that bootstrap options and/or targets that don't have these >> libraries as system libraries will break in some obscure way without it. >> But I don't have the bandwidth to track down the history behind their >> inclusion. > > That has not seemed to be the case in Iain's testing on a system without these libraries as system libraries. Unless we change to (or add) a bootstrap where we use shared libstdc++ in the compiler, I think that is the case. As I mentioned in an earlier post, unfortunately we do not yet have a way to distinguish module builds for host from module builds for target (when a library is used for both - which is the case for libstdc++, libbacktrace and libgrust at least), This means that either the target library has to be built without a shared version (libbacktrace does this), or the host versions get built with a shared library which is not used (libstdc++) .. AFAICT the only reason we build libgomp and libatomic in bootstrap phase 1 and 2 is because they are dependents of the unused shared libstdc++. Ideally, we’d fix Makefile.{tpl,def} to allow the same module to have different recipies for host and target builds, but that’s also not a 5 minute hack…. > I can't think of why we would need to depend on the current stage target libraries, and we already weren't depending on the previous stage target libraries. I believe the only target code we run is tests, and if the tests need the target libraries in RPATH that should happen in the testsuite. Which could also be improved (we do not in Dejagnu really distinguish runpaths needed by the compiler from those needed by the built executables) > It's arguable that we should pass TARGET_LIB_PATH down to make it easier for the testsuites to find them, in case they are currently relying on them being part of RPATH. > My impression from Iain's testing is that this isn't actually needed. there’s actually a fair amount of specific code to locate dependent libs in places (some of which I just cleaned up a bit since it was now causing fails with Darwin’s new linker complaining about duplicated libs and so on). So we are not currently expecting this information to be passed down. > I wouldn't mind keeping TARGET_LIB_PATH unused, but I'm not sure why that would be better than bringing it back if we turn out to need it. +1 > > >> I insist that the entire approach of choosing the same set of target >> library directories regardless of the freshness relationship between >> e.g. a system libstdc++ and the one we're building can't possibly be an >> overall improvement, it's only trading problems in some scenarios (where >> we're building an older libstdc++) for problems in other scenarios >> (where we're building a newer libstdc++). The latter is unfortunately >> far more likely, which is reason enough for the current arrangement, but >> libstdc++ problems will likely only hit if the gap between system and >> being-built libraries is large enough (say, new symbols in the newer >> libstdc++ used by the compiler, but not available in the system >> library). > > If bootstrap doesn't actually need the target libraries, as seems to be the case, then I think removing them from RPATH trades the former problem for no problem. > >> I'm really uncomfortable with this change, especially at this stage. >> I'd much rather have a relatively obscure workaround for this relatively >> obscure problem, while keeping the defaults that have accumulated lots >> of testing on lots of configurations. > > I'm happy to defer this change to GCC 15 stage 1. > >> An idea that occurred to me is to have some configure option or just a >> make variable that would be prepended to RPATH_ENVVAR, so that it would >> preempt TARGET_LIB_PATH. That would be a far more conservative change, >> that I think we could make even at this stage. WDYT? > > I'm hoping for a fix that doesn't require individual users to know about a workaround. > > Jason<0001-lean.patch> Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-21 23:36 ` Iain Sandoe @ 2024-02-23 9:09 ` Iain Sandoe 0 siblings, 0 replies; 31+ messages in thread From: Iain Sandoe @ 2024-02-23 9:09 UTC (permalink / raw) To: Jason Merrill Cc: Alexandre Oliva, Paolo Bonzini, GCC Patches, Ian Lance Taylor > On 21 Feb 2024, at 23:36, Iain Sandoe <idsandoe@googlemail.com> wrote: > >> On 21 Feb 2024, at 23:06, Jason Merrill <jason@redhat.com> wrote: >> >> On 2/20/24 00:45, Alexandre Oliva wrote: >>> On Feb 16, 2024, Jason Merrill <jason@redhat.com> wrote: >>>> So, for stage2+, let's add just prev- libgcc. >>> I'm pretty sure this will break bootstrap-lean where libgcc_s isn't a >>> system library, and we're building post-bootstrap host tools :-( >>> We need the current stage lib after the prev stage is removed. >> >> That's a good point, we should make sure it doesn't break. It looks to me like stage3-bubble removes stage1 after we're done building stage3, which should be fine, but compare removes the stage2 libgcc that we might still need to run stage3. So indeed I guess we still want both prev and current libgcc directories in RPATH to handle the case where we've removed the previous stage, as below. > > I’ll try that on darwin and aarch64 linux (I quite often need to use bootstrap-lean on the latter becuase of low disk space) I tested this addition with bootstrap-lean on i686-darwin9 (needs libgcc_s to boostrap), x86_64-darwin21/23 (does not use libstdc++ in the system) .. aarch64-linux-gnu .. and it worked OK. Note that the testsuite does still have some glitches with bootstrap-lean (but those are independent of this patch). (we really need to adapt the idea of “host compiler/plugin compiler” to be the stage3 one when we bootstrap-lean). Iain > >>> I also doubt that TARGET_LIB_PATH was defined and used for no reason. >>> My hunch is that bootstrap options and/or targets that don't have these >>> libraries as system libraries will break in some obscure way without it. >>> But I don't have the bandwidth to track down the history behind their >>> inclusion. >> >> That has not seemed to be the case in Iain's testing on a system without these libraries as system libraries. > > Unless we change to (or add) a bootstrap where we use shared libstdc++ in the compiler, I think that is the case. > > As I mentioned in an earlier post, unfortunately we do not yet have a way to distinguish module builds for host from module builds for target (when a library is used for both - which is the case for libstdc++, libbacktrace and libgrust at least), This means that either the target library has to be built without a shared version (libbacktrace does this), or the host versions get built with a shared library which is not used (libstdc++) .. AFAICT the only reason we build libgomp and libatomic in bootstrap phase 1 and 2 is because they are dependents of the unused shared libstdc++. > > Ideally, we’d fix Makefile.{tpl,def} to allow the same module to have different recipies for host and target builds, but that’s also not a 5 minute hack…. > >> I can't think of why we would need to depend on the current stage target libraries, and we already weren't depending on the previous stage target libraries. I believe the only target code we run is tests, and if the tests need the target libraries in RPATH that should happen in the testsuite. > > Which could also be improved (we do not in Dejagnu really distinguish runpaths needed by the compiler from those needed by the built executables) > >> It's arguable that we should pass TARGET_LIB_PATH down to make it easier for the testsuites to find them, in case they are currently relying on them being part of RPATH. > >> My impression from Iain's testing is that this isn't actually needed. > > there’s actually a fair amount of specific code to locate dependent libs in places (some of which I just cleaned up a bit since it was now causing fails with Darwin’s new linker complaining about duplicated libs and so on). So we are not currently expecting this information to be passed down. > >> I wouldn't mind keeping TARGET_LIB_PATH unused, but I'm not sure why that would be better than bringing it back if we turn out to need it. > > +1 >> >> >>> I insist that the entire approach of choosing the same set of target >>> library directories regardless of the freshness relationship between >>> e.g. a system libstdc++ and the one we're building can't possibly be an >>> overall improvement, it's only trading problems in some scenarios (where >>> we're building an older libstdc++) for problems in other scenarios >>> (where we're building a newer libstdc++). The latter is unfortunately >>> far more likely, which is reason enough for the current arrangement, but >>> libstdc++ problems will likely only hit if the gap between system and >>> being-built libraries is large enough (say, new symbols in the newer >>> libstdc++ used by the compiler, but not available in the system >>> library). >> >> If bootstrap doesn't actually need the target libraries, as seems to be the case, then I think removing them from RPATH trades the former problem for no problem. >> >>> I'm really uncomfortable with this change, especially at this stage. >>> I'd much rather have a relatively obscure workaround for this relatively >>> obscure problem, while keeping the defaults that have accumulated lots >>> of testing on lots of configurations. >> >> I'm happy to defer this change to GCC 15 stage 1. >> >>> An idea that occurred to me is to have some configure option or just a >>> make variable that would be prepended to RPATH_ENVVAR, so that it would >>> preempt TARGET_LIB_PATH. That would be a far more conservative change, >>> that I think we could make even at this stage. WDYT? >> >> I'm hoping for a fix that doesn't require individual users to know about a workaround. >> >> Jason<0001-lean.patch> > > Iain ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-21 23:06 ` Jason Merrill 2024-02-21 23:36 ` Iain Sandoe @ 2024-02-23 11:23 ` Alexandre Oliva 2024-02-23 19:07 ` Jason Merrill 1 sibling, 1 reply; 31+ messages in thread From: Alexandre Oliva @ 2024-02-23 11:23 UTC (permalink / raw) To: Jason Merrill; +Cc: Iain Sandoe, Paolo Bonzini, GCC Patches, Ian Lance Taylor On Feb 21, 2024, Jason Merrill <jason@redhat.com> wrote: > So indeed I guess we still > want both prev and current libgcc directories in RPATH to handle the > case where we've removed the previous stage, as below. *nod*, thanks > I can't think of why we would need to depend on the current stage > target libraries, > If bootstrap doesn't actually need the target libraries. ISTM we may be miscommunicating. I'm not so worried about bootstrap itself as I am about post-bootstrap host tools. Those are unusual in that, after native bootstraps, they're built using the just-built (last-stage) compiler and target libraries, rather than the host compiler and system libraries. While configuring them, we need LD_LIBRARY_PATH (or similar) set up so that native execution tests can pass, at the very least; while building them, we may need LD_LIBRARY_PATH set up so that dependent libraries are found and link correctly. > I'm hoping for a fix that doesn't require individual users to know > about a workaround. I'm sure we'd all appreciate that, but AFAICT the conflicting requirements in scenarios of different freshness of libraries (system vs build) and both system tools and host tools dependencies on them would require a lot more intelligence in our build system to detect and react to the circumstances, deciding whether or not native tools used behind the scenes by the build will run with the just-built libraries, and whether or not the post-bootstrap host tools linked with the just-built libraries will run with the system libraries. There may even be scenarios in which these conflicting requirements would paint users into an inescapable corner AFAICT. The most worrying aspect is not libstdc++, but libgcc_s; it's needed as a dynamic library by languages that support exceptions, and depending on relative freshness of the toolchain being built and system libraries, newer symbols may be required by system programs and by just-built host tools. There doesn't seem to be an easy way around that. You have to either privilege one scenario over the other, like we do now, or introduce cleverness to detect and cope with such conflicting requirements, which we're not even trying to do. The approach in your patch changes the situation from privileging one scenario to privileging a mixed scenario (just-built libgcc but system libstdc++), and I'm pretty sure hunch that this mixed approach is likely to break down the road, even if not right now. -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer More tolerance and less prejudice are key for inclusion and diversity Excluding neuro-others for not behaving ""normal"" is *not* inclusive ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-23 11:23 ` Alexandre Oliva @ 2024-02-23 19:07 ` Jason Merrill 2024-02-24 12:19 ` Alexandre Oliva 0 siblings, 1 reply; 31+ messages in thread From: Jason Merrill @ 2024-02-23 19:07 UTC (permalink / raw) To: Alexandre Oliva; +Cc: Iain Sandoe, Paolo Bonzini, GCC Patches On 2/23/24 06:23, Alexandre Oliva wrote: > > I'm not so worried about bootstrap itself as I am about post-bootstrap > host tools. Those are unusual in that, after native bootstraps, they're > built using the just-built (last-stage) compiler and target libraries, > rather than the host compiler and system libraries. While configuring > them, we need LD_LIBRARY_PATH (or similar) set up so that native > execution tests can pass, at the very least; while building them, we may > need LD_LIBRARY_PATH set up so that dependent libraries are found and > link correctly. So, like gdb? Sure, if we built a stage3 gdb with the stage2 gcc and linked against the stage2 libstdc++.so, it would want to be able to find the stage2 libstdc++.so, or the stage3 in case of bootstrap-lean. The problem, as you say, comes when you want to both bootstrap and build tools that aren't involved in the bootstrap process. To support that perhaps we want POSTBOOTSTRAP_HOST_EXPORTS for host modules without the bootstrap tag, and add the TARGET_LIB_PATH directories there? Jason ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] 2024-02-23 19:07 ` Jason Merrill @ 2024-02-24 12:19 ` Alexandre Oliva 0 siblings, 0 replies; 31+ messages in thread From: Alexandre Oliva @ 2024-02-24 12:19 UTC (permalink / raw) To: Jason Merrill; +Cc: Iain Sandoe, Paolo Bonzini, GCC Patches On Feb 23, 2024, Jason Merrill <jason@redhat.com> wrote: > The problem, as you say, comes when you want to both bootstrap and > build tools that aren't involved in the bootstrap process. It's more visible there, because those don't actively refrain from linking dynamically with libstdc++. But even bootstrapped matter that involves exceptions would have to link in libgcc_s, and that would bring about the same sort of issue. > To support that perhaps we want POSTBOOTSTRAP_HOST_EXPORTS for host > modules without the bootstrap tag, and add the TARGET_LIB_PATH > directories there? That would be welcome, but it doesn't really address the problem, does it? Namely, the problem that we may face two different kinds of scenarios, and each one calls for an opposite solution. 1. system tools used in the build depend on system libraries that are newer than the ones we're about to build This is the scenario that you're attempting to address with these patches. The problem here is that the libraries being built are older than the system libraries, and and system tools won't run if dynamically linked with the older libraries about to be built. 2. the libraries we're about to build are newer than corresponding system libraries, if any This is the scenario that the current build system aims for. Any build tools that rely on older system libraries are likely to work just as well with the newly built libraries. Any newly built libraries linked into programs used and run as part of the build have to be present in LD_LIBRARY_PATH lest we end up trying to use the older system libraries, which may seem to work in some settings, but is bound to break if the differences are large enough. For maximum clarity, consider a bootstrap with LTO using a linker plugin. The linker plugin is built against the newly-built libraries. The linker that attempts to load the plugin also requires the same libraries. Do you see how tending to 1. breaks 2., and vice-versa? Now add ASAN to the picture, for another set of newly-built libraries used during bootstrap. Also use a prebuilt linker with ASAN enabled, for maximum clarity of the problem I'm getting at. Do you see the problem? Do you agree that patching the build system to solve a problem in scenario 1. *will* cause problems in scenario 2., *unless* the fix can distinguish the two scenarios and behave accordingly, but that getting that right is triky and error prone? Do you agree that, until we get there, it's probably better to optimize for the more common scenario? Do you agree that the more common scenario is probably 2.? Do you agree that, until we get a solution that works for both 1. and 2. automatically, offering a reasonably simple workaround for 1., while aiming to work for 2., would be a desirable stopgap? Do you agree that adding support for users to prepend directories to the search path, enabling them to preempt build libraries with (symlinks to?) select newer system libraries, and documenting situations in which this could be needed, is a reasonably simple and desirable stopgap that enables 1. to work while defaulting to the presumed more common case 2.? Here's a patchlet that shows the crux of what I have in mind: (nevermind we'd make the change elsewhere, document it further elsewhere, set an empty default and arrange to passed down to sub-$(MAKE)s) diff --git a/Makefile.in b/Makefile.in index edb0c8a9a427f..10c7646ef98c4 100644 --- a/Makefile.in +++ b/Makefile.in @@ -771,7 +771,12 @@ TARGET_LIB_PATH_libatomic = $$r/$(TARGET_SUBDIR)/libatomic/.libs: # This is the list of directories that may be needed in RPATH_ENVVAR # so that programs built for the host machine work. -HOST_LIB_PATH = $(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl) +# Users may set PREEMPT_HOST_LIB_PATH to a directory holding symlinks +# to system libraries required by build tools (say the linker) that +# are newer (as in higher-versioned) than the corresponding libraries +# we're building. If older libraries were to override the newer +# system libraries, that could prevent the build tools from running. +HOST_LIB_PATH = $(PREEMPT_HOST_LIB_PATH):$(HOST_LIB_PATH_gmp)$(HOST_LIB_PATH_mpfr)$(HOST_LIB_PATH_mpc)$(HOST_LIB_PATH_isl) # Define HOST_LIB_PATH_gcc here, for the sake of TARGET_LIB_PATH, ouch @if gcc Now, for a more general solution that doesn't require user intervention, configure could go about looking for system libraries in the default search path, or in RPATH_ENV_VAR, that share the soname with those we're about to build, identify preexisting libraries that are newer than those we're about to build, populate a build-tree directory with symlinks to them, and default PREEMPT_HOST_LIB_PATH to that directory. WDYT? -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer More tolerance and less prejudice are key for inclusion and diversity Excluding neuro-others for not behaving ""normal"" is *not* inclusive ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2024-02-24 12:19 UTC | newest] Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-02-07 1:54 [PATCH RFA] build: drop target libs from LD_LIBRARY_PATH [PR105688] Jason Merrill 2024-02-07 20:36 ` Ian Lance Taylor 2024-02-08 5:36 ` Alexandre Oliva 2024-02-08 15:04 ` Iain Sandoe 2024-02-08 17:12 ` Jason Merrill 2024-02-08 17:16 ` Jason Merrill 2024-02-08 17:51 ` Iain Sandoe 2024-02-08 19:25 ` Jason Merrill 2024-02-08 19:52 ` Iain Sandoe 2024-02-22 18:06 ` Gaius Mulley 2024-02-22 20:30 ` Iain Sandoe 2024-02-24 10:23 ` Gaius Mulley 2024-02-08 17:55 ` Paolo Bonzini 2024-02-08 21:44 ` Jason Merrill 2024-02-09 10:56 ` Iain Sandoe 2024-02-09 23:21 ` Iain Sandoe 2024-02-10 10:46 ` Iain Sandoe 2024-02-10 12:07 ` Jason Merrill 2024-02-10 12:30 ` Iain Sandoe 2024-02-12 19:59 ` Jason Merrill 2024-02-14 22:59 ` Iain Sandoe 2024-02-14 23:33 ` Iain Sandoe 2024-02-16 21:05 ` Jason Merrill 2024-02-19 20:08 ` Iain Sandoe 2024-02-20 5:45 ` Alexandre Oliva 2024-02-21 23:06 ` Jason Merrill 2024-02-21 23:36 ` Iain Sandoe 2024-02-23 9:09 ` Iain Sandoe 2024-02-23 11:23 ` Alexandre Oliva 2024-02-23 19:07 ` Jason Merrill 2024-02-24 12:19 ` Alexandre Oliva
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).