[PATCH] configury: --enable-link-serialization support

[PATCH] configury: --enable-link-serialization support

[Jakub Jelinek]

Hi!

Reposting with self-contained description per Joseph's request:

When performing LTO bootstraps, especially when using tmpfs for /tmp,
one can run a machine to halt when using higher levels of parallelism
and a large number of FEs, because there are too many concurrent LTO
link commands running at the same time and each one of them puts most of the
middle-end/backend objects into /tmp.

We have --enable-link-mutex configure option, but --enable-link-mutex has
a big problem that it decreases number of available jobs by the number of
link commands waiting for the lock, so e.g. when doing make -j32 build with
11 different big programs linked with $(LLINKER) we end up with just 22
effective jobs, and with e.g. make -j8 with those 11 different big programs
we actually most likely serialize everything during linking onto a single job.

The following patch implements a new configure option,
--enable-link-serialization, which implements different serialization and
as it doesn't use the mutex, just modifying the old option to be implemented
differently would be strange.  We can deprecate and later remove the old
option.  The new option doesn't use any shell mutexes, but uses make
dependencies.

The option is implemented inside of gcc/ configure and Makefiles,
which means that even inside of gcc/ make all (as well as e.g. make lto-dump)
will serialize and build all previous large binaries when configured this
way.
One can always make -j32 cc1 DO_LINK_SERIALIZATION=
to avoid that.
Furthermore, I've implemented the idea I wrote about, so that
--enable-link-serialization
is the same as
--enable-link-serialization=1
and means the large link commands are serialized, one can (the default)
--disable-link-serialization
which will cause all links to be parallelizable, but one can also
--enable-link-serialization=3
etc. which says that at most 3 of the large link commands can run
concurrently.
And finally I've implemented (only if the serialization is enabled) simple
progress bars for the linking.
With --enable-link-serialization and e.g. the 5 large links I have in my
current tree (cc1, cc1plus, f951, lto1 and lto-dump), before the linking it
prints
Linking |==--      | 20%
and after it
Linking |====      | 40%
(each == characters stand for already finished links, each --
characters stand for the link being started).
With --enable-link-serialization=3 it will change the way the start is
printed, one will get:
Linking |--        | 0%
at the start of cc1 link,
Linking |>>--      | 0%
at the start of the second large link and
Linking |>>>>--    | 0%
at the start of the third large link, where the >> characters stand for
already pending links.  The printing at the end of link command is
the same as with the full serialization, i.e. for the above 3:
Linking |==        | 20%
Linking |====      | 40%
Linking |======    | 60%
but one could actually get them in any order depending on which of those 3
finishes first - to get it 100% accurate I'd need to add some directory with
files representing finished links or similar, doesn't seem worth it.

Bootstrapped/regtested again last night on x86_64-linux and i686-linux, ok
for trunk?

2020-11-18  Jakub Jelinek  <jakub@redhat.com>

gcc/
	* configure.ac: Add $lang.prev rules, INDEX.$lang and SERIAL_LIST and
	SERIAL_COUNT variables to Make-hooks.
	(--enable-link-serialization): New configure option.
	* Makefile.in (DO_LINK_SERIALIZATION, LINK_PROGRESS): New variables.
	* doc/install.texi (--enable-link-serialization): Document.
	* configure: Regenerated.
gcc/c/
	* Make-lang.in (c.serial): New goal.
	(.PHONY): Add c.serial c.prev.
	(cc1$(exeext)): Call LINK_PROGRESS.
gcc/cp/
	* Make-lang.in (c++.serial): New goal.
	(.PHONY): Add c++.serial c++.prev.
	(cc1plus$(exeext)): Depend on c++.prev.  Call LINK_PROGRESS.
gcc/fortran/
	* Make-lang.in (fortran.serial): New goal.
	(.PHONY): Add fortran.serial fortran.prev.
	(f951$(exeext)): Depend on fortran.prev.  Call LINK_PROGRESS.
gcc/lto/
	* Make-lang.in (lto, lto1.serial, lto2.serial): New goals.
	(.PHONY): Add lto lto1.serial lto1.prev lto2.serial lto2.prev.
	(lto.all.cross, lto.start.encap): Remove dependencies.
	($(LTO_EXE)): Depend on lto1.prev.  Call LINK_PROGRESS.
	($(LTO_DUMP_EXE)): Depend on lto2.prev.  Call LINK_PROGRESS.
gcc/objc/
	* Make-lang.in (objc.serial): New goal.
	(.PHONY): Add objc.serial objc.prev.
	(cc1obj$(exeext)): Depend on objc.prev.  Call LINK_PROGRESS.
gcc/objcp/
	* Make-lang.in (obj-c++.serial): New goal.
	(.PHONY): Add obj-c++.serial obj-c++.prev.
	(cc1objplus$(exeext)): Depend on obj-c++.prev.  Call LINK_PROGRESS.
gcc/ada/
	* gcc-interface/Make-lang.in (ada.serial): New goal.
	(.PHONY): Add ada.serial ada.prev.
	(gnat1$(exeext)): Depend on ada.prev.  Call LINK_PROGRESS.
gcc/brig/
	* Make-lang.in (brig.serial): New goal.
	(.PHONY): Add brig.serial brig.prev.
	(brig1$(exeext)): Depend on brig.prev.  Call LINK_PROGRESS.
gcc/go/
	* Make-lang.in (go.serial): New goal.
	(.PHONY): Add go.serial go.prev.
	(go1$(exeext)): Depend on go.prev.  Call LINK_PROGRESS.
gcc/jit/
	* Make-lang.in (jit.serial): New goal.
	(.PHONY): Add jit.serial jit.prev.
	($(LIBGCCJIT_FILENAME)): Depend on jit.prev.  Call LINK_PROGRESS.
gcc/d/
	* Make-lang.in (d.serial): New goal.
	(.PHONY): Add d.serial d.prev.
	(d21$(exeext)): Depend on d.prev.  Call LINK_PROGRESS.
	
--- gcc/configure.ac.jj	2020-09-08 12:24:39.199542406 +0200
+++ gcc/configure.ac	2020-09-08 13:00:48.788408222 +0200
@@ -6666,6 +6666,29 @@ else
 fi
 AC_SUBST(DO_LINK_MUTEX)
 
+dnl Whether to prevent multiple GCC front-ends from linking at the same time
+
+AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
+  AC_ARG_ENABLE(link-serialization,
+[AS_HELP_STRING([--enable-link-serialization],
+		[avoid linking multiple GCC front-ends at once using make
+		 dependencies to avoid thrashing on the build machine])],
+      do_link_serialization=$enableval,
+      do_link_serialization=no)
+AC_MSG_RESULT($do_link_serialization)
+
+case "$do_link_serialization" in
+  yes)
+    DO_LINK_SERIALIZATION=1;;
+  [[1-9]] | [[1-9]][[0-9]] | [[1-9]][[0-9]][[0-9]])
+    DO_LINK_SERIALIZATION=$do_link_serialization;;
+  no)
+    DO_LINK_SERIALIZATION=;;
+  *)
+    AC_MSG_ERROR(bad value ${do_link_serialization} given for --enable-link-serialization) ;;
+esac
+AC_SUBST(DO_LINK_SERIALIZATION)
+
 # --------------
 # Language hooks
 # --------------
@@ -6826,6 +6849,43 @@ do
 	echo "lang.$t: $x" >> Make-hooks
 done
 
+echo "ifeq (\$(DO_LINK_SERIALIZATION),)" >> Make-hooks
+echo "SERIAL_LIST =" >> Make-hooks
+echo else >> Make-hooks
+lang_cnt=0
+lang_list=
+prev=c.serial
+serialization_languages=c
+for lang in $all_selected_languages
+do
+	test $lang = c && continue
+	if test $lang = lto; then
+		serialization_languages="$serialization_languages lto1 lto2"
+	else
+		serialization_languages="$serialization_languages $lang"
+	fi
+done
+for lang in $serialization_languages
+do
+	test $lang = c && continue
+	lang_cnt=`expr $lang_cnt + 1`
+	lang_list=" $prev$lang_list"
+	prev=${lang}.serial
+done
+echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
+echo endif >> Make-hooks
+echo "SERIAL_COUNT = `expr $lang_cnt + 1`" >> Make-hooks
+echo "INDEX.c = 0" >> Make-hooks
+lang_idx=1
+for lang in $serialization_languages
+do
+	test $lang = c && continue
+	echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+	echo "INDEX.$lang = $lang_idx" >> Make-hooks
+	lang_cnt=`expr $lang_cnt - 1`
+	lang_idx=`expr $lang_idx + 1`
+done
+
 # --------
 # Option include files
 # --------
--- gcc/Makefile.in.jj	2020-09-03 20:10:13.572996335 +0200
+++ gcc/Makefile.in	2020-09-08 14:04:39.765714704 +0200
@@ -1739,6 +1739,8 @@ $(FULL_DRIVER_NAME): ./xgcc$(exeext)
 # Otherwise $(SELFTEST_DEPS) is empty when used from <LANG>/Make-lang.in.
 SELFTEST_DEPS = $(GCC_PASSES) stmp-int-hdrs $(srcdir)/testsuite/selftests
 
+DO_LINK_SERIALIZATION = @DO_LINK_SERIALIZATION@
+
 # Language makefile fragments.
 
 # The following targets define the interface between us and the languages.
@@ -1756,6 +1758,23 @@ SELFTEST_DEPS = $(GCC_PASSES) stmp-int-h
 # language hooks, generated by configure
 @language_hooks@
 
+ifeq ($(DO_LINK_SERIALIZATION),)
+LINK_PROGRESS = :
+else
+LINK_PROGRESS = msg="Linking |"; cnt=0; if test "$(2)" = start; then \
+  idx=0; cnt2=$(DO_LINK_SERIALIZATION); \
+  while test $$cnt2 -le $(1); do msg="$${msg}=="; cnt2=`expr $$cnt2 + 1`; idx=`expr $$idx + 1`; done; \
+  cnt=$$idx; \
+  while test $$cnt -lt $(1); do msg="$${msg}>>"; cnt=`expr $$cnt + 1`; done; \
+  msg="$${msg}--"; cnt=`expr $$cnt + 1`; \
+  else \
+  idx=`expr $(1) + 1`; \
+  while test $$cnt -le $(1); do msg="$${msg}=="; cnt=`expr $$cnt + 1`; done; \
+  fi; \
+  while test $$cnt -lt $(SERIAL_COUNT); do msg="$${msg}  "; cnt=`expr $$cnt + 1`; done; \
+  msg="$${msg}| `expr 100 \* $$idx / $(SERIAL_COUNT)`%"; echo "$$msg"
+endif
+
 # Wire in install-gnatlib invocation with `make install' for a configuration
 # with top-level libada disabled.
 gnat_install_lib = @gnat_install_lib@
--- gcc/c/Make-lang.in.jj	2020-09-03 20:10:13.608995812 +0200
+++ gcc/c/Make-lang.in	2020-09-08 13:35:55.907221237 +0200
@@ -37,9 +37,10 @@
 #\f
 # Define the names for selecting c in LANGUAGES.
 c: cc1$(exeext)
+c.serial: c
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: c gcc
+.PHONY: c gcc c.serial
 
 # The C front end driver.  This is different from the drivers for other
 # front ends, because there is no C language specific driver (i.e. nothing
@@ -82,8 +83,10 @@ cc1-checksum.c : build/genchecksum$(buil
 	fi
 
 cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS)
+	@$(call LINK_PROGRESS,$(INDEX.c),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \
 	  cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.c),end)
 
 cc1.fda: ../stage1-gcc/cc1$(exeext) ../prev-gcc/$(PERF_DATA)
 	$(CREATE_GCOV) -binary ../stage1-gcc/cc1$(exeext) -gcov cc1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
--- gcc/cp/Make-lang.in.jj	2020-09-03 20:10:13.683994722 +0200
+++ gcc/cp/Make-lang.in	2020-09-08 13:35:15.793814755 +0200
@@ -47,9 +47,10 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
 # into the C++ rule, but that needs a little bit of work
 # to do the right thing within all.cross.
 c++: cc1plus$(exeext)
+c++.serial: c++
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: c++
+.PHONY: c++ c++.serial c++.prev
 
 CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
 
@@ -116,9 +117,11 @@ cc1plus-checksum.c : build/genchecksum$(
 	  $(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
 	fi
 
-cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) c++.prev
+	@$(call LINK_PROGRESS,$(INDEX.c++),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.c++),end)
 
 ifeq ($(ENABLE_MAINTAINER_RULES), true)
 # Special build rule.  This is a maintainer rule, that is only
--- gcc/fortran/Make-lang.in.jj	2020-09-03 20:10:13.737993937 +0200
+++ gcc/fortran/Make-lang.in	2020-09-08 13:36:53.382370858 +0200
@@ -72,9 +72,10 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
 #\f
 # Define the names for selecting gfortran in LANGUAGES.
 fortran: f951$(exeext)
+fortran.serial: fortran
 
 # Tell GNU make to ignore files by these names if they exist.
-.PHONY: fortran
+.PHONY: fortran fortran.serial fortran.prev
 
 CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
 
@@ -92,11 +93,12 @@ gfortran-cross$(exeext): gfortran$(exeex
 	cp gfortran$(exeext) gfortran-cross$(exeext)
 
 # The compiler itself is called f951.
-f951$(exeext): $(F95_OBJS) \
-		$(BACKEND) $(LIBDEPS) attribs.o
+f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o fortran.prev
+	@$(call LINK_PROGRESS,$(INDEX.fortran),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
 		$(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.fortran),end)
 
 gt-fortran-trans.h    : s-gtype; @true
 #\f
--- gcc/lto/Make-lang.in.jj	2020-09-03 20:10:13.759993617 +0200
+++ gcc/lto/Make-lang.in	2020-09-08 13:37:15.038050442 +0200
@@ -41,10 +41,16 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
 
 # Rules
 
+lto: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto1.serial: $(LTO_EXE)
+lto2.serial: $(LTO_DUMP_EXE)
+
+.PHONY: lto lto1.serial lto1.prev lto2.serial lto2.prev
+
 # These hooks are used by the main GCC Makefile.  Consult that
 # Makefile for documentation.
-lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
-lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto.all.cross:
+lto.start.encap:
 lto.rest.encap:
 lto.tags:
 lto.install-common: installdirs
@@ -84,13 +90,17 @@ lto.stagefeedback:
 # Use strict warnings for this front end.
 lto-warn = $(STRICT_WARN)
 
-$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) lto1.prev
+	@$(call LINK_PROGRESS,$(INDEX.lto1),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
+	@$(call LINK_PROGRESS,$(INDEX.lto1),end)
 
-$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) lto2.prev
+	@$(call LINK_PROGRESS,$(INDEX.lto2),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
+	@$(call LINK_PROGRESS,$(INDEX.lto2),end)
 
 lto/lto-dump.o: $(LTO_OBJS)
 
--- gcc/objc/Make-lang.in.jj	2020-09-03 20:10:13.764993545 +0200
+++ gcc/objc/Make-lang.in	2020-09-08 13:36:29.303727116 +0200
@@ -38,9 +38,10 @@
 #\f
 # Define the names for selecting Objective-C in LANGUAGES.
 objc: cc1obj$(exeext)
+objc.serial: objc
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: objc
+.PHONY: objc objc.serial objc.prev
 
 # Use maximal warnings for this front end.
 objc-warn = $(STRICT_WARN)
@@ -62,10 +63,13 @@ cc1obj-checksum.c : build/genchecksum$(b
         $(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
 	$(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
 
-cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
+cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \
+		 $(LIBDEPS) objc.prev
+	@$(call LINK_PROGRESS,$(INDEX.objc),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
 	      $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.objc),end)
 
 objc.srcextra:
 
--- gcc/objcp/Make-lang.in.jj	2020-09-03 20:10:13.771993443 +0200
+++ gcc/objcp/Make-lang.in	2020-09-08 13:36:40.822556689 +0200
@@ -39,9 +39,10 @@
 #\f
 # Define the names for selecting Objective-C++ in LANGUAGES.
 obj-c++: cc1objplus$(exeext)
+obj-c++.serial: obj-c++
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: obj-c++
+.PHONY: obj-c++ obj-c++.serial obj-c++.prev
 
 # Use maximal warnings for this front end.  Also, make ObjC and C++
 # headers accessible.
@@ -66,9 +67,12 @@ cc1objplus-checksum.c : build/genchecksu
 	$(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
 	cc1objplus-checksum.c
 
-cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) \
+		     $(LIBDEPS) obj-c++.prev
+	@$(call LINK_PROGRESS,$(INDEX.obj-c++),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.obj-c++),end)
 
 # Objective C++ language specific files.
 
--- gcc/ada/gcc-interface/Make-lang.in.jj	2020-09-03 20:10:13.592996044 +0200
+++ gcc/ada/gcc-interface/Make-lang.in	2020-09-08 13:35:25.692668295 +0200
@@ -146,9 +146,10 @@ endif
 
 # Define the names for selecting Ada in LANGUAGES.
 ada: gnat1$(exeext) gnatbind$(exeext)
+ada.serial: gnat1$(exeext)
 
 # Tell GNU Make to ignore these, if they exist.
-.PHONY: ada
+.PHONY: ada ada.serial ada.prev
 
 # Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
 # we're linking against at least libcommon which contains C++ compiled code.
@@ -666,10 +667,13 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
 # Needs to be built with CC=gcc
 # Since the RTL should be built with the latest compiler, remove the
 #  stamp target in the parent directory whenever gnat1 is rebuilt
-gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
+gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a \
+		$(LIBDEPS) ada.prev
+	@$(call LINK_PROGRESS,$(INDEX.ada),start)
 	+$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
 	  libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
 	$(RM) stamp-gnatlib2-rts stamp-tools
+	@$(call LINK_PROGRESS,$(INDEX.ada),end)
 
 gnatbind$(exeext): ada/b_gnatb.o $(CONFIG_H) $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBDEPS)
 	+$(GCC_LINK) -o $@ ada/b_gnatb.o $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBS) $(SYSLIBS) $(CFLAGS)
--- gcc/brig/Make-lang.in.jj	2020-09-03 20:10:13.596995986 +0200
+++ gcc/brig/Make-lang.in	2020-09-08 13:35:37.111499344 +0200
@@ -29,8 +29,9 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
 
 # The name for selecting brig in LANGUAGES.
 brig: brig1$(exeext)
+brig.serial: brig
 
-.PHONY: brig
+.PHONY: brig brig.serial brig.prev
 
 CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
 
@@ -81,15 +82,12 @@ BRIG_OBJS = \
 
 brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
 
-# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
-# 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
-# 	      $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
-
-
-brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) brig.prev
+	@$(call LINK_PROGRESS,$(INDEX.brig),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
 		  $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.brig),end)
 
 # Documentation.
 
--- gcc/go/Make-lang.in.jj	2020-09-03 20:10:13.737993937 +0200
+++ gcc/go/Make-lang.in	2020-09-08 13:35:47.201350056 +0200
@@ -27,8 +27,9 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
 
 # The name for selecting go in LANGUAGES.
 go: go1$(exeext)
+go.serial: go
 
-.PHONY: go
+.PHONY: go go.serial go.prev
 
 CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
 
@@ -78,9 +79,11 @@ GO_OBJS = \
 
 go_OBJS = $(GO_OBJS) go/gospec.o
 
-go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) go.prev
+	@$(call LINK_PROGRESS,$(INDEX.go),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.go),end)
 
 # Documentation.
 
--- gcc/jit/Make-lang.in.jj	2020-09-03 20:10:13.750993749 +0200
+++ gcc/jit/Make-lang.in	2020-09-08 13:36:12.289978848 +0200
@@ -81,8 +81,10 @@ jit: $(LIBGCCJIT_FILENAME) \
 	$(FULL_DRIVER_NAME)
 endif
 
+jit.serial: $(LIBGCCJIT_FILENAME)
+
 # Tell GNU make to ignore these if they exist.
-.PHONY: jit
+.PHONY: jit jit.serial jit.prev
 
 jit_OBJS = attribs.o \
 	jit/dummy-frontend.o \
@@ -117,12 +119,14 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
 	libbackend.a libcommon-target.a libcommon.a \
 	$(CPPLIB) $(LIBDECNUMBER) \
 	$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
-	$(EXTRA_GCC_OBJS)
+	$(EXTRA_GCC_OBJS) jit.prev
+	@$(call LINK_PROGRESS,$(INDEX.jit),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
 	     $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
 	     $(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
 	     $(EXTRA_GCC_OBJS) \
 	     $(LIBGCCJIT_EXTRA_OPTS)
+	@$(call LINK_PROGRESS,$(INDEX.jit),end)
 
 # Create symlinks when not building for Windows
 ifeq (,$(findstring mingw,$(target)))
--- gcc/d/Make-lang.in.jj	2020-09-03 20:10:13.696994533 +0200
+++ gcc/d/Make-lang.in	2020-09-08 13:36:21.474842957 +0200
@@ -27,9 +27,10 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
 
 # The name for selecting d in LANGUAGES.
 d: d21$(exeext)
+d.serial: d
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: d
+.PHONY: d d.serial d.prev
 
 # Create the compiler driver for D.
 CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
@@ -162,9 +163,11 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
 
 d_OBJS = $(D_ALL_OBJS) d/d-spec.o
 
-d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) d.prev
+	@$(call LINK_PROGRESS,$(INDEX.d),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+	@$(call LINK_PROGRESS,$(INDEX.d),end)
 
 # Documentation.
 
--- gcc/doc/install.texi.jj	2020-09-03 20:10:13.719994200 +0200
+++ gcc/doc/install.texi	2020-09-08 13:13:16.934334012 +0200
@@ -1533,6 +1533,14 @@ When building GCC, use a mutex to avoid
 multiple languages at the same time, to avoid thrashing on build
 systems with limited free memory.  The default is not to use such a mutex.
 
+@item --enable-link-serialization
+When building GCC, use make dependencies to serialize linking the compilers for
+multiple languages, to avoid thrashing on build
+systems with limited free memory.  The default is not to add such
+dependencies and thus with parallel make potentially link different
+compilers concurrently.  If the argument is a positive integer, allow
+that number of concurrent link processes for the large binaries.
+
 @item --enable-maintainer-mode
 The build rules that regenerate the Autoconf and Automake output files as
 well as the GCC master message catalog @file{gcc.pot} are normally
--- gcc/configure.jj	2020-09-03 20:10:13.657995099 +0200
+++ gcc/configure	2020-09-08 13:00:52.702350290 +0200
@@ -710,6 +710,7 @@ subdirs
 dollar
 gcc_tooldir
 enable_lto
+DO_LINK_SERIALIZATION
 DO_LINK_MUTEX
 MAINT
 zlibinc
@@ -1013,6 +1014,7 @@ with_gc
 with_system_zlib
 enable_maintainer_mode
 enable_link_mutex
+enable_link_serialization
 enable_version_specific_runtime_libs
 enable_plugin
 enable_host_shared
@@ -1768,6 +1770,10 @@ Optional Features:
                           sometimes confusing) to the casual installer
   --enable-link-mutex     avoid linking multiple front-ends at once to avoid
                           thrashing on the build machine
+  --enable-link-serialization
+                          avoid linking multiple GCC front-ends at once using
+                          make dependencies to avoid thrashing on the build
+                          machine
   --enable-version-specific-runtime-libs
                           specify that runtime libraries should be installed
                           in a compiler-specific directory
@@ -19030,7 +19036,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 19033 "configure"
+#line 19039 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -19136,7 +19142,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 19139 "configure"
+#line 19145 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -30511,6 +30517,31 @@ else
 fi
 
 
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
+$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
+  # Check whether --enable-link-serialization was given.
+if test "${enable_link_serialization+set}" = set; then :
+  enableval=$enable_link_serialization; do_link_serialization=$enableval
+else
+  do_link_serialization=no
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
+$as_echo "$do_link_serialization" >&6; }
+
+case "$do_link_serialization" in
+  yes)
+    DO_LINK_SERIALIZATION=1;;
+  [1-9] | [1-9][0-9] | [1-9][0-9][0-9])
+    DO_LINK_SERIALIZATION=$do_link_serialization;;
+  no)
+    DO_LINK_SERIALIZATION=;;
+  *)
+    as_fn_error $? "bad value ${do_link_serialization} given for --enable-link-serialization" "$LINENO" 5 ;;
+esac
+
+
 # --------------
 # Language hooks
 # --------------
@@ -30671,6 +30702,43 @@ do
 	echo "lang.$t: $x" >> Make-hooks
 done
 
+echo "ifeq (\$(DO_LINK_SERIALIZATION),)" >> Make-hooks
+echo "SERIAL_LIST =" >> Make-hooks
+echo else >> Make-hooks
+lang_cnt=0
+lang_list=
+prev=c.serial
+serialization_languages=c
+for lang in $all_selected_languages
+do
+	test $lang = c && continue
+	if test $lang = lto; then
+		serialization_languages="$serialization_languages lto1 lto2"
+	else
+		serialization_languages="$serialization_languages $lang"
+	fi
+done
+for lang in $serialization_languages
+do
+	test $lang = c && continue
+	lang_cnt=`expr $lang_cnt + 1`
+	lang_list=" $prev$lang_list"
+	prev=${lang}.serial
+done
+echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
+echo endif >> Make-hooks
+echo "SERIAL_COUNT = `expr $lang_cnt + 1`" >> Make-hooks
+echo "INDEX.c = 0" >> Make-hooks
+lang_idx=1
+for lang in $serialization_languages
+do
+	test $lang = c && continue
+	echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+	echo "INDEX.$lang = $lang_idx" >> Make-hooks
+	lang_cnt=`expr $lang_cnt - 1`
+	lang_idx=`expr $lang_idx + 1`
+done
+
 # --------
 # Option include files
 # --------

	Jakub

Re: [PATCH] configury: --enable-link-serialization support

[Joseph Myers]

On Wed, 18 Nov 2020, Jakub Jelinek via Gcc-patches wrote:

> Bootstrapped/regtested again last night on x86_64-linux and i686-linux, ok
> for trunk?

OK.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] configury: --enable-link-serialization support

[Eric Botcazou]

> 2020-11-18  Jakub Jelinek  <jakub@redhat.com>
> 
> gcc/
> 	* configure.ac: Add $lang.prev rules, INDEX.$lang and SERIAL_LIST 
and
> 	SERIAL_COUNT variables to Make-hooks.
> 	(--enable-link-serialization): New configure option.
> 	* Makefile.in (DO_LINK_SERIALIZATION, LINK_PROGRESS): New 
variables.
> 	* doc/install.texi (--enable-link-serialization): Document.
> 	* configure: Regenerated.
> gcc/c/
> 	* Make-lang.in (c.serial): New goal.
> 	(.PHONY): Add c.serial c.prev.
> 	(cc1$(exeext)): Call LINK_PROGRESS.
> gcc/cp/
> 	* Make-lang.in (c++.serial): New goal.
> 	(.PHONY): Add c++.serial c++.prev.
> 	(cc1plus$(exeext)): Depend on c++.prev.  Call LINK_PROGRESS.
> gcc/fortran/
> 	* Make-lang.in (fortran.serial): New goal.
> 	(.PHONY): Add fortran.serial fortran.prev.
> 	(f951$(exeext)): Depend on fortran.prev.  Call LINK_PROGRESS.
> gcc/lto/
> 	* Make-lang.in (lto, lto1.serial, lto2.serial): New goals.
> 	(.PHONY): Add lto lto1.serial lto1.prev lto2.serial lto2.prev.
> 	(lto.all.cross, lto.start.encap): Remove dependencies.
> 	($(LTO_EXE)): Depend on lto1.prev.  Call LINK_PROGRESS.
> 	($(LTO_DUMP_EXE)): Depend on lto2.prev.  Call LINK_PROGRESS.
> gcc/objc/
> 	* Make-lang.in (objc.serial): New goal.
> 	(.PHONY): Add objc.serial objc.prev.
> 	(cc1obj$(exeext)): Depend on objc.prev.  Call LINK_PROGRESS.
> gcc/objcp/
> 	* Make-lang.in (obj-c++.serial): New goal.
> 	(.PHONY): Add obj-c++.serial obj-c++.prev.
> 	(cc1objplus$(exeext)): Depend on obj-c++.prev.  Call LINK_PROGRESS.
> gcc/ada/
> 	* gcc-interface/Make-lang.in (ada.serial): New goal.
> 	(.PHONY): Add ada.serial ada.prev.
> 	(gnat1$(exeext)): Depend on ada.prev.  Call LINK_PROGRESS.
> gcc/brig/
> 	* Make-lang.in (brig.serial): New goal.
> 	(.PHONY): Add brig.serial brig.prev.
> 	(brig1$(exeext)): Depend on brig.prev.  Call LINK_PROGRESS.
> gcc/go/
> 	* Make-lang.in (go.serial): New goal.
> 	(.PHONY): Add go.serial go.prev.
> 	(go1$(exeext)): Depend on go.prev.  Call LINK_PROGRESS.
> gcc/jit/
> 	* Make-lang.in (jit.serial): New goal.
> 	(.PHONY): Add jit.serial jit.prev.
> 	($(LIBGCCJIT_FILENAME)): Depend on jit.prev.  Call LINK_PROGRESS.
> gcc/d/
> 	* Make-lang.in (d.serial): New goal.
> 	(.PHONY): Add d.serial d.prev.
> 	(d21$(exeext)): Depend on d.prev.  Call LINK_PROGRESS.

This seems to cause the binaries to be always relinked, for example from the 
gcc/ directory of the build tree:

make
[relink of gnat1, brig1, cc1plus, d21, f951, go1, lto1, ...]
make
[relink of gnat1, brig1, cc1plus, d21, f951, go1, lto1, ...]

-- 
Eric Botcazou

Re: [PATCH] configury: --enable-link-serialization support

[Jakub Jelinek]

On Thu, Nov 19, 2020 at 12:55:42PM +0100, Eric Botcazou wrote:
> > 	* Make-lang.in (d.serial): New goal.
> > 	(.PHONY): Add d.serial d.prev.
> > 	(d21$(exeext)): Depend on d.prev.  Call LINK_PROGRESS.
> 
> This seems to cause the binaries to be always relinked, for example from the 
> gcc/ directory of the build tree:
> 
> make
> [relink of gnat1, brig1, cc1plus, d21, f951, go1, lto1, ...]
> make
> [relink of gnat1, brig1, cc1plus, d21, f951, go1, lto1, ...]

Does it?
Only with --enable-link-serialization, or without that too?
Will have a look.

	Jakub

Re: [PATCH] configury: --enable-link-serialization support

[Eric Botcazou]

> Does it?
> Only with --enable-link-serialization, or without that too?

It's the default, i.e. without --enable-link-serialization.

> Will have a look.

Thanks.

-- 
Eric Botcazou

Re: [PATCH] configury: --enable-link-serialization support

[Martin Liška]

On 11/19/20 1:10 PM, Eric Botcazou wrote:
>> Does it?
>> Only with --enable-link-serialization, or without that too?
> 
> It's the default, i.e. without --enable-link-serialization.
> 
>> Will have a look.
> 
> Thanks.
> 

I see the same, I don't use --enable-link-serialization.
Just run:
make -j16 && make -j16

Martin

Re: [PATCH] configury: --enable-link-serialization support

[Jakub Jelinek]

On Thu, Nov 19, 2020 at 01:10:18PM +0100, Eric Botcazou wrote:
> > Does it?
> > Only with --enable-link-serialization, or without that too?
> 
> It's the default, i.e. without --enable-link-serialization.
> 
> > Will have a look.
> 
> Thanks.

So, I think the problem is that for make .PHONY targets are just
"rebuilt" always, so it is very much undesirable for the cc1plus$(exeext)
etc. dependencies to include .PHONY targets, but I was using
them - cc1plus.prev which would depend on some *.serial and
e.g. cc1.serial depending on c and c depending on cc1$(exeext).

The following so far only very lightly tested patch rewrites this
so that *.serial and *.prev aren't .PHONY targets, but instead just
make variables.

I was worried that the order in which the language makefile fragments are
included (which is quite random, what order we get from the filesystem
matching */config-lang.in) would be a problem but it seems to work fine.

2020-11-19  Jakub Jelinek  <jakub@redhat.com>

gcc/
	* configure.ac: In SERIAL_LIST use lang words without .serial
	suffix.  Change $lang.prev from a target to variable and instead
	of depending on *.serial expand to the *.serial variable if
	the word is in the SERIAL_LIST at all, otherwise to nothing.
	* configure: Regenerated.
gcc/c/
	* Make-lang.in (c.serial): Change from goal to a variable.
	(.PHONY): Drop c.serial.
gcc/ada/
	* gcc-interface/Make-lang.in (ada.serial): Change from goal to a
	variable.
	(.PHONY): Drop ada.serial and ada.prev.
	(gnat1$(exeext)): Depend on $(ada.serial) rather than ada.serial.
gcc/brig/
	* Make-lang.in (brig.serial): Change from goal to a variable.
	(.PHONY): Drop brig.serial and brig.prev.
	(brig1$(exeext)): Depend on $(brig.serial) rather than brig.serial.
gcc/cp/
	* Make-lang.in (c++.serial): Change from goal to a variable.
	(.PHONY): Drop c++.serial and c++.prev.
	(cc1plus$(exeext)): Depend on $(c++.serial) rather than c++.serial.
gcc/d/
	* Make-lang.in (d.serial): Change from goal to a variable.
	(.PHONY): Drop d.serial and d.prev.
	(d21$(exeext)): Depend on $(d.serial) rather than d.serial.
gcc/fortran/
	* Make-lang.in (fortran.serial): Change from goal to a variable.
	(.PHONY): Drop fortran.serial and fortran.prev.
	(f951$(exeext)): Depend on $(fortran.serial) rather than
	fortran.serial.
gcc/go/
	* Make-lang.in (go.serial): Change from goal to a variable.
	(.PHONY): Drop go.serial and go.prev.
	(go1$(exeext)): Depend on $(go.serial) rather than go.serial.
gcc/jit/
	* Make-lang.in (jit.serial): Change from goal to a
	variable.
	(.PHONY): Drop jit.serial and jit.prev.
	($(LIBGCCJIT_FILENAME)): Depend on $(jit.serial) rather than
	jit.serial.
gcc/lto/
	* Make-lang.in (lto1.serial, lto2.serial): Change from goals to
	variables.
	(.PHONY): Drop lto1.serial, lto2.serial, lto1.prev and lto2.prev.
	($(LTO_EXE)): Depend on $(lto1.serial) rather than lto1.serial.
	($(LTO_DUMP_EXE)): Depend on $(lto2.serial) rather than lto2.serial.
gcc/objc/
	* Make-lang.in (objc.serial): Change from goal to a variable.
	(.PHONY): Drop objc.serial and objc.prev.
	(cc1obj$(exeext)): Depend on $(objc.serial) rather than objc.serial.
gcc/objcp/
	* Make-lang.in (obj-c++.serial): Change from goal to a variable.
	(.PHONY): Drop obj-c++.serial and obj-c++.prev.
	(cc1objplus$(exeext)): Depend on $(obj-c++.serial) rather than
	obj-c++.serial.

--- gcc/configure.ac.jj	2020-11-19 12:34:10.763514118 +0100
+++ gcc/configure.ac	2020-11-19 15:23:24.121386881 +0100
@@ -6864,7 +6864,7 @@ echo "SERIAL_LIST =" >> Make-hooks
 echo else >> Make-hooks
 lang_cnt=0
 lang_list=
-prev=c.serial
+prev=c
 serialization_languages=c
 for lang in $all_selected_languages
 do
@@ -6880,7 +6880,7 @@ do
 	test $lang = c && continue
 	lang_cnt=`expr $lang_cnt + 1`
 	lang_list=" $prev$lang_list"
-	prev=${lang}.serial
+	prev=${lang}
 done
 echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
 echo endif >> Make-hooks
@@ -6890,7 +6890,7 @@ lang_idx=1
 for lang in $serialization_languages
 do
 	test $lang = c && continue
-	echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+	echo "$lang.prev = \$(if \$(word $lang_cnt,\$(SERIAL_LIST)),\$(\$(word $lang_cnt,\$(SERIAL_LIST)).serial))" >> Make-hooks
 	echo "INDEX.$lang = $lang_idx" >> Make-hooks
 	lang_cnt=`expr $lang_cnt - 1`
 	lang_idx=`expr $lang_idx + 1`
--- gcc/c/Make-lang.in.jj	2020-11-19 12:34:10.752514244 +0100
+++ gcc/c/Make-lang.in	2020-11-19 14:24:32.367043794 +0100
@@ -37,10 +37,10 @@
 #\f
 # Define the names for selecting c in LANGUAGES.
 c: cc1$(exeext)
-c.serial: c
+c.serial = cc1$(exeext)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: c gcc c.serial
+.PHONY: c gcc
 
 # The C front end driver.  This is different from the drivers for other
 # front ends, because there is no C language specific driver (i.e. nothing
--- gcc/ada/gcc-interface/Make-lang.in.jj	2020-11-19 12:34:10.738514404 +0100
+++ gcc/ada/gcc-interface/Make-lang.in	2020-11-19 14:23:02.475060181 +0100
@@ -146,10 +146,10 @@ endif
 
 # Define the names for selecting Ada in LANGUAGES.
 ada: gnat1$(exeext) gnatbind$(exeext)
-ada.serial: gnat1$(exeext)
+ada.serial = gnat1$(exeext)
 
 # Tell GNU Make to ignore these, if they exist.
-.PHONY: ada ada.serial ada.prev
+.PHONY: ada
 
 # Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
 # we're linking against at least libcommon which contains C++ compiled code.
@@ -669,7 +669,7 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
 # Since the RTL should be built with the latest compiler, remove the
 #  stamp target in the parent directory whenever gnat1 is rebuilt
 gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a \
-		$(LIBDEPS) ada.prev
+		$(LIBDEPS) $(ada.prev)
 	@$(call LINK_PROGRESS,$(INDEX.ada),start)
 	+$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
 	  libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
--- gcc/brig/Make-lang.in.jj	2020-11-19 12:34:10.746514312 +0100
+++ gcc/brig/Make-lang.in	2020-11-19 14:23:32.440721366 +0100
@@ -29,9 +29,9 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
 
 # The name for selecting brig in LANGUAGES.
 brig: brig1$(exeext)
-brig.serial: brig
+brig.serial = brig1$(exeext)
 
-.PHONY: brig brig.serial brig.prev
+.PHONY: brig
 
 CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
 
@@ -82,7 +82,7 @@ BRIG_OBJS = \
 
 brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
 
-brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) brig.prev
+brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) $(brig.prev)
 	@$(call LINK_PROGRESS,$(INDEX.brig),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
--- gcc/cp/Make-lang.in.jj	2020-11-19 12:34:10.764514107 +0100
+++ gcc/cp/Make-lang.in	2020-11-19 14:22:31.322412416 +0100
@@ -47,10 +47,10 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
 # into the C++ rule, but that needs a little bit of work
 # to do the right thing within all.cross.
 c++: cc1plus$(exeext)
-c++.serial: c++
+c++.serial = cc1plus$(exeext)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: c++ c++.serial c++.prev
+.PHONY: c++
 
 CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
 
@@ -117,7 +117,7 @@ cc1plus-checksum.c : build/genchecksum$(
 	  $(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
 	fi
 
-cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) c++.prev
+cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) $(c++.prev)
 	@$(call LINK_PROGRESS,$(INDEX.c++),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/d/Make-lang.in.jj	2020-11-19 12:34:10.773514004 +0100
+++ gcc/d/Make-lang.in	2020-11-19 14:26:31.676694789 +0100
@@ -27,10 +27,10 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
 
 # The name for selecting d in LANGUAGES.
 d: d21$(exeext)
-d.serial: d
+d.serial = d21$(exeext)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: d d.serial d.prev
+.PHONY: d
 
 # Create the compiler driver for D.
 CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
@@ -163,7 +163,7 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
 
 d_OBJS = $(D_ALL_OBJS) d/d-spec.o
 
-d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) d.prev
+d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) $(d.prev)
 	@$(call LINK_PROGRESS,$(INDEX.d),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/fortran/Make-lang.in.jj	2020-11-19 12:34:10.833513317 +0100
+++ gcc/fortran/Make-lang.in	2020-11-19 14:28:15.684518799 +0100
@@ -72,10 +72,10 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
 #\f
 # Define the names for selecting gfortran in LANGUAGES.
 fortran: f951$(exeext)
-fortran.serial: fortran
+fortran.serial = f951$(exeext)
 
 # Tell GNU make to ignore files by these names if they exist.
-.PHONY: fortran fortran.serial fortran.prev
+.PHONY: fortran
 
 CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
 
@@ -93,7 +93,7 @@ gfortran-cross$(exeext): gfortran$(exeex
 	cp gfortran$(exeext) gfortran-cross$(exeext)
 
 # The compiler itself is called f951.
-f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o fortran.prev
+f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o $(fortran.prev)
 	@$(call LINK_PROGRESS,$(INDEX.fortran),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
--- gcc/go/Make-lang.in.jj	2020-11-19 12:34:10.833513317 +0100
+++ gcc/go/Make-lang.in	2020-11-19 14:23:57.018443472 +0100
@@ -27,9 +27,9 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
 
 # The name for selecting go in LANGUAGES.
 go: go1$(exeext)
-go.serial: go
+go.serial = go1$(exeext)
 
-.PHONY: go go.serial go.prev
+.PHONY: go
 
 CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
 
@@ -79,7 +79,7 @@ GO_OBJS = \
 
 go_OBJS = $(GO_OBJS) go/gospec.o
 
-go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) go.prev
+go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) $(go.prev)
 	@$(call LINK_PROGRESS,$(INDEX.go),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/jit/Make-lang.in.jj	2020-11-19 12:34:10.848513146 +0100
+++ gcc/jit/Make-lang.in	2020-11-19 14:24:55.565781492 +0100
@@ -81,10 +81,10 @@ jit: $(LIBGCCJIT_FILENAME) \
 	$(FULL_DRIVER_NAME)
 endif
 
-jit.serial: $(LIBGCCJIT_FILENAME)
+jit.serial = $(LIBGCCJIT_FILENAME)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: jit jit.serial jit.prev
+.PHONY: jit
 
 jit_OBJS = attribs.o \
 	jit/dummy-frontend.o \
@@ -119,7 +119,7 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
 	libbackend.a libcommon-target.a libcommon.a \
 	$(CPPLIB) $(LIBDECNUMBER) \
 	$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
-	$(EXTRA_GCC_OBJS) jit.prev
+	$(EXTRA_GCC_OBJS) $(jit.prev)
 	@$(call LINK_PROGRESS,$(INDEX.jit),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
 	     $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
--- gcc/lto/Make-lang.in.jj	2020-11-19 12:34:10.850513123 +0100
+++ gcc/lto/Make-lang.in	2020-11-19 14:28:55.894064159 +0100
@@ -42,10 +42,10 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
 # Rules
 
 lto: $(LTO_EXE) $(LTO_DUMP_EXE)
-lto1.serial: $(LTO_EXE)
-lto2.serial: $(LTO_DUMP_EXE)
+lto1.serial = $(LTO_EXE)
+lto2.serial = $(LTO_DUMP_EXE)
 
-.PHONY: lto lto1.serial lto1.prev lto2.serial lto2.prev
+.PHONY: lto
 
 # These hooks are used by the main GCC Makefile.  Consult that
 # Makefile for documentation.
@@ -90,13 +90,13 @@ lto.stagefeedback:
 # Use strict warnings for this front end.
 lto-warn = $(STRICT_WARN)
 
-$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) lto1.prev
+$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) $(lto1.prev)
 	@$(call LINK_PROGRESS,$(INDEX.lto1),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
 	@$(call LINK_PROGRESS,$(INDEX.lto1),end)
 
-$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) lto2.prev
+$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) $(lto2.prev)
 	@$(call LINK_PROGRESS,$(INDEX.lto2),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
--- gcc/objc/Make-lang.in.jj	2020-11-19 12:34:10.852513100 +0100
+++ gcc/objc/Make-lang.in	2020-11-19 14:26:53.544447536 +0100
@@ -38,10 +38,10 @@
 #\f
 # Define the names for selecting Objective-C in LANGUAGES.
 objc: cc1obj$(exeext)
-objc.serial: objc
+objc.serial = cc1obj$(exeext)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: objc objc.serial objc.prev
+.PHONY: objc
 
 # Use maximal warnings for this front end.
 objc-warn = $(STRICT_WARN)
@@ -64,7 +64,7 @@ cc1obj-checksum.c : build/genchecksum$(b
 	$(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
 
 cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \
-		 $(LIBDEPS) objc.prev
+		 $(LIBDEPS) $(objc.prev)
 	@$(call LINK_PROGRESS,$(INDEX.objc),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 	      $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
--- gcc/objcp/Make-lang.in.jj	2020-11-19 12:34:10.852513100 +0100
+++ gcc/objcp/Make-lang.in	2020-11-19 14:27:49.570814060 +0100
@@ -39,10 +39,10 @@
 #\f
 # Define the names for selecting Objective-C++ in LANGUAGES.
 obj-c++: cc1objplus$(exeext)
-obj-c++.serial: obj-c++
+obj-c++.serial = cc1objplus$(exeext)
 
 # Tell GNU make to ignore these if they exist.
-.PHONY: obj-c++ obj-c++.serial obj-c++.prev
+.PHONY: obj-c++
 
 # Use maximal warnings for this front end.  Also, make ObjC and C++
 # headers accessible.
@@ -68,7 +68,7 @@ cc1objplus-checksum.c : build/genchecksu
 	cc1objplus-checksum.c
 
 cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) \
-		     $(LIBDEPS) obj-c++.prev
+		     $(LIBDEPS) $(obj-c++.prev)
 	@$(call LINK_PROGRESS,$(INDEX.obj-c++),start)
 	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
 		$(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/configure.jj	2020-11-19 12:34:10.762514130 +0100
+++ gcc/configure	2020-11-19 15:23:37.063244235 +0100
@@ -30769,7 +30769,7 @@ echo "SERIAL_LIST =" >> Make-hooks
 echo else >> Make-hooks
 lang_cnt=0
 lang_list=
-prev=c.serial
+prev=c
 serialization_languages=c
 for lang in $all_selected_languages
 do
@@ -30785,7 +30785,7 @@ do
 	test $lang = c && continue
 	lang_cnt=`expr $lang_cnt + 1`
 	lang_list=" $prev$lang_list"
-	prev=${lang}.serial
+	prev=${lang}
 done
 echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
 echo endif >> Make-hooks
@@ -30795,7 +30795,7 @@ lang_idx=1
 for lang in $serialization_languages
 do
 	test $lang = c && continue
-	echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+	echo "$lang.prev = \$(if \$(word $lang_cnt,\$(SERIAL_LIST)),\$(\$(word $lang_cnt,\$(SERIAL_LIST)).serial))" >> Make-hooks
 	echo "INDEX.$lang = $lang_idx" >> Make-hooks
 	lang_cnt=`expr $lang_cnt - 1`
 	lang_idx=`expr $lang_idx + 1`


	Jakub

Re: [PATCH] configury: --enable-link-serialization support

[Jakub Jelinek]

On Thu, Nov 19, 2020 at 03:50:27PM +0100, Jakub Jelinek via Gcc-patches wrote:
> So, I think the problem is that for make .PHONY targets are just
> "rebuilt" always, so it is very much undesirable for the cc1plus$(exeext)
> etc. dependencies to include .PHONY targets, but I was using
> them - cc1plus.prev which would depend on some *.serial and
> e.g. cc1.serial depending on c and c depending on cc1$(exeext).
> 
> The following so far only very lightly tested patch rewrites this
> so that *.serial and *.prev aren't .PHONY targets, but instead just
> make variables.
> 
> I was worried that the order in which the language makefile fragments are
> included (which is quite random, what order we get from the filesystem
> matching */config-lang.in) would be a problem but it seems to work fine.

Successfully bootstrapped/regtested on x86_64-linux and i686-linux,
including make install which looked problematic in PR97911.

Ok for trunk?

> 2020-11-19  Jakub Jelinek  <jakub@redhat.com>
> 
> gcc/
> 	* configure.ac: In SERIAL_LIST use lang words without .serial
> 	suffix.  Change $lang.prev from a target to variable and instead
> 	of depending on *.serial expand to the *.serial variable if
> 	the word is in the SERIAL_LIST at all, otherwise to nothing.
> 	* configure: Regenerated.
> gcc/c/
> 	* Make-lang.in (c.serial): Change from goal to a variable.
> 	(.PHONY): Drop c.serial.
> gcc/ada/
> 	* gcc-interface/Make-lang.in (ada.serial): Change from goal to a
> 	variable.
> 	(.PHONY): Drop ada.serial and ada.prev.
> 	(gnat1$(exeext)): Depend on $(ada.serial) rather than ada.serial.
> gcc/brig/
> 	* Make-lang.in (brig.serial): Change from goal to a variable.
> 	(.PHONY): Drop brig.serial and brig.prev.
> 	(brig1$(exeext)): Depend on $(brig.serial) rather than brig.serial.
> gcc/cp/
> 	* Make-lang.in (c++.serial): Change from goal to a variable.
> 	(.PHONY): Drop c++.serial and c++.prev.
> 	(cc1plus$(exeext)): Depend on $(c++.serial) rather than c++.serial.
> gcc/d/
> 	* Make-lang.in (d.serial): Change from goal to a variable.
> 	(.PHONY): Drop d.serial and d.prev.
> 	(d21$(exeext)): Depend on $(d.serial) rather than d.serial.
> gcc/fortran/
> 	* Make-lang.in (fortran.serial): Change from goal to a variable.
> 	(.PHONY): Drop fortran.serial and fortran.prev.
> 	(f951$(exeext)): Depend on $(fortran.serial) rather than
> 	fortran.serial.
> gcc/go/
> 	* Make-lang.in (go.serial): Change from goal to a variable.
> 	(.PHONY): Drop go.serial and go.prev.
> 	(go1$(exeext)): Depend on $(go.serial) rather than go.serial.
> gcc/jit/
> 	* Make-lang.in (jit.serial): Change from goal to a
> 	variable.
> 	(.PHONY): Drop jit.serial and jit.prev.
> 	($(LIBGCCJIT_FILENAME)): Depend on $(jit.serial) rather than
> 	jit.serial.
> gcc/lto/
> 	* Make-lang.in (lto1.serial, lto2.serial): Change from goals to
> 	variables.
> 	(.PHONY): Drop lto1.serial, lto2.serial, lto1.prev and lto2.prev.
> 	($(LTO_EXE)): Depend on $(lto1.serial) rather than lto1.serial.
> 	($(LTO_DUMP_EXE)): Depend on $(lto2.serial) rather than lto2.serial.
> gcc/objc/
> 	* Make-lang.in (objc.serial): Change from goal to a variable.
> 	(.PHONY): Drop objc.serial and objc.prev.
> 	(cc1obj$(exeext)): Depend on $(objc.serial) rather than objc.serial.
> gcc/objcp/
> 	* Make-lang.in (obj-c++.serial): Change from goal to a variable.
> 	(.PHONY): Drop obj-c++.serial and obj-c++.prev.
> 	(cc1objplus$(exeext)): Depend on $(obj-c++.serial) rather than
> 	obj-c++.serial.

	Jakub

Re: [PATCH] configury: --enable-link-serialization support

[Eric Botcazou]

> Successfully bootstrapped/regtested on x86_64-linux and i686-linux,
> including make install which looked problematic in PR97911.
> 
> Ok for trunk?

I cannot really approve, but this looks like a step in the right direction.

-- 
Eric Botcazou

Re: [PATCH] configury: --enable-link-serialization support

[Richard Biener]

On Fri, Nov 20, 2020 at 12:30 AM Eric Botcazou <botcazou@adacore.com> wrote:
>
> > Successfully bootstrapped/regtested on x86_64-linux and i686-linux,
> > including make install which looked problematic in PR97911.
> >
> > Ok for trunk?
>
> I cannot really approve, but this looks like a step in the right direction.

OK.

Thanks,
Richard.

> --
> Eric Botcazou
>
>