* RFC: --enable-link-serialization support
@ 2020-09-03 13:36 Jakub Jelinek
2020-09-03 13:53 ` Richard Biener
0 siblings, 1 reply; 6+ messages in thread
From: Jakub Jelinek @ 2020-09-03 13:36 UTC (permalink / raw)
To: gcc-patches; +Cc: Richard Biener
Hi!
As discussed earlier on IRC, --enable-link-mutex has a big problem that it
decreases number of available jobs by the number of link commands waiting
for the lock, so e.g. when doing make -j32 build with 11 different
big programs linked with $(LLINKER) we end up with just 22 effective jobs,
and with e.g. make -j8 with those 11 different big programs we actually most
likely serialize everything during linking onto a single job.
The following patch is an attempt to handle it using Make dependencies
instead, let configure create a chain, e.g.
c++.prev: c.serial
fortran.prev: c++.serial
lto.prev: fortran.serial
in Make-hooks, which says that cc1plus can be only linked after cc1 is
linked, then f951, then lto1 and then lto-dump.
One thing I don't like very much on this implementation is that it uses
recursive make, something we apparently don't use in gcc/ right now.
One reason for that is that I wanted to make the case when I do:
make -j32 cc1plus
by hand to avoid uselessly building cc1, gnat1 etc. (whatever is in
the CONFIG_LANGUAGES before), i.e. only serialize if done from a bootstrap.
But perhaps there could be a way out of that, by moving the
--enable-link-serialization to toplevel configure and arrange for it to just
pass some variable to the gcc make in that case. Then the Make-hooks could
look like:
ifeq (DO_LINK_SERIALIZATION,true)
ada.prev: c.serial
brig.prev: ada.serial
c++.prev: brig.serial
d.prev: c++.serial
fortran.prev: d.serial
go.prev: fortran.serial
lto.prev: go.serial
objc.prev: lto.serial
obj-c++.prev: objc.serial
else
ada.prev:
brig.prev:
c++.prev:
d.prev:
fortran.prev:
go.prev:
lto.prev:
objc.prev:
obj-c++.prev:
endif
and then just add c++.prev dependency to cc1obj$(exeext) dependencies
unconditionally.
Thoughts on that?
2020-09-03 Jakub Jelinek <jakub@redhat.com>
gcc/
* configure.ac: Add --enable-link-serialization support, add into
Make-hooks rules on which language is previous in the list of
all selected languages.
* Makefile.in (LANGUAGES): Add *.serial instead of * if
--enable-link-serialization is on.
* configure: Regenerated.
gcc/c/
* Make-lang.in (c.serial): Depend on c.
(.PHONY): Add c.serial.
gcc/cp/
* Make-lang.in (.PHONY): Add c++.serial c++.prev.
(CC1PLUS_DEPS): New variable.
(cc1plus$(exeext)): Use it.
(c++.serial): New goal.
gcc/ada/
* Make-lang.in (.PHONY): Add ada.serial ada.prev.
(GNAT1_DEPS): New variable.
(gnat1$(exeext)): Use it.
(ada.serial): New goal.
gcc/brig/
* Make-lang.in (.PHONY): Add brig.serial brig.prev.
(BRIG1_DEPS): New variable.
(brig1$(exeext)): Use it.
(brig.serial): New goal.
gcc/go/
* Make-lang.in (.PHONY): Add go.serial go.prev.
(GO1_DEPS): New variable.
(go1$(exeext)): Use it.
(go.serial): New goal.
gcc/jit/
* Make-lang.in (.PHONY): Add jit.serial jit.prev jit.extra.
(jit.extra): New goal.
(jit): Use it.
(LIBGCCJIT_FILENAME_DEPS): New variable.
($(LIBGCCJIT_FILENAME)): Use it.
(jit.serial): New goal.
gcc/objc/
* Make-lang.in (.PHONY): Add objc.serial objc.prev.
(CC1OBJ_DEPS): New variable.
(cc1obj$(exeext)): Use it.
(objc.serial): New goal.
gcc/objcp/
* Make-lang.in (.PHONY): Add obj-c++.serial obj-c++.prev.
(CC1OBJPLUS_DEPS): New variable.
(cc1objplus$(exeext)): Use it.
(obj-c++.serial): New goal.
gcc/fortran/
* Make-lang.in (.PHONY): Add fortran.serial fortran.prev.
(F951_DEPS): New variable.
(f951$(exeext)): Use it.
(fortran.serial): New goal.
gcc/lto/
* Make-lang.in (.PHONY): Add lto lto.serial lto.prev.
(lto): Depend on $(LTO_EXE) $(LTO_DUMP_EXE).
(lto.all-cross, lto.start.encap): Don't depend on anything.
(LTO_EXE_DEPS): New variable.
($(LTO_EXE)): Use it.
(LTO_DUMP_EXE_DEPS): New variable.
($(LTO_DUMP_EXE)): Use it.
(lto.serial): New goal.
--- gcc/configure.ac.jj 2020-08-26 17:09:45.829254709 +0200
+++ gcc/configure.ac 2020-09-03 14:41:02.884746491 +0200
@@ -6380,6 +6380,24 @@ else
fi
AC_SUBST(DO_LINK_MUTEX)
+dnl Another way of doing that, purely using make dependencies
+
+AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
+ AC_ARG_ENABLE(link-serialization,
+[AS_HELP_STRING([--enable-link-serialization],
+ [avoid linking multiple front-ends at once to avoid thrashing
+ on the build machine through make dependencies])],
+ do_link_serialization=$enableval,
+ do_link_serialization=no)
+AC_MSG_RESULT($do_link_serialization)
+
+if test "$do_link_serialization" = "yes"; then
+ DO_LINK_SERIALIZATION=true
+else
+ DO_LINK_SERIALIZATION=false
+fi
+AC_SUBST(DO_LINK_SERIALIZATION)
+
# --------------
# Language hooks
# --------------
@@ -6540,6 +6558,14 @@ do
echo "lang.$t: $x" >> Make-hooks
done
+prev=c
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: $prev.serial" >> Make-hooks
+ prev=$lang
+done
+
# --------
# Option include files
# --------
--- gcc/Makefile.in.jj 2020-08-18 07:50:18.440924074 +0200
+++ gcc/Makefile.in 2020-09-03 13:53:10.041872344 +0200
@@ -137,7 +137,11 @@ SUBDIRS =@subdirs@ build
# Selection of languages to be made.
CONFIG_LANGUAGES = @all_selected_languages@
+ifeq (@DO_LINK_SERIALIZATION@,true)
+LANGUAGES = $(addsuffix .serial,c $(CONFIG_LANGUAGES))
+else
LANGUAGES = c $(CONFIG_LANGUAGES)
+endif
ifeq (@enable_gcov@,yes)
LANGUAGES += gcov$(exeext) gcov-dump$(exeext) gcov-tool$(exeext)
endif
--- gcc/c/Make-lang.in.jj 2020-07-28 15:39:09.669760889 +0200
+++ gcc/c/Make-lang.in 2020-09-03 14:24:41.861122758 +0200
@@ -37,9 +37,10 @@
#\f
# Define the names for selecting c in LANGUAGES.
c: cc1$(exeext)
+c.serial: c
# Tell GNU make to ignore these if they exist.
-.PHONY: c gcc
+.PHONY: c gcc c.serial
# The C front end driver. This is different from the drivers for other
# front ends, because there is no C language specific driver (i.e. nothing
--- gcc/cp/Make-lang.in.jj 2020-07-28 15:39:09.770759500 +0200
+++ gcc/cp/Make-lang.in 2020-09-03 14:44:21.099842403 +0200
@@ -49,7 +49,7 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
c++: cc1plus$(exeext)
# Tell GNU make to ignore these if they exist.
-.PHONY: c++
+.PHONY: c++ c++.serial c++.prev
CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
@@ -116,10 +116,14 @@ cc1plus-checksum.c : build/genchecksum$(
$(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
fi
-cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
+CC1PLUS_DEPS = $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1plus$(exeext): $(CC1PLUS_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+c++.serial: c++.prev $(CC1PLUS_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) c++
+
ifeq ($(ENABLE_MAINTAINER_RULES), true)
# Special build rule. This is a maintainer rule, that is only
# available when GCC is configured with --enable-maintainer-mode. In
--- gcc/ada/gcc-interface/Make-lang.in.jj 2020-07-28 15:39:09.470763625 +0200
+++ gcc/ada/gcc-interface/Make-lang.in 2020-09-03 14:54:45.672695769 +0200
@@ -148,7 +148,7 @@ endif
ada: gnat1$(exeext) gnatbind$(exeext)
# Tell GNU Make to ignore these, if they exist.
-.PHONY: ada
+.PHONY: ada ada.serial ada.prev
# Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
# we're linking against at least libcommon which contains C++ compiled code.
@@ -666,11 +666,15 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
# Needs to be built with CC=gcc
# Since the RTL should be built with the latest compiler, remove the
# stamp target in the parent directory whenever gnat1 is rebuilt
-gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
+GNAT1_DEPS = $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
+gnat1$(exeext): $(GNAT1_DEPS)
+$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
$(RM) stamp-gnatlib2-rts stamp-tools
+ada.serial: gnatbind$(exeext) ada.prev $(GNAT1_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) ada
+
gnatbind$(exeext): ada/b_gnatb.o $(CONFIG_H) $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBDEPS)
+$(GCC_LINK) -o $@ ada/b_gnatb.o $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBS) $(SYSLIBS) $(CFLAGS)
--- gcc/brig/Make-lang.in.jj 2020-01-12 11:54:36.193416728 +0100
+++ gcc/brig/Make-lang.in 2020-09-03 14:56:26.067225541 +0200
@@ -30,7 +30,7 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
# The name for selecting brig in LANGUAGES.
brig: brig1$(exeext)
-.PHONY: brig
+.PHONY: brig brig.serial brig.prev
CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
@@ -81,16 +81,15 @@ BRIG_OBJS = \
brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
-# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
-# +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
-# $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
-
-
-brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+BRIG1_DEPS = $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+brig1$(exeext): $(BRIG1_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
$(BACKENDLIBS)
+brig.serial: brig.prev $(BRIG1_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) brig
+
# Documentation.
BRIG_TEXI_FILES = \
--- gcc/go/Make-lang.in.jj 2020-01-12 11:54:36.630410135 +0100
+++ gcc/go/Make-lang.in 2020-09-03 14:57:04.455663361 +0200
@@ -28,7 +28,7 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
# The name for selecting go in LANGUAGES.
go: go1$(exeext)
-.PHONY: go
+.PHONY: go go.serial go.prev
CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
@@ -78,10 +78,14 @@ GO_OBJS = \
go_OBJS = $(GO_OBJS) go/gospec.o
-go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+GO1_DEPS = $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+go1$(exeext): $(GO1_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+go.serial: go.prev $(GO1_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) go
+
# Documentation.
GO_TEXI_FILES = \
--- gcc/jit/Make-lang.in.jj 2020-07-28 15:39:09.956756942 +0200
+++ gcc/jit/Make-lang.in 2020-09-03 14:51:31.109545079 +0200
@@ -48,8 +48,7 @@ ifneq (,$(findstring mingw,$(target)))
LIBGCCJIT_FILENAME = libgccjit-$(LIBGCCJIT_VERSION_NUM).dll
LIBGCCJIT_IMPORT_LIB = libgccjit.dll.a
-jit: $(LIBGCCJIT_FILENAME) \
- $(FULL_DRIVER_NAME)
+jit.extra: $(FULL_DRIVER_NAME)
else
@@ -75,14 +74,16 @@ LIBGCCJIT_SONAME_OPTION = \
$(if $(LD_SONAME_OPTION), \
-Wl$(COMMA)$(LD_SONAME_OPTION)$(COMMA)$(LIBGCCJIT_SONAME))
-jit: $(LIBGCCJIT_FILENAME) \
- $(LIBGCCJIT_SYMLINK) \
+jit.extra: $(LIBGCCJIT_SYMLINK) \
$(LIBGCCJIT_LINKER_NAME_SYMLINK) \
$(FULL_DRIVER_NAME)
+
endif
+jit: $(LIBGCCJIT_FILENAME) jit.extra
+
# Tell GNU make to ignore these if they exist.
-.PHONY: jit
+.PHONY: jit jit.serial jit.prev jit.extra
jit_OBJS = attribs.o \
jit/dummy-frontend.o \
@@ -113,17 +114,21 @@ endif
# We avoid using $(BACKEND) from Makefile.in in order to avoid pulling
# in main.o
-$(LIBGCCJIT_FILENAME): $(jit_OBJS) \
+LIBGCCJIT_FILENAME_DEPS = $(jit_OBJS) \
libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) \
$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
$(EXTRA_GCC_OBJS)
+$(LIBGCCJIT_FILENAME): $(LIBGCCJIT_FILENAME_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
$(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
$(EXTRA_GCC_OBJS) \
$(LIBGCCJIT_EXTRA_OPTS)
+jit.serial: jit.prev $(LIBGCCJIT_FILENAME_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) jit
+
# Create symlinks when not building for Windows
ifeq (,$(findstring mingw,$(target)))
$(LIBGCCJIT_SONAME_SYMLINK): $(LIBGCCJIT_FILENAME)
--- gcc/d/Make-lang.in.jj 2020-08-26 10:27:10.690552343 +0200
+++ gcc/d/Make-lang.in 2020-09-03 14:52:31.675658110 +0200
@@ -29,7 +29,7 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
d: d21$(exeext)
# Tell GNU make to ignore these if they exist.
-.PHONY: d
+.PHONY: d d.serial d.prev
# Create the compiler driver for D.
CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
@@ -162,10 +162,14 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
d_OBJS = $(D_ALL_OBJS) d/d-spec.o
-d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+D21_DEPS = $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+d21$(exeext): $(D21_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+d.serial: d.prev $(D21_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) d
+
# Documentation.
D_TEXI_FILES = \
--- gcc/objc/Make-lang.in.jj 2020-01-12 11:54:36.681409365 +0100
+++ gcc/objc/Make-lang.in 2020-09-03 14:57:54.789926228 +0200
@@ -40,7 +40,7 @@
objc: cc1obj$(exeext)
# Tell GNU make to ignore these if they exist.
-.PHONY: objc
+.PHONY: objc objc.serial objc.prev
# Use maximal warnings for this front end.
objc-warn = $(STRICT_WARN)
@@ -62,11 +62,15 @@ cc1obj-checksum.c : build/genchecksum$(b
$(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
$(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
-cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
+CC1OBJ_DEPS = $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
+cc1obj$(exeext): $(CC1OBJ_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
$(BACKEND) $(LIBS) $(BACKENDLIBS)
+objc.serial: objc.prev $(CC1OBJ_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) objc
+
objc.srcextra:
#\f
--- gcc/objcp/Make-lang.in.jj 2020-01-12 11:54:36.685409305 +0100
+++ gcc/objcp/Make-lang.in 2020-09-03 14:58:44.181202905 +0200
@@ -41,7 +41,7 @@
obj-c++: cc1objplus$(exeext)
# Tell GNU make to ignore these if they exist.
-.PHONY: obj-c++
+.PHONY: obj-c++ obj-c++.serial obj-c++.prev
# Use maximal warnings for this front end. Also, make ObjC and C++
# headers accessible.
@@ -66,10 +66,14 @@ cc1objplus-checksum.c : build/genchecksu
$(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
cc1objplus-checksum.c
-cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
+CC1OBJPLUS_DEPS = $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1objplus$(exeext): $(CC1OBJPLUS_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+obj-c++.serial: obj-c++.prev $(CC1OBJPLUS_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) obj-c++
+
# Objective C++ language specific files.
objcp/objc-runtime-shared-support.o: objc/objc-runtime-shared-support.c
--- gcc/fortran/Make-lang.in.jj 2020-01-12 11:54:36.582410859 +0100
+++ gcc/fortran/Make-lang.in 2020-09-03 14:53:48.550532310 +0200
@@ -74,7 +74,7 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
fortran: f951$(exeext)
# Tell GNU make to ignore files by these names if they exist.
-.PHONY: fortran
+.PHONY: fortran fortran.serial fortran.prev
CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
@@ -92,12 +92,15 @@ gfortran-cross$(exeext): gfortran$(exeex
cp gfortran$(exeext) gfortran-cross$(exeext)
# The compiler itself is called f951.
-f951$(exeext): $(F95_OBJS) \
- $(BACKEND) $(LIBDEPS) attribs.o
+F951_DEPS = $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o
+f951$(exeext): $(F951_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
$(BACKENDLIBS)
+fortran.serial: fortran.prev $(F951_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) fortran
+
gt-fortran-trans.h : s-gtype; @true
#\f
# Build hooks:
--- gcc/lto/Make-lang.in.jj 2020-01-12 11:54:36.677409426 +0100
+++ gcc/lto/Make-lang.in 2020-09-03 14:45:51.957511831 +0200
@@ -41,10 +41,14 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
# Rules
+lto: $(LTO_EXE) $(LTO_DUMP_EXE)
+
+.PHONY: lto lto.serial lto.prev
+
# These hooks are used by the main GCC Makefile. Consult that
# Makefile for documentation.
-lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
-lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto.all.cross:
+lto.start.encap:
lto.rest.encap:
lto.tags:
lto.install-common: installdirs
@@ -84,14 +88,20 @@ lto.stagefeedback:
# Use strict warnings for this front end.
lto-warn = $(STRICT_WARN)
-$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+LTO_EXE_DEPS = $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_EXE): $(LTO_EXE_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
-$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
+LTO_DUMP_EXE_DEPS = $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_DUMP_EXE): $(LTO_DUMP_EXE_DEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
+lto.serial: lto.prev $(LTO_EXE_DEPS) $(LTO_DUMP_EXE_DEPS)
+ $(MAKE) $(FLAGS_TO_PASS) $(LTO_EXE)
+ $(MAKE) $(FLAGS_TO_PASS) $(LTO_DUMP_EXE)
+
lto/lto-dump.o: $(LTO_OBJS)
lto1.fda: ../prev-gcc/lto1$(exeext) ../prev-gcc/$(PERF_DATA)
--- gcc/configure.jj 2020-08-26 17:09:45.828254723 +0200
+++ gcc/configure 2020-09-03 14:41:09.632647617 +0200
@@ -710,6 +710,7 @@ subdirs
dollar
gcc_tooldir
enable_lto
+DO_LINK_SERIALIZATION
DO_LINK_MUTEX
MAINT
zlibinc
@@ -1012,6 +1013,7 @@ with_gc
with_system_zlib
enable_maintainer_mode
enable_link_mutex
+enable_link_serialization
enable_version_specific_runtime_libs
enable_plugin
enable_host_shared
@@ -1767,6 +1769,10 @@ Optional Features:
sometimes confusing) to the casual installer
--enable-link-mutex avoid linking multiple front-ends at once to avoid
thrashing on the build machine
+ --enable-link-serialization
+ avoid linking multiple front-ends at once to avoid
+ thrashing on the build machine through make
+ dependencies
--enable-version-specific-runtime-libs
specify that runtime libraries should be installed
in a compiler-specific directory
@@ -19013,7 +19019,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 19016 "configure"
+#line 19022 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -19119,7 +19125,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 19122 "configure"
+#line 19128 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -30049,6 +30055,26 @@ else
fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
+$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
+ # Check whether --enable-link-serialization was given.
+if test "${enable_link_serialization+set}" = set; then :
+ enableval=$enable_link_serialization; do_link_serialization=$enableval
+else
+ do_link_serialization=no
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
+$as_echo "$do_link_serialization" >&6; }
+
+if test "$do_link_serialization" = "yes"; then
+ DO_LINK_SERIALIZATION=true
+else
+ DO_LINK_SERIALIZATION=false
+fi
+
+
# --------------
# Language hooks
# --------------
@@ -30209,6 +30235,14 @@ do
echo "lang.$t: $x" >> Make-hooks
done
+prev=c
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: $prev.serial" >> Make-hooks
+ prev=$lang
+done
+
# --------
# Option include files
# --------
Jakub
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: RFC: --enable-link-serialization support
2020-09-03 13:36 RFC: --enable-link-serialization support Jakub Jelinek
@ 2020-09-03 13:53 ` Richard Biener
2020-09-03 14:49 ` [PATCH] " Jakub Jelinek
0 siblings, 1 reply; 6+ messages in thread
From: Richard Biener @ 2020-09-03 13:53 UTC (permalink / raw)
To: Jakub Jelinek; +Cc: gcc-patches
On Thu, 3 Sep 2020, Jakub Jelinek wrote:
> Hi!
>
> As discussed earlier on IRC, --enable-link-mutex has a big problem that it
> decreases number of available jobs by the number of link commands waiting
> for the lock, so e.g. when doing make -j32 build with 11 different
> big programs linked with $(LLINKER) we end up with just 22 effective jobs,
> and with e.g. make -j8 with those 11 different big programs we actually most
> likely serialize everything during linking onto a single job.
>
> The following patch is an attempt to handle it using Make dependencies
> instead, let configure create a chain, e.g.
> c++.prev: c.serial
> fortran.prev: c++.serial
> lto.prev: fortran.serial
> in Make-hooks, which says that cc1plus can be only linked after cc1 is
> linked, then f951, then lto1 and then lto-dump.
>
> One thing I don't like very much on this implementation is that it uses
> recursive make, something we apparently don't use in gcc/ right now.
>
> One reason for that is that I wanted to make the case when I do:
> make -j32 cc1plus
> by hand to avoid uselessly building cc1, gnat1 etc. (whatever is in
> the CONFIG_LANGUAGES before), i.e. only serialize if done from a bootstrap.
But is that an issue in practice? I usually do not do make -j32 cc1plus
in a tree that was configured for bootstrap, nor do I use
--enable-link-serialization in that case.
Btw, do you plan to keep --enable-link-mutex the way it is? What it
provides ontop of --enable-link-serialization is that progress
metering for very long running LTO link stages. Not sure if that
works for the last LTO link or only when there's some other link
waiting.
No review on the actual patch - it looks like what I'd have tried
but I'm no make expert ;)
There's a bug about the bad --enable-link-mutex behavior btw (but
you don't fix that, just provide an alternate implementation).
Thanks,
Richard.
> But perhaps there could be a way out of that, by moving the
> --enable-link-serialization to toplevel configure and arrange for it to just
> pass some variable to the gcc make in that case. Then the Make-hooks could
> look like:
> ifeq (DO_LINK_SERIALIZATION,true)
> ada.prev: c.serial
> brig.prev: ada.serial
> c++.prev: brig.serial
> d.prev: c++.serial
> fortran.prev: d.serial
> go.prev: fortran.serial
> lto.prev: go.serial
> objc.prev: lto.serial
> obj-c++.prev: objc.serial
> else
> ada.prev:
> brig.prev:
> c++.prev:
> d.prev:
> fortran.prev:
> go.prev:
> lto.prev:
> objc.prev:
> obj-c++.prev:
> endif
> and then just add c++.prev dependency to cc1obj$(exeext) dependencies
> unconditionally.
> Thoughts on that?
>
> 2020-09-03 Jakub Jelinek <jakub@redhat.com>
>
> gcc/
> * configure.ac: Add --enable-link-serialization support, add into
> Make-hooks rules on which language is previous in the list of
> all selected languages.
> * Makefile.in (LANGUAGES): Add *.serial instead of * if
> --enable-link-serialization is on.
> * configure: Regenerated.
> gcc/c/
> * Make-lang.in (c.serial): Depend on c.
> (.PHONY): Add c.serial.
> gcc/cp/
> * Make-lang.in (.PHONY): Add c++.serial c++.prev.
> (CC1PLUS_DEPS): New variable.
> (cc1plus$(exeext)): Use it.
> (c++.serial): New goal.
> gcc/ada/
> * Make-lang.in (.PHONY): Add ada.serial ada.prev.
> (GNAT1_DEPS): New variable.
> (gnat1$(exeext)): Use it.
> (ada.serial): New goal.
> gcc/brig/
> * Make-lang.in (.PHONY): Add brig.serial brig.prev.
> (BRIG1_DEPS): New variable.
> (brig1$(exeext)): Use it.
> (brig.serial): New goal.
> gcc/go/
> * Make-lang.in (.PHONY): Add go.serial go.prev.
> (GO1_DEPS): New variable.
> (go1$(exeext)): Use it.
> (go.serial): New goal.
> gcc/jit/
> * Make-lang.in (.PHONY): Add jit.serial jit.prev jit.extra.
> (jit.extra): New goal.
> (jit): Use it.
> (LIBGCCJIT_FILENAME_DEPS): New variable.
> ($(LIBGCCJIT_FILENAME)): Use it.
> (jit.serial): New goal.
> gcc/objc/
> * Make-lang.in (.PHONY): Add objc.serial objc.prev.
> (CC1OBJ_DEPS): New variable.
> (cc1obj$(exeext)): Use it.
> (objc.serial): New goal.
> gcc/objcp/
> * Make-lang.in (.PHONY): Add obj-c++.serial obj-c++.prev.
> (CC1OBJPLUS_DEPS): New variable.
> (cc1objplus$(exeext)): Use it.
> (obj-c++.serial): New goal.
> gcc/fortran/
> * Make-lang.in (.PHONY): Add fortran.serial fortran.prev.
> (F951_DEPS): New variable.
> (f951$(exeext)): Use it.
> (fortran.serial): New goal.
> gcc/lto/
> * Make-lang.in (.PHONY): Add lto lto.serial lto.prev.
> (lto): Depend on $(LTO_EXE) $(LTO_DUMP_EXE).
> (lto.all-cross, lto.start.encap): Don't depend on anything.
> (LTO_EXE_DEPS): New variable.
> ($(LTO_EXE)): Use it.
> (LTO_DUMP_EXE_DEPS): New variable.
> ($(LTO_DUMP_EXE)): Use it.
> (lto.serial): New goal.
>
> --- gcc/configure.ac.jj 2020-08-26 17:09:45.829254709 +0200
> +++ gcc/configure.ac 2020-09-03 14:41:02.884746491 +0200
> @@ -6380,6 +6380,24 @@ else
> fi
> AC_SUBST(DO_LINK_MUTEX)
>
> +dnl Another way of doing that, purely using make dependencies
> +
> +AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
> + AC_ARG_ENABLE(link-serialization,
> +[AS_HELP_STRING([--enable-link-serialization],
> + [avoid linking multiple front-ends at once to avoid thrashing
> + on the build machine through make dependencies])],
> + do_link_serialization=$enableval,
> + do_link_serialization=no)
> +AC_MSG_RESULT($do_link_serialization)
> +
> +if test "$do_link_serialization" = "yes"; then
> + DO_LINK_SERIALIZATION=true
> +else
> + DO_LINK_SERIALIZATION=false
> +fi
> +AC_SUBST(DO_LINK_SERIALIZATION)
> +
> # --------------
> # Language hooks
> # --------------
> @@ -6540,6 +6558,14 @@ do
> echo "lang.$t: $x" >> Make-hooks
> done
>
> +prev=c
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev: $prev.serial" >> Make-hooks
> + prev=$lang
> +done
> +
> # --------
> # Option include files
> # --------
> --- gcc/Makefile.in.jj 2020-08-18 07:50:18.440924074 +0200
> +++ gcc/Makefile.in 2020-09-03 13:53:10.041872344 +0200
> @@ -137,7 +137,11 @@ SUBDIRS =@subdirs@ build
>
> # Selection of languages to be made.
> CONFIG_LANGUAGES = @all_selected_languages@
> +ifeq (@DO_LINK_SERIALIZATION@,true)
> +LANGUAGES = $(addsuffix .serial,c $(CONFIG_LANGUAGES))
> +else
> LANGUAGES = c $(CONFIG_LANGUAGES)
> +endif
> ifeq (@enable_gcov@,yes)
> LANGUAGES += gcov$(exeext) gcov-dump$(exeext) gcov-tool$(exeext)
> endif
> --- gcc/c/Make-lang.in.jj 2020-07-28 15:39:09.669760889 +0200
> +++ gcc/c/Make-lang.in 2020-09-03 14:24:41.861122758 +0200
> @@ -37,9 +37,10 @@
> #\f
> # Define the names for selecting c in LANGUAGES.
> c: cc1$(exeext)
> +c.serial: c
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: c gcc
> +.PHONY: c gcc c.serial
>
> # The C front end driver. This is different from the drivers for other
> # front ends, because there is no C language specific driver (i.e. nothing
> --- gcc/cp/Make-lang.in.jj 2020-07-28 15:39:09.770759500 +0200
> +++ gcc/cp/Make-lang.in 2020-09-03 14:44:21.099842403 +0200
> @@ -49,7 +49,7 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
> c++: cc1plus$(exeext)
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: c++
> +.PHONY: c++ c++.serial c++.prev
>
> CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
>
> @@ -116,10 +116,14 @@ cc1plus-checksum.c : build/genchecksum$(
> $(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
> fi
>
> -cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
> +CC1PLUS_DEPS = $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1plus$(exeext): $(CC1PLUS_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> +c++.serial: c++.prev $(CC1PLUS_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) c++
> +
> ifeq ($(ENABLE_MAINTAINER_RULES), true)
> # Special build rule. This is a maintainer rule, that is only
> # available when GCC is configured with --enable-maintainer-mode. In
> --- gcc/ada/gcc-interface/Make-lang.in.jj 2020-07-28 15:39:09.470763625 +0200
> +++ gcc/ada/gcc-interface/Make-lang.in 2020-09-03 14:54:45.672695769 +0200
> @@ -148,7 +148,7 @@ endif
> ada: gnat1$(exeext) gnatbind$(exeext)
>
> # Tell GNU Make to ignore these, if they exist.
> -.PHONY: ada
> +.PHONY: ada ada.serial ada.prev
>
> # Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
> # we're linking against at least libcommon which contains C++ compiled code.
> @@ -666,11 +666,15 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
> # Needs to be built with CC=gcc
> # Since the RTL should be built with the latest compiler, remove the
> # stamp target in the parent directory whenever gnat1 is rebuilt
> -gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
> +GNAT1_DEPS = $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
> +gnat1$(exeext): $(GNAT1_DEPS)
> +$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
> libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
> $(RM) stamp-gnatlib2-rts stamp-tools
>
> +ada.serial: gnatbind$(exeext) ada.prev $(GNAT1_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) ada
> +
> gnatbind$(exeext): ada/b_gnatb.o $(CONFIG_H) $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBDEPS)
> +$(GCC_LINK) -o $@ ada/b_gnatb.o $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBS) $(SYSLIBS) $(CFLAGS)
>
> --- gcc/brig/Make-lang.in.jj 2020-01-12 11:54:36.193416728 +0100
> +++ gcc/brig/Make-lang.in 2020-09-03 14:56:26.067225541 +0200
> @@ -30,7 +30,7 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
> # The name for selecting brig in LANGUAGES.
> brig: brig1$(exeext)
>
> -.PHONY: brig
> +.PHONY: brig brig.serial brig.prev
>
> CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
>
> @@ -81,16 +81,15 @@ BRIG_OBJS = \
>
> brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
>
> -# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> -# +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> -# $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
> -
> -
> -brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +BRIG1_DEPS = $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +brig1$(exeext): $(BRIG1_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
> $(BACKENDLIBS)
>
> +brig.serial: brig.prev $(BRIG1_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) brig
> +
> # Documentation.
>
> BRIG_TEXI_FILES = \
> --- gcc/go/Make-lang.in.jj 2020-01-12 11:54:36.630410135 +0100
> +++ gcc/go/Make-lang.in 2020-09-03 14:57:04.455663361 +0200
> @@ -28,7 +28,7 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
> # The name for selecting go in LANGUAGES.
> go: go1$(exeext)
>
> -.PHONY: go
> +.PHONY: go go.serial go.prev
>
> CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
>
> @@ -78,10 +78,14 @@ GO_OBJS = \
>
> go_OBJS = $(GO_OBJS) go/gospec.o
>
> -go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +GO1_DEPS = $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +go1$(exeext): $(GO1_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> +go.serial: go.prev $(GO1_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) go
> +
> # Documentation.
>
> GO_TEXI_FILES = \
> --- gcc/jit/Make-lang.in.jj 2020-07-28 15:39:09.956756942 +0200
> +++ gcc/jit/Make-lang.in 2020-09-03 14:51:31.109545079 +0200
> @@ -48,8 +48,7 @@ ifneq (,$(findstring mingw,$(target)))
> LIBGCCJIT_FILENAME = libgccjit-$(LIBGCCJIT_VERSION_NUM).dll
> LIBGCCJIT_IMPORT_LIB = libgccjit.dll.a
>
> -jit: $(LIBGCCJIT_FILENAME) \
> - $(FULL_DRIVER_NAME)
> +jit.extra: $(FULL_DRIVER_NAME)
>
> else
>
> @@ -75,14 +74,16 @@ LIBGCCJIT_SONAME_OPTION = \
> $(if $(LD_SONAME_OPTION), \
> -Wl$(COMMA)$(LD_SONAME_OPTION)$(COMMA)$(LIBGCCJIT_SONAME))
>
> -jit: $(LIBGCCJIT_FILENAME) \
> - $(LIBGCCJIT_SYMLINK) \
> +jit.extra: $(LIBGCCJIT_SYMLINK) \
> $(LIBGCCJIT_LINKER_NAME_SYMLINK) \
> $(FULL_DRIVER_NAME)
> +
> endif
>
> +jit: $(LIBGCCJIT_FILENAME) jit.extra
> +
> # Tell GNU make to ignore these if they exist.
> -.PHONY: jit
> +.PHONY: jit jit.serial jit.prev jit.extra
>
> jit_OBJS = attribs.o \
> jit/dummy-frontend.o \
> @@ -113,17 +114,21 @@ endif
>
> # We avoid using $(BACKEND) from Makefile.in in order to avoid pulling
> # in main.o
> -$(LIBGCCJIT_FILENAME): $(jit_OBJS) \
> +LIBGCCJIT_FILENAME_DEPS = $(jit_OBJS) \
> libbackend.a libcommon-target.a libcommon.a \
> $(CPPLIB) $(LIBDECNUMBER) \
> $(LIBDEPS) $(srcdir)/jit/libgccjit.map \
> $(EXTRA_GCC_OBJS)
> +$(LIBGCCJIT_FILENAME): $(LIBGCCJIT_FILENAME_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
> $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
> $(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
> $(EXTRA_GCC_OBJS) \
> $(LIBGCCJIT_EXTRA_OPTS)
>
> +jit.serial: jit.prev $(LIBGCCJIT_FILENAME_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) jit
> +
> # Create symlinks when not building for Windows
> ifeq (,$(findstring mingw,$(target)))
> $(LIBGCCJIT_SONAME_SYMLINK): $(LIBGCCJIT_FILENAME)
> --- gcc/d/Make-lang.in.jj 2020-08-26 10:27:10.690552343 +0200
> +++ gcc/d/Make-lang.in 2020-09-03 14:52:31.675658110 +0200
> @@ -29,7 +29,7 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
> d: d21$(exeext)
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: d
> +.PHONY: d d.serial d.prev
>
> # Create the compiler driver for D.
> CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
> @@ -162,10 +162,14 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
>
> d_OBJS = $(D_ALL_OBJS) d/d-spec.o
>
> -d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +D21_DEPS = $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +d21$(exeext): $(D21_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> +d.serial: d.prev $(D21_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) d
> +
> # Documentation.
>
> D_TEXI_FILES = \
> --- gcc/objc/Make-lang.in.jj 2020-01-12 11:54:36.681409365 +0100
> +++ gcc/objc/Make-lang.in 2020-09-03 14:57:54.789926228 +0200
> @@ -40,7 +40,7 @@
> objc: cc1obj$(exeext)
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: objc
> +.PHONY: objc objc.serial objc.prev
>
> # Use maximal warnings for this front end.
> objc-warn = $(STRICT_WARN)
> @@ -62,11 +62,15 @@ cc1obj-checksum.c : build/genchecksum$(b
> $(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
> $(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
>
> -cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
> +CC1OBJ_DEPS = $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1obj$(exeext): $(CC1OBJ_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
> $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> +objc.serial: objc.prev $(CC1OBJ_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) objc
> +
> objc.srcextra:
>
> #\f
> --- gcc/objcp/Make-lang.in.jj 2020-01-12 11:54:36.685409305 +0100
> +++ gcc/objcp/Make-lang.in 2020-09-03 14:58:44.181202905 +0200
> @@ -41,7 +41,7 @@
> obj-c++: cc1objplus$(exeext)
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: obj-c++
> +.PHONY: obj-c++ obj-c++.serial obj-c++.prev
>
> # Use maximal warnings for this front end. Also, make ObjC and C++
> # headers accessible.
> @@ -66,10 +66,14 @@ cc1objplus-checksum.c : build/genchecksu
> $(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
> cc1objplus-checksum.c
>
> -cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
> +CC1OBJPLUS_DEPS = $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1objplus$(exeext): $(CC1OBJPLUS_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> +obj-c++.serial: obj-c++.prev $(CC1OBJPLUS_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) obj-c++
> +
> # Objective C++ language specific files.
>
> objcp/objc-runtime-shared-support.o: objc/objc-runtime-shared-support.c
> --- gcc/fortran/Make-lang.in.jj 2020-01-12 11:54:36.582410859 +0100
> +++ gcc/fortran/Make-lang.in 2020-09-03 14:53:48.550532310 +0200
> @@ -74,7 +74,7 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
> fortran: f951$(exeext)
>
> # Tell GNU make to ignore files by these names if they exist.
> -.PHONY: fortran
> +.PHONY: fortran fortran.serial fortran.prev
>
> CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
>
> @@ -92,12 +92,15 @@ gfortran-cross$(exeext): gfortran$(exeex
> cp gfortran$(exeext) gfortran-cross$(exeext)
>
> # The compiler itself is called f951.
> -f951$(exeext): $(F95_OBJS) \
> - $(BACKEND) $(LIBDEPS) attribs.o
> +F951_DEPS = $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o
> +f951$(exeext): $(F951_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
> $(BACKENDLIBS)
>
> +fortran.serial: fortran.prev $(F951_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) fortran
> +
> gt-fortran-trans.h : s-gtype; @true
> #\f
> # Build hooks:
> --- gcc/lto/Make-lang.in.jj 2020-01-12 11:54:36.677409426 +0100
> +++ gcc/lto/Make-lang.in 2020-09-03 14:45:51.957511831 +0200
> @@ -41,10 +41,14 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
>
> # Rules
>
> +lto: $(LTO_EXE) $(LTO_DUMP_EXE)
> +
> +.PHONY: lto lto.serial lto.prev
> +
> # These hooks are used by the main GCC Makefile. Consult that
> # Makefile for documentation.
> -lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
> -lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
> +lto.all.cross:
> +lto.start.encap:
> lto.rest.encap:
> lto.tags:
> lto.install-common: installdirs
> @@ -84,14 +88,20 @@ lto.stagefeedback:
> # Use strict warnings for this front end.
> lto-warn = $(STRICT_WARN)
>
> -$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
> +LTO_EXE_DEPS = $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
> +$(LTO_EXE): $(LTO_EXE_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
>
> -$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
> +LTO_DUMP_EXE_DEPS = $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
> +$(LTO_DUMP_EXE): $(LTO_DUMP_EXE_DEPS)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
>
> +lto.serial: lto.prev $(LTO_EXE_DEPS) $(LTO_DUMP_EXE_DEPS)
> + $(MAKE) $(FLAGS_TO_PASS) $(LTO_EXE)
> + $(MAKE) $(FLAGS_TO_PASS) $(LTO_DUMP_EXE)
> +
> lto/lto-dump.o: $(LTO_OBJS)
>
> lto1.fda: ../prev-gcc/lto1$(exeext) ../prev-gcc/$(PERF_DATA)
> --- gcc/configure.jj 2020-08-26 17:09:45.828254723 +0200
> +++ gcc/configure 2020-09-03 14:41:09.632647617 +0200
> @@ -710,6 +710,7 @@ subdirs
> dollar
> gcc_tooldir
> enable_lto
> +DO_LINK_SERIALIZATION
> DO_LINK_MUTEX
> MAINT
> zlibinc
> @@ -1012,6 +1013,7 @@ with_gc
> with_system_zlib
> enable_maintainer_mode
> enable_link_mutex
> +enable_link_serialization
> enable_version_specific_runtime_libs
> enable_plugin
> enable_host_shared
> @@ -1767,6 +1769,10 @@ Optional Features:
> sometimes confusing) to the casual installer
> --enable-link-mutex avoid linking multiple front-ends at once to avoid
> thrashing on the build machine
> + --enable-link-serialization
> + avoid linking multiple front-ends at once to avoid
> + thrashing on the build machine through make
> + dependencies
> --enable-version-specific-runtime-libs
> specify that runtime libraries should be installed
> in a compiler-specific directory
> @@ -19013,7 +19019,7 @@ else
> lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
> lt_status=$lt_dlunknown
> cat > conftest.$ac_ext <<_LT_EOF
> -#line 19016 "configure"
> +#line 19022 "configure"
> #include "confdefs.h"
>
> #if HAVE_DLFCN_H
> @@ -19119,7 +19125,7 @@ else
> lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
> lt_status=$lt_dlunknown
> cat > conftest.$ac_ext <<_LT_EOF
> -#line 19122 "configure"
> +#line 19128 "configure"
> #include "confdefs.h"
>
> #if HAVE_DLFCN_H
> @@ -30049,6 +30055,26 @@ else
> fi
>
>
> +
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
> +$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
> + # Check whether --enable-link-serialization was given.
> +if test "${enable_link_serialization+set}" = set; then :
> + enableval=$enable_link_serialization; do_link_serialization=$enableval
> +else
> + do_link_serialization=no
> +fi
> +
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
> +$as_echo "$do_link_serialization" >&6; }
> +
> +if test "$do_link_serialization" = "yes"; then
> + DO_LINK_SERIALIZATION=true
> +else
> + DO_LINK_SERIALIZATION=false
> +fi
> +
> +
> # --------------
> # Language hooks
> # --------------
> @@ -30209,6 +30235,14 @@ do
> echo "lang.$t: $x" >> Make-hooks
> done
>
> +prev=c
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev: $prev.serial" >> Make-hooks
> + prev=$lang
> +done
> +
> # --------
> # Option include files
> # --------
>
> Jakub
>
>
--
Richard Biener <rguenther@suse.de>
SUSE Software Solutions Germany GmbH, Maxfeldstrasse 5, 90409 Nuernberg,
Germany; GF: Felix Imendörffer; HRB 36809 (AG Nuernberg)
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH] --enable-link-serialization support
2020-09-03 13:53 ` Richard Biener
@ 2020-09-03 14:49 ` Jakub Jelinek
2020-09-04 8:32 ` Jakub Jelinek
2020-09-07 21:58 ` Jason Merrill
0 siblings, 2 replies; 6+ messages in thread
From: Jakub Jelinek @ 2020-09-03 14:49 UTC (permalink / raw)
To: Richard Biener; +Cc: gcc-patches
On Thu, Sep 03, 2020 at 03:53:35PM +0200, Richard Biener wrote:
> On Thu, 3 Sep 2020, Jakub Jelinek wrote:
> But is that an issue in practice? I usually do not do make -j32 cc1plus
> in a tree that was configured for bootstrap, nor do I use
> --enable-link-serialization in that case.
Guess most often true, but one could still do it when debugging some
particular problem during bootstrap.
> Btw, do you plan to keep --enable-link-mutex the way it is? What it
> provides ontop of --enable-link-serialization is that progress
> metering for very long running LTO link stages. Not sure if that
> works for the last LTO link or only when there's some other link
> waiting.
I kept it as is for now. Primarily I didn't want to use
--enable-linux-mutex for the new thing when it has nothing to do with a
mutex. I think with both options together it will just print useless
message that it acquired the lock immediately in each case.
And not really sure how and what should we print as progress indicator,
perhaps Make-hooks could contain in some variable number of the languages
(well, one can use $(words $(CONFIG_LANGUAGES)) for that purpose)
and in other vars record for each FE its serial number) and then add
@echo Linking C++ - $(c++.idx) out of $(words $(CONFIG_LANGUAGES))
right before the $(LLINK) lines, perhaps only added if $(DO_LINK_SERIALIZATION)
is true. Or perhaps do some ANSI colors progress bar for that too.
Anyway, here is the updated patch that doesn't use recursive make for that,
I think that could run into latent Makefile dependency issues and the like.
For new FEs, the new requirements are basically that one should add
$lang.serial $lang.prev to .PHONY, make $lang.serial depend on the binary
linked with $(LLINKER) and add $lang.prev to the dependencies of that
binary. Unless, as in the lto special case, there are multiple such
binaries, then $lang.serial should depend on the last one, $lang.prev
should be the dependy of the first one and there should be dependencies
between the binaries guarded on $(DO_LINK_SERIALIZATION) equal to true.
2020-09-03 Jakub Jelinek <jakub@redhat.com>
* configure.ac (--enable-link-serialization): New configure option.
* Makefile.tpl (EXTRA_GCC_FLAGS): Pass DO_LINK_SERIALIZATION=true
or false to recursive make in gcc subdirectory.
* configure: Regenerated.
* Makefile.in: Regenerated.
gcc/
* configure.ac: Add $lang.prev: $prev.serial or $lang.prev: rules to
Make-hooks.
* doc/install.texi (--enable-link-serialization): Document.
* configure: Regenerated.
gcc/c/
* Make-lang.in (c.serial): New goal.
(.PHONY): Add c.serial c.prev.
gcc/cp/
* Make-lang.in (c++.serial): New goal.
(.PHONY): Add c++.serial c++.prev.
(cc1plus$(exeext)): Depend on c++.prev.
gcc/fortran/
* Make-lang.in (fortran.serial): New goal.
(.PHONY): Add fortran.serial fortran.prev.
(f951$(exeext)): Depend on fortran.prev.
gcc/lto/
* Make-lang.in (lto, lto.serial): New goals.
(.PHONY): Add fortran.serial fortran.prev.
(lto.all.cross, lto.start.encap): Remove dependencies.
($(LTO_EXE)): Depend on lto.prev.
(LTO_DUMP_EXE_PREV): New variable.
($(LTO_DUMP_EXE)): Depend on $(LTO_DUMP_EXE_PREV).
gcc/objc/
* Make-lang.in (objc.serial): New goal.
(.PHONY): Add objc.serial objc.prev.
(cc1obj$(exeext)): Depend on objc.prev.
gcc/objcp/
* Make-lang.in (obj-c++.serial): New goal.
(.PHONY): Add obj-c++.serial obj-c++.prev.
(cc1objplus$(exeext)): Depend on obj-c++.prev.
gcc/ada/
* gcc-interface/Make-lang.in (ada.serial): New goal.
(.PHONY): Add ada.serial ada.prev.
(gnat1$(exeext)): Depend on ada.prev.
gcc/brig/
* Make-lang.in (brig.serial): New goal.
(.PHONY): Add brig.serial brig.prev.
(brig1$(exeext)): Depend on brig.prev.
gcc/go/
* Make-lang.in (go.serial): New goal.
(.PHONY): Add go.serial go.prev.
(go1$(exeext)): Depend on go.prev.
gcc/jit/
* Make-lang.in (jit.serial): New goal.
(.PHONY): Add jit.serial jit.prev.
($(LIBGCCJIT_FILENAME)): Depend on jit.prev.
gcc/d/
* Make-lang.in (d.serial): New goal.
(.PHONY): Add d.serial d.prev.
(d21$(exeext)): Depend on d.prev.
--- configure.ac.jj 2020-08-24 10:00:01.248259486 +0200
+++ configure.ac 2020-09-03 16:09:51.010782912 +0200
@@ -1866,6 +1866,24 @@ AC_ARG_ENABLE(linker-plugin-flags,
extra_linker_plugin_flags=)
AC_SUBST(extra_linker_plugin_flags)
+dnl Whether to prevent multiple GCC front-ends from linking at the same time
+
+AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
+ AC_ARG_ENABLE(link-serialization,
+[AS_HELP_STRING([--enable-link-serialization],
+ [avoid linking multiple GCC front-ends at once using make
+ dependencies to avoid thrashing on the build machine])],
+ do_link_serialization=$enableval,
+ do_link_serialization=no)
+AC_MSG_RESULT($do_link_serialization)
+
+if test "$do_link_serialization" = "yes"; then
+ DO_LINK_SERIALIZATION=true
+else
+ DO_LINK_SERIALIZATION=false
+fi
+AC_SUBST(DO_LINK_SERIALIZATION)
+
# Enable --enable-host-shared.
# Checked early to determine whether jit is an 'all' language
AC_ARG_ENABLE(host-shared,
--- Makefile.tpl.jj 2020-01-12 11:54:35.751423396 +0100
+++ Makefile.tpl 2020-09-03 16:14:03.381097293 +0200
@@ -734,7 +734,8 @@ TARGET_FLAGS_TO_PASS = $(BASE_FLAGS_TO_P
EXTRA_GCC_FLAGS = \
"GCC_FOR_TARGET=$(GCC_FOR_TARGET)" \
"`echo 'STMP_FIXPROTO=$(STMP_FIXPROTO)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
- "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
+ "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
+ "DO_LINK_SERIALIZATION=@DO_LINK_SERIALIZATION@"
GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
--- gcc/configure.ac.jj 2020-08-26 17:09:45.829254709 +0200
+++ gcc/configure.ac 2020-09-03 16:05:26.985638746 +0200
@@ -6540,6 +6540,22 @@ do
echo "lang.$t: $x" >> Make-hooks
done
+echo "ifeq (\$(DO_LINK_SERIALIZATION),true)" >> Make-hooks
+prev=c
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: $prev.serial" >> Make-hooks
+ prev=$lang
+done
+echo else >> Make-hooks
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev:" >> Make-hooks
+done
+echo endif >> Make-hooks
+
# --------
# Option include files
# --------
--- gcc/c/Make-lang.in.jj 2020-07-28 15:39:09.669760889 +0200
+++ gcc/c/Make-lang.in 2020-09-03 14:24:41.861122758 +0200
@@ -37,9 +37,10 @@
#\f
# Define the names for selecting c in LANGUAGES.
c: cc1$(exeext)
+c.serial: c
# Tell GNU make to ignore these if they exist.
-.PHONY: c gcc
+.PHONY: c gcc c.serial
# The C front end driver. This is different from the drivers for other
# front ends, because there is no C language specific driver (i.e. nothing
--- gcc/cp/Make-lang.in.jj 2020-07-28 15:39:09.770759500 +0200
+++ gcc/cp/Make-lang.in 2020-09-03 15:46:34.413189898 +0200
@@ -47,9 +47,10 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
# into the C++ rule, but that needs a little bit of work
# to do the right thing within all.cross.
c++: cc1plus$(exeext)
+c++.serial: c++
# Tell GNU make to ignore these if they exist.
-.PHONY: c++
+.PHONY: c++ c++.serial c++.prev
CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
@@ -116,7 +117,7 @@ cc1plus-checksum.c : build/genchecksum$(
$(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
fi
-cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) c++.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/fortran/Make-lang.in.jj 2020-01-12 11:54:36.582410859 +0100
+++ gcc/fortran/Make-lang.in 2020-09-03 15:52:35.805901622 +0200
@@ -72,9 +72,10 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
#\f
# Define the names for selecting gfortran in LANGUAGES.
fortran: f951$(exeext)
+fortran.serial: fortran
# Tell GNU make to ignore files by these names if they exist.
-.PHONY: fortran
+.PHONY: fortran fortran.serial fortran.prev
CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
@@ -92,8 +93,7 @@ gfortran-cross$(exeext): gfortran$(exeex
cp gfortran$(exeext) gfortran-cross$(exeext)
# The compiler itself is called f951.
-f951$(exeext): $(F95_OBJS) \
- $(BACKEND) $(LIBDEPS) attribs.o
+f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o fortran.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
$(BACKENDLIBS)
--- gcc/lto/Make-lang.in.jj 2020-01-12 11:54:36.677409426 +0100
+++ gcc/lto/Make-lang.in 2020-09-03 16:04:44.140264456 +0200
@@ -41,10 +41,15 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
# Rules
+lto: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto.serial: $(LTO_DUMP_EXE)
+
+.PHONY: lto lto.serial lto.prev
+
# These hooks are used by the main GCC Makefile. Consult that
# Makefile for documentation.
-lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
-lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto.all.cross:
+lto.start.encap:
lto.rest.encap:
lto.tags:
lto.install-common: installdirs
@@ -84,11 +89,16 @@ lto.stagefeedback:
# Use strict warnings for this front end.
lto-warn = $(STRICT_WARN)
-$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) lto.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
-$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
+ifeq ($(DO_LINK_SERIALIZATION),true)
+LTO_DUMP_EXE_PREV = $(LTO_EXE)
+else
+LTO_DUMP_EXE_PREV =
+endif
+$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) $(LTO_DUMP_EXE_PREV)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
--- gcc/objc/Make-lang.in.jj 2020-01-12 11:54:36.681409365 +0100
+++ gcc/objc/Make-lang.in 2020-09-03 15:50:47.542485854 +0200
@@ -38,9 +38,10 @@
#\f
# Define the names for selecting Objective-C in LANGUAGES.
objc: cc1obj$(exeext)
+objc.serial: objc
# Tell GNU make to ignore these if they exist.
-.PHONY: objc
+.PHONY: objc objc.serial objc.prev
# Use maximal warnings for this front end.
objc-warn = $(STRICT_WARN)
@@ -62,7 +63,8 @@ cc1obj-checksum.c : build/genchecksum$(b
$(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
$(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
-cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
+cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \
+ $(LIBDEPS) objc.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
$(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/objcp/Make-lang.in.jj 2020-01-12 11:54:36.685409305 +0100
+++ gcc/objcp/Make-lang.in 2020-09-03 15:49:40.975459927 +0200
@@ -39,9 +39,10 @@
#\f
# Define the names for selecting Objective-C++ in LANGUAGES.
obj-c++: cc1objplus$(exeext)
+obj-c++.serial: obj-c++
# Tell GNU make to ignore these if they exist.
-.PHONY: obj-c++
+.PHONY: obj-c++ obj-c++.serial obj-c++.prev
# Use maximal warnings for this front end. Also, make ObjC and C++
# headers accessible.
@@ -66,7 +67,8 @@ cc1objplus-checksum.c : build/genchecksu
$(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
cc1objplus-checksum.c
-cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) \
+ $(LIBDEPS) obj-c++.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/ada/gcc-interface/Make-lang.in.jj 2020-07-28 15:39:09.470763625 +0200
+++ gcc/ada/gcc-interface/Make-lang.in 2020-09-03 15:55:04.140734768 +0200
@@ -146,9 +146,10 @@ endif
# Define the names for selecting Ada in LANGUAGES.
ada: gnat1$(exeext) gnatbind$(exeext)
+ada.serial: gnat1$(exeext)
# Tell GNU Make to ignore these, if they exist.
-.PHONY: ada
+.PHONY: ada ada.serial ada.prev
# Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
# we're linking against at least libcommon which contains C++ compiled code.
@@ -666,7 +667,8 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
# Needs to be built with CC=gcc
# Since the RTL should be built with the latest compiler, remove the
# stamp target in the parent directory whenever gnat1 is rebuilt
-gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
+gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a \
+ $(LIBDEPS) ada.prev
+$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
$(RM) stamp-gnatlib2-rts stamp-tools
--- gcc/brig/Make-lang.in.jj 2020-01-12 11:54:36.193416728 +0100
+++ gcc/brig/Make-lang.in 2020-09-03 15:53:16.563305843 +0200
@@ -29,8 +29,9 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
# The name for selecting brig in LANGUAGES.
brig: brig1$(exeext)
+brig.serial: brig
-.PHONY: brig
+.PHONY: brig brig.serial brig.prev
CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
@@ -81,12 +82,7 @@ BRIG_OBJS = \
brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
-# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
-# +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
-# $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
-
-
-brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) brig.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
$(BACKENDLIBS)
--- gcc/go/Make-lang.in.jj 2020-01-12 11:54:36.630410135 +0100
+++ gcc/go/Make-lang.in 2020-09-03 15:54:02.497635018 +0200
@@ -27,8 +27,9 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
# The name for selecting go in LANGUAGES.
go: go1$(exeext)
+go.serial: go
-.PHONY: go
+.PHONY: go go.serial go.prev
CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
@@ -78,7 +79,7 @@ GO_OBJS = \
go_OBJS = $(GO_OBJS) go/gospec.o
-go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) go.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/jit/Make-lang.in.jj 2020-07-28 15:39:09.956756942 +0200
+++ gcc/jit/Make-lang.in 2020-09-03 16:00:58.490559851 +0200
@@ -81,8 +81,10 @@ jit: $(LIBGCCJIT_FILENAME) \
$(FULL_DRIVER_NAME)
endif
+jit.serial: $(LIBGCCJIT_FILENAME)
+
# Tell GNU make to ignore these if they exist.
-.PHONY: jit
+.PHONY: jit jit.serial jit.prev
jit_OBJS = attribs.o \
jit/dummy-frontend.o \
@@ -117,7 +119,7 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) \
$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
- $(EXTRA_GCC_OBJS)
+ $(EXTRA_GCC_OBJS) jit.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
$(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
--- gcc/d/Make-lang.in.jj 2020-08-26 10:27:10.690552343 +0200
+++ gcc/d/Make-lang.in 2020-09-03 15:48:15.010717845 +0200
@@ -27,9 +27,10 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
# The name for selecting d in LANGUAGES.
d: d21$(exeext)
+d.serial: d
# Tell GNU make to ignore these if they exist.
-.PHONY: d
+.PHONY: d d.serial d.prev
# Create the compiler driver for D.
CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
@@ -162,7 +163,7 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
d_OBJS = $(D_ALL_OBJS) d/d-spec.o
-d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) d.prev
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
--- gcc/doc/install.texi.jj 2020-08-25 07:15:43.982903777 +0200
+++ gcc/doc/install.texi 2020-09-03 16:16:41.403789530 +0200
@@ -1533,6 +1533,13 @@ When building GCC, use a mutex to avoid
multiple languages at the same time, to avoid thrashing on build
systems with limited free memory. The default is not to use such a mutex.
+@item --enable-link-serialization
+When building GCC, use make dependencies to serialize linking the compilers for
+multiple languages, to avoid thrashing on build
+systems with limited free memory. The default is not to add such
+dependencies and thus with parallel make potentially link different
+compilers concurrently.
+
@item --enable-maintainer-mode
The build rules that regenerate the Autoconf and Automake output files as
well as the GCC master message catalog @file{gcc.pot} are normally
--- configure.jj 2020-08-24 10:00:01.233259699 +0200
+++ configure 2020-09-03 16:09:58.121679071 +0200
@@ -680,6 +680,7 @@ extra_host_zlib_configure_flags
extra_host_libiberty_configure_flags
stage1_languages
host_shared
+DO_LINK_SERIALIZATION
extra_linker_plugin_flags
extra_linker_plugin_configure_flags
islinc
@@ -819,6 +820,7 @@ enable_isl_version_check
enable_lto
enable_linker_plugin_configure_flags
enable_linker_plugin_flags
+enable_link_serialization
enable_host_shared
enable_stage1_languages
enable_objc_gc
@@ -1538,6 +1540,10 @@ Optional Features:
--enable-linker-plugin-flags=FLAGS
additional flags for configuring and building linker
plugins [none]
+ --enable-link-serialization
+ avoid linking multiple GCC front-ends at once using
+ make dependencies to avoid thrashing on the build
+ machine
--enable-host-shared build host code as shared libraries
--enable-stage1-languages[=all]
choose additional languages to build during stage1.
@@ -8264,6 +8270,26 @@ fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
+$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
+ # Check whether --enable-link-serialization was given.
+if test "${enable_link_serialization+set}" = set; then :
+ enableval=$enable_link_serialization; do_link_serialization=$enableval
+else
+ do_link_serialization=no
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
+$as_echo "$do_link_serialization" >&6; }
+
+if test "$do_link_serialization" = "yes"; then
+ DO_LINK_SERIALIZATION=true
+else
+ DO_LINK_SERIALIZATION=false
+fi
+
+
# Enable --enable-host-shared.
# Checked early to determine whether jit is an 'all' language
# Check whether --enable-host-shared was given.
--- Makefile.in.jj 2020-04-09 21:19:04.790224960 +0200
+++ Makefile.in 2020-09-03 16:14:29.543715216 +0200
@@ -979,7 +979,8 @@ TARGET_FLAGS_TO_PASS = $(BASE_FLAGS_TO_P
EXTRA_GCC_FLAGS = \
"GCC_FOR_TARGET=$(GCC_FOR_TARGET)" \
"`echo 'STMP_FIXPROTO=$(STMP_FIXPROTO)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
- "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
+ "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
+ "DO_LINK_SERIALIZATION=@DO_LINK_SERIALIZATION@"
GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
--- gcc/configure.jj 2020-08-26 17:09:45.828254723 +0200
+++ gcc/configure 2020-09-03 16:05:40.070447656 +0200
@@ -30209,6 +30209,22 @@ do
echo "lang.$t: $x" >> Make-hooks
done
+echo "ifeq (\$(DO_LINK_SERIALIZATION),true)" >> Make-hooks
+prev=c
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: $prev.serial" >> Make-hooks
+ prev=$lang
+done
+echo else >> Make-hooks
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev:" >> Make-hooks
+done
+echo endif >> Make-hooks
+
# --------
# Option include files
# --------
Jakub
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] --enable-link-serialization support
2020-09-03 14:49 ` [PATCH] " Jakub Jelinek
@ 2020-09-04 8:32 ` Jakub Jelinek
2020-09-07 21:58 ` Jason Merrill
1 sibling, 0 replies; 6+ messages in thread
From: Jakub Jelinek @ 2020-09-04 8:32 UTC (permalink / raw)
To: Paolo Bonzini, Nathanael Nerode, Alexandre Oliva, Ralf Wildenhues
Cc: gcc-patches, Richard Biener
Hi!
CCing build maintainers now.
On Thu, Sep 03, 2020 at 04:49:09PM +0200, Jakub Jelinek via Gcc-patches wrote:
> On Thu, Sep 03, 2020 at 03:53:35PM +0200, Richard Biener wrote:
> > No review on the actual patch - it looks like what I'd have tried
> > but I'm no make expert ;)
Successfully bootstrapped/regtested now, once on x86_64-linux with:
../configure --enable-languages=default,ada,obj-c++,lto,go,brig,d --enable-link-serialization --with-build-config=bootstrap-lto --enable-checking=yes,rtl,extra && make -j16 bootstrap > LOG 2>&1 && GXX_TESTSUITE_STDS=98,11,14,17,2a make -j32 -k check > LOGC 2>&1; ../contrib/test_summary > LOGT 2>&1
i.e. with LTO bootstrap and this serialization on and verified that all the
large program LTO links were serialized, and normally on x86_64-linux
and i686-linux (i.e. with serialization turned off, non-lto bootstraps)
and verified that all the links were concurrent.
Ok for trunk?
If we wanted to go further, it could have also arguments,
--disable-link-serialization (the default, and =no, and =0) would mean allow all LLINKERs
concurrently,
--enable-link-serialization (and =yes, =1) would serialize and
--enable-link-serialization=N for N > 0 would ensure at most N concurrent
LLINKER jobs.
This could be done by adjusting the patch, so that configure knows there are
two jobs for lto and use lto1.{prev,serial} and lto2.{prev,serial}, and do
emit into Make-hooks:
ifeq ($(DO_LINK_SERIALIZATION),)
SERIAL_LIST =
else
# Note, configure would put the list sorted backwards and without the last one (obj-c++.serial in my case)
SERIAL_LIST = $(wordlist $(DO_LINK_SERIALIZATION),10,objc.serial lto2.serial lto1.serial go.serial fortran.serial d.serial c++.serial brig.serial ada.serial c.serial)
endif
ada.prev: $(word 10,$(SERIAL_LIST))
brig.prev: $(word 9,$(SERIAL_LIST))
c++.prev: $(word 8,$(SERIAL_LIST))
d.prev: $(word 7,$(SERIAL_LIST))
fortran.prev: $(word 6,$(SERIAL_LIST))
go.prev: $(word 5,$(SERIAL_LIST))
lto1.prev: $(word 4,$(SERIAL_LIST))
lto2.prev: $(word 3,$(SERIAL_LIST))
objc.prev: $(word 2,$(SERIAL_LIST))
obj-c++.prev: $(word 1,$(SERIAL_LIST))
The toplevel would then pass DO_LINK_SERIALIZATION=1024 or something
similarly high (or DO_LINK_SERIALIZATION= ) for the default
--disable-link-serialization case, 1 for the yes or 1 case etc.
I think this would arrange the same thing as the patch does now for
the DO_LINK_SERIALIZATION= and DO_LINK_SERIALIZATION=1 (currently false and
true) cases, ada.prev would depend on c.serial, brig.prev on ada.prev etc.,
but e.g. for =2 ada.prev wouldn't depend on anything (i.e. it can be linked
concurretly with c), brig.prev would depend on c.prev (so it needs to wait
until cc1 is done), c++.prev would depend on ada.prev (so it would wait
until gnat1 is done), etc.
> > On Thu, 3 Sep 2020, Jakub Jelinek wrote:
> > But is that an issue in practice? I usually do not do make -j32 cc1plus
> > in a tree that was configured for bootstrap, nor do I use
> > --enable-link-serialization in that case.
>
> Guess most often true, but one could still do it when debugging some
> particular problem during bootstrap.
>
> > Btw, do you plan to keep --enable-link-mutex the way it is? What it
> > provides ontop of --enable-link-serialization is that progress
> > metering for very long running LTO link stages. Not sure if that
> > works for the last LTO link or only when there's some other link
> > waiting.
>
> I kept it as is for now. Primarily I didn't want to use
> --enable-linux-mutex for the new thing when it has nothing to do with a
> mutex. I think with both options together it will just print useless
> message that it acquired the lock immediately in each case.
> And not really sure how and what should we print as progress indicator,
> perhaps Make-hooks could contain in some variable number of the languages
> (well, one can use $(words $(CONFIG_LANGUAGES)) for that purpose)
> and in other vars record for each FE its serial number) and then add
> @echo Linking C++ - $(c++.idx) out of $(words $(CONFIG_LANGUAGES))
> right before the $(LLINK) lines, perhaps only added if $(DO_LINK_SERIALIZATION)
> is true. Or perhaps do some ANSI colors progress bar for that too.
>
> Anyway, here is the updated patch that doesn't use recursive make for that,
> I think that could run into latent Makefile dependency issues and the like.
>
> For new FEs, the new requirements are basically that one should add
> $lang.serial $lang.prev to .PHONY, make $lang.serial depend on the binary
> linked with $(LLINKER) and add $lang.prev to the dependencies of that
> binary. Unless, as in the lto special case, there are multiple such
> binaries, then $lang.serial should depend on the last one, $lang.prev
> should be the dependy of the first one and there should be dependencies
> between the binaries guarded on $(DO_LINK_SERIALIZATION) equal to true.
>
> 2020-09-03 Jakub Jelinek <jakub@redhat.com>
>
> * configure.ac (--enable-link-serialization): New configure option.
> * Makefile.tpl (EXTRA_GCC_FLAGS): Pass DO_LINK_SERIALIZATION=true
> or false to recursive make in gcc subdirectory.
> * configure: Regenerated.
> * Makefile.in: Regenerated.
> gcc/
> * configure.ac: Add $lang.prev: $prev.serial or $lang.prev: rules to
> Make-hooks.
> * doc/install.texi (--enable-link-serialization): Document.
> * configure: Regenerated.
> gcc/c/
> * Make-lang.in (c.serial): New goal.
> (.PHONY): Add c.serial c.prev.
> gcc/cp/
> * Make-lang.in (c++.serial): New goal.
> (.PHONY): Add c++.serial c++.prev.
> (cc1plus$(exeext)): Depend on c++.prev.
> gcc/fortran/
> * Make-lang.in (fortran.serial): New goal.
> (.PHONY): Add fortran.serial fortran.prev.
> (f951$(exeext)): Depend on fortran.prev.
> gcc/lto/
> * Make-lang.in (lto, lto.serial): New goals.
> (.PHONY): Add fortran.serial fortran.prev.
> (lto.all.cross, lto.start.encap): Remove dependencies.
> ($(LTO_EXE)): Depend on lto.prev.
> (LTO_DUMP_EXE_PREV): New variable.
> ($(LTO_DUMP_EXE)): Depend on $(LTO_DUMP_EXE_PREV).
> gcc/objc/
> * Make-lang.in (objc.serial): New goal.
> (.PHONY): Add objc.serial objc.prev.
> (cc1obj$(exeext)): Depend on objc.prev.
> gcc/objcp/
> * Make-lang.in (obj-c++.serial): New goal.
> (.PHONY): Add obj-c++.serial obj-c++.prev.
> (cc1objplus$(exeext)): Depend on obj-c++.prev.
> gcc/ada/
> * gcc-interface/Make-lang.in (ada.serial): New goal.
> (.PHONY): Add ada.serial ada.prev.
> (gnat1$(exeext)): Depend on ada.prev.
> gcc/brig/
> * Make-lang.in (brig.serial): New goal.
> (.PHONY): Add brig.serial brig.prev.
> (brig1$(exeext)): Depend on brig.prev.
> gcc/go/
> * Make-lang.in (go.serial): New goal.
> (.PHONY): Add go.serial go.prev.
> (go1$(exeext)): Depend on go.prev.
> gcc/jit/
> * Make-lang.in (jit.serial): New goal.
> (.PHONY): Add jit.serial jit.prev.
> ($(LIBGCCJIT_FILENAME)): Depend on jit.prev.
> gcc/d/
> * Make-lang.in (d.serial): New goal.
> (.PHONY): Add d.serial d.prev.
> (d21$(exeext)): Depend on d.prev.
>
> --- configure.ac.jj 2020-08-24 10:00:01.248259486 +0200
> +++ configure.ac 2020-09-03 16:09:51.010782912 +0200
> @@ -1866,6 +1866,24 @@ AC_ARG_ENABLE(linker-plugin-flags,
> extra_linker_plugin_flags=)
> AC_SUBST(extra_linker_plugin_flags)
>
> +dnl Whether to prevent multiple GCC front-ends from linking at the same time
> +
> +AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
> + AC_ARG_ENABLE(link-serialization,
> +[AS_HELP_STRING([--enable-link-serialization],
> + [avoid linking multiple GCC front-ends at once using make
> + dependencies to avoid thrashing on the build machine])],
> + do_link_serialization=$enableval,
> + do_link_serialization=no)
> +AC_MSG_RESULT($do_link_serialization)
> +
> +if test "$do_link_serialization" = "yes"; then
> + DO_LINK_SERIALIZATION=true
> +else
> + DO_LINK_SERIALIZATION=false
> +fi
> +AC_SUBST(DO_LINK_SERIALIZATION)
> +
> # Enable --enable-host-shared.
> # Checked early to determine whether jit is an 'all' language
> AC_ARG_ENABLE(host-shared,
> --- Makefile.tpl.jj 2020-01-12 11:54:35.751423396 +0100
> +++ Makefile.tpl 2020-09-03 16:14:03.381097293 +0200
> @@ -734,7 +734,8 @@ TARGET_FLAGS_TO_PASS = $(BASE_FLAGS_TO_P
> EXTRA_GCC_FLAGS = \
> "GCC_FOR_TARGET=$(GCC_FOR_TARGET)" \
> "`echo 'STMP_FIXPROTO=$(STMP_FIXPROTO)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
> - "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
> + "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
> + "DO_LINK_SERIALIZATION=@DO_LINK_SERIALIZATION@"
>
> GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
>
> --- gcc/configure.ac.jj 2020-08-26 17:09:45.829254709 +0200
> +++ gcc/configure.ac 2020-09-03 16:05:26.985638746 +0200
> @@ -6540,6 +6540,22 @@ do
> echo "lang.$t: $x" >> Make-hooks
> done
>
> +echo "ifeq (\$(DO_LINK_SERIALIZATION),true)" >> Make-hooks
> +prev=c
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev: $prev.serial" >> Make-hooks
> + prev=$lang
> +done
> +echo else >> Make-hooks
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev:" >> Make-hooks
> +done
> +echo endif >> Make-hooks
> +
> # --------
> # Option include files
> # --------
> --- gcc/c/Make-lang.in.jj 2020-07-28 15:39:09.669760889 +0200
> +++ gcc/c/Make-lang.in 2020-09-03 14:24:41.861122758 +0200
> @@ -37,9 +37,10 @@
> #\f
> # Define the names for selecting c in LANGUAGES.
> c: cc1$(exeext)
> +c.serial: c
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: c gcc
> +.PHONY: c gcc c.serial
>
> # The C front end driver. This is different from the drivers for other
> # front ends, because there is no C language specific driver (i.e. nothing
> --- gcc/cp/Make-lang.in.jj 2020-07-28 15:39:09.770759500 +0200
> +++ gcc/cp/Make-lang.in 2020-09-03 15:46:34.413189898 +0200
> @@ -47,9 +47,10 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
> # into the C++ rule, but that needs a little bit of work
> # to do the right thing within all.cross.
> c++: cc1plus$(exeext)
> +c++.serial: c++
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: c++
> +.PHONY: c++ c++.serial c++.prev
>
> CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
>
> @@ -116,7 +117,7 @@ cc1plus-checksum.c : build/genchecksum$(
> $(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
> fi
>
> -cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) c++.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> --- gcc/fortran/Make-lang.in.jj 2020-01-12 11:54:36.582410859 +0100
> +++ gcc/fortran/Make-lang.in 2020-09-03 15:52:35.805901622 +0200
> @@ -72,9 +72,10 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
> #\f
> # Define the names for selecting gfortran in LANGUAGES.
> fortran: f951$(exeext)
> +fortran.serial: fortran
>
> # Tell GNU make to ignore files by these names if they exist.
> -.PHONY: fortran
> +.PHONY: fortran fortran.serial fortran.prev
>
> CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
>
> @@ -92,8 +93,7 @@ gfortran-cross$(exeext): gfortran$(exeex
> cp gfortran$(exeext) gfortran-cross$(exeext)
>
> # The compiler itself is called f951.
> -f951$(exeext): $(F95_OBJS) \
> - $(BACKEND) $(LIBDEPS) attribs.o
> +f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o fortran.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
> $(BACKENDLIBS)
> --- gcc/lto/Make-lang.in.jj 2020-01-12 11:54:36.677409426 +0100
> +++ gcc/lto/Make-lang.in 2020-09-03 16:04:44.140264456 +0200
> @@ -41,10 +41,15 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
>
> # Rules
>
> +lto: $(LTO_EXE) $(LTO_DUMP_EXE)
> +lto.serial: $(LTO_DUMP_EXE)
> +
> +.PHONY: lto lto.serial lto.prev
> +
> # These hooks are used by the main GCC Makefile. Consult that
> # Makefile for documentation.
> -lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
> -lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
> +lto.all.cross:
> +lto.start.encap:
> lto.rest.encap:
> lto.tags:
> lto.install-common: installdirs
> @@ -84,11 +89,16 @@ lto.stagefeedback:
> # Use strict warnings for this front end.
> lto-warn = $(STRICT_WARN)
>
> -$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
> +$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) lto.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
>
> -$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
> +ifeq ($(DO_LINK_SERIALIZATION),true)
> +LTO_DUMP_EXE_PREV = $(LTO_EXE)
> +else
> +LTO_DUMP_EXE_PREV =
> +endif
> +$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) $(LTO_DUMP_EXE_PREV)
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
>
> --- gcc/objc/Make-lang.in.jj 2020-01-12 11:54:36.681409365 +0100
> +++ gcc/objc/Make-lang.in 2020-09-03 15:50:47.542485854 +0200
> @@ -38,9 +38,10 @@
> #\f
> # Define the names for selecting Objective-C in LANGUAGES.
> objc: cc1obj$(exeext)
> +objc.serial: objc
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: objc
> +.PHONY: objc objc.serial objc.prev
>
> # Use maximal warnings for this front end.
> objc-warn = $(STRICT_WARN)
> @@ -62,7 +63,8 @@ cc1obj-checksum.c : build/genchecksum$(b
> $(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
> $(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
>
> -cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \
> + $(LIBDEPS) objc.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
> $(BACKEND) $(LIBS) $(BACKENDLIBS)
> --- gcc/objcp/Make-lang.in.jj 2020-01-12 11:54:36.685409305 +0100
> +++ gcc/objcp/Make-lang.in 2020-09-03 15:49:40.975459927 +0200
> @@ -39,9 +39,10 @@
> #\f
> # Define the names for selecting Objective-C++ in LANGUAGES.
> obj-c++: cc1objplus$(exeext)
> +obj-c++.serial: obj-c++
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: obj-c++
> +.PHONY: obj-c++ obj-c++.serial obj-c++.prev
>
> # Use maximal warnings for this front end. Also, make ObjC and C++
> # headers accessible.
> @@ -66,7 +67,8 @@ cc1objplus-checksum.c : build/genchecksu
> $(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
> cc1objplus-checksum.c
>
> -cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
> +cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) \
> + $(LIBDEPS) obj-c++.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> --- gcc/ada/gcc-interface/Make-lang.in.jj 2020-07-28 15:39:09.470763625 +0200
> +++ gcc/ada/gcc-interface/Make-lang.in 2020-09-03 15:55:04.140734768 +0200
> @@ -146,9 +146,10 @@ endif
>
> # Define the names for selecting Ada in LANGUAGES.
> ada: gnat1$(exeext) gnatbind$(exeext)
> +ada.serial: gnat1$(exeext)
>
> # Tell GNU Make to ignore these, if they exist.
> -.PHONY: ada
> +.PHONY: ada ada.serial ada.prev
>
> # Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
> # we're linking against at least libcommon which contains C++ compiled code.
> @@ -666,7 +667,8 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
> # Needs to be built with CC=gcc
> # Since the RTL should be built with the latest compiler, remove the
> # stamp target in the parent directory whenever gnat1 is rebuilt
> -gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
> +gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a \
> + $(LIBDEPS) ada.prev
> +$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
> libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
> $(RM) stamp-gnatlib2-rts stamp-tools
> --- gcc/brig/Make-lang.in.jj 2020-01-12 11:54:36.193416728 +0100
> +++ gcc/brig/Make-lang.in 2020-09-03 15:53:16.563305843 +0200
> @@ -29,8 +29,9 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
>
> # The name for selecting brig in LANGUAGES.
> brig: brig1$(exeext)
> +brig.serial: brig
>
> -.PHONY: brig
> +.PHONY: brig brig.serial brig.prev
>
> CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
>
> @@ -81,12 +82,7 @@ BRIG_OBJS = \
>
> brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
>
> -# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> -# +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> -# $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
> -
> -
> -brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) brig.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
> $(BACKENDLIBS)
> --- gcc/go/Make-lang.in.jj 2020-01-12 11:54:36.630410135 +0100
> +++ gcc/go/Make-lang.in 2020-09-03 15:54:02.497635018 +0200
> @@ -27,8 +27,9 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
>
> # The name for selecting go in LANGUAGES.
> go: go1$(exeext)
> +go.serial: go
>
> -.PHONY: go
> +.PHONY: go go.serial go.prev
>
> CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
>
> @@ -78,7 +79,7 @@ GO_OBJS = \
>
> go_OBJS = $(GO_OBJS) go/gospec.o
>
> -go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) go.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> --- gcc/jit/Make-lang.in.jj 2020-07-28 15:39:09.956756942 +0200
> +++ gcc/jit/Make-lang.in 2020-09-03 16:00:58.490559851 +0200
> @@ -81,8 +81,10 @@ jit: $(LIBGCCJIT_FILENAME) \
> $(FULL_DRIVER_NAME)
> endif
>
> +jit.serial: $(LIBGCCJIT_FILENAME)
> +
> # Tell GNU make to ignore these if they exist.
> -.PHONY: jit
> +.PHONY: jit jit.serial jit.prev
>
> jit_OBJS = attribs.o \
> jit/dummy-frontend.o \
> @@ -117,7 +119,7 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
> libbackend.a libcommon-target.a libcommon.a \
> $(CPPLIB) $(LIBDECNUMBER) \
> $(LIBDEPS) $(srcdir)/jit/libgccjit.map \
> - $(EXTRA_GCC_OBJS)
> + $(EXTRA_GCC_OBJS) jit.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
> $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
> $(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
> --- gcc/d/Make-lang.in.jj 2020-08-26 10:27:10.690552343 +0200
> +++ gcc/d/Make-lang.in 2020-09-03 15:48:15.010717845 +0200
> @@ -27,9 +27,10 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
>
> # The name for selecting d in LANGUAGES.
> d: d21$(exeext)
> +d.serial: d
>
> # Tell GNU make to ignore these if they exist.
> -.PHONY: d
> +.PHONY: d d.serial d.prev
>
> # Create the compiler driver for D.
> CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
> @@ -162,7 +163,7 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
>
> d_OBJS = $(D_ALL_OBJS) d/d-spec.o
>
> -d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
> +d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) d.prev
> +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
> $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
>
> --- gcc/doc/install.texi.jj 2020-08-25 07:15:43.982903777 +0200
> +++ gcc/doc/install.texi 2020-09-03 16:16:41.403789530 +0200
> @@ -1533,6 +1533,13 @@ When building GCC, use a mutex to avoid
> multiple languages at the same time, to avoid thrashing on build
> systems with limited free memory. The default is not to use such a mutex.
>
> +@item --enable-link-serialization
> +When building GCC, use make dependencies to serialize linking the compilers for
> +multiple languages, to avoid thrashing on build
> +systems with limited free memory. The default is not to add such
> +dependencies and thus with parallel make potentially link different
> +compilers concurrently.
> +
> @item --enable-maintainer-mode
> The build rules that regenerate the Autoconf and Automake output files as
> well as the GCC master message catalog @file{gcc.pot} are normally
> --- configure.jj 2020-08-24 10:00:01.233259699 +0200
> +++ configure 2020-09-03 16:09:58.121679071 +0200
> @@ -680,6 +680,7 @@ extra_host_zlib_configure_flags
> extra_host_libiberty_configure_flags
> stage1_languages
> host_shared
> +DO_LINK_SERIALIZATION
> extra_linker_plugin_flags
> extra_linker_plugin_configure_flags
> islinc
> @@ -819,6 +820,7 @@ enable_isl_version_check
> enable_lto
> enable_linker_plugin_configure_flags
> enable_linker_plugin_flags
> +enable_link_serialization
> enable_host_shared
> enable_stage1_languages
> enable_objc_gc
> @@ -1538,6 +1540,10 @@ Optional Features:
> --enable-linker-plugin-flags=FLAGS
> additional flags for configuring and building linker
> plugins [none]
> + --enable-link-serialization
> + avoid linking multiple GCC front-ends at once using
> + make dependencies to avoid thrashing on the build
> + machine
> --enable-host-shared build host code as shared libraries
> --enable-stage1-languages[=all]
> choose additional languages to build during stage1.
> @@ -8264,6 +8270,26 @@ fi
>
>
>
> +
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
> +$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
> + # Check whether --enable-link-serialization was given.
> +if test "${enable_link_serialization+set}" = set; then :
> + enableval=$enable_link_serialization; do_link_serialization=$enableval
> +else
> + do_link_serialization=no
> +fi
> +
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
> +$as_echo "$do_link_serialization" >&6; }
> +
> +if test "$do_link_serialization" = "yes"; then
> + DO_LINK_SERIALIZATION=true
> +else
> + DO_LINK_SERIALIZATION=false
> +fi
> +
> +
> # Enable --enable-host-shared.
> # Checked early to determine whether jit is an 'all' language
> # Check whether --enable-host-shared was given.
> --- Makefile.in.jj 2020-04-09 21:19:04.790224960 +0200
> +++ Makefile.in 2020-09-03 16:14:29.543715216 +0200
> @@ -979,7 +979,8 @@ TARGET_FLAGS_TO_PASS = $(BASE_FLAGS_TO_P
> EXTRA_GCC_FLAGS = \
> "GCC_FOR_TARGET=$(GCC_FOR_TARGET)" \
> "`echo 'STMP_FIXPROTO=$(STMP_FIXPROTO)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
> - "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`"
> + "`echo 'LIMITS_H_TEST=$(LIMITS_H_TEST)' | sed -e s'/[^=][^=]*=$$/XFOO=/'`" \
> + "DO_LINK_SERIALIZATION=@DO_LINK_SERIALIZATION@"
>
> GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
>
> --- gcc/configure.jj 2020-08-26 17:09:45.828254723 +0200
> +++ gcc/configure 2020-09-03 16:05:40.070447656 +0200
> @@ -30209,6 +30209,22 @@ do
> echo "lang.$t: $x" >> Make-hooks
> done
>
> +echo "ifeq (\$(DO_LINK_SERIALIZATION),true)" >> Make-hooks
> +prev=c
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev: $prev.serial" >> Make-hooks
> + prev=$lang
> +done
> +echo else >> Make-hooks
> +for lang in $all_selected_languages
> +do
> + test $lang = c && continue
> + echo "$lang.prev:" >> Make-hooks
> +done
> +echo endif >> Make-hooks
> +
> # --------
> # Option include files
> # --------
>
>
Jakub
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] --enable-link-serialization support
2020-09-03 14:49 ` [PATCH] " Jakub Jelinek
2020-09-04 8:32 ` Jakub Jelinek
@ 2020-09-07 21:58 ` Jason Merrill
2020-09-08 12:28 ` [PATCH v2] " Jakub Jelinek
1 sibling, 1 reply; 6+ messages in thread
From: Jason Merrill @ 2020-09-07 21:58 UTC (permalink / raw)
To: Jakub Jelinek; +Cc: Richard Biener, gcc-patches List
On Thu, Sep 3, 2020 at 10:49 AM Jakub Jelinek via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> On Thu, Sep 03, 2020 at 03:53:35PM +0200, Richard Biener wrote:
> > On Thu, 3 Sep 2020, Jakub Jelinek wrote:
> > But is that an issue in practice? I usually do not do make -j32 cc1plus
> > in a tree that was configured for bootstrap, nor do I use
> > --enable-link-serialization in that case.
>
> Guess most often true, but one could still do it when debugging some
> particular problem during bootstrap.
>
> > Btw, do you plan to keep --enable-link-mutex the way it is? What it
> > provides ontop of --enable-link-serialization is that progress
> > metering for very long running LTO link stages. Not sure if that
> > works for the last LTO link or only when there's some other link
> > waiting.
Only when there's another link waiting.
> I kept it as is for now. Primarily I didn't want to use
> --enable-linux-mutex for the new thing when it has nothing to do with a
> mutex. I think with both options together it will just print useless
> message that it acquired the lock immediately in each case.
Yes.
For this to replace --enable-link-mutex for me, I'd want it to work
with plain 'make' in the gcc directory, not just at top level.
Jason
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2] --enable-link-serialization support
2020-09-07 21:58 ` Jason Merrill
@ 2020-09-08 12:28 ` Jakub Jelinek
0 siblings, 0 replies; 6+ messages in thread
From: Jakub Jelinek @ 2020-09-08 12:28 UTC (permalink / raw)
To: Richard Biener, Jason Merrill, Paolo Bonzini, Nathanael Nerode,
Alexandre Oliva, Ralf Wildenhues
Cc: gcc-patches List
On Mon, Sep 07, 2020 at 05:58:04PM -0400, Jason Merrill via Gcc-patches wrote:
> On Thu, Sep 3, 2020 at 10:49 AM Jakub Jelinek via Gcc-patches
> <gcc-patches@gcc.gnu.org> wrote:
> >
> > On Thu, Sep 03, 2020 at 03:53:35PM +0200, Richard Biener wrote:
> > > On Thu, 3 Sep 2020, Jakub Jelinek wrote:
> > > But is that an issue in practice? I usually do not do make -j32 cc1plus
> > > in a tree that was configured for bootstrap, nor do I use
> > > --enable-link-serialization in that case.
> >
> > Guess most often true, but one could still do it when debugging some
> > particular problem during bootstrap.
Ok, based on your and Jason's comment, I've moved it back to gcc/ configure,
which means that even inside of gcc/ make all (as well as e.g. make lto-dump)
will serialize and build all previous large binaries when configured this
way.
One can always make -j32 cc1 DO_LINK_SERIALIZATION=
to avoid that.
Furthermore, I've implemented the idea I wrote about, so that
--enable-link-serialization
is the same as
--enable-link-serialization=1
and means the large link commands are serialized, one can as before (the
default)
--disable-link-serialization
which will cause all links to be parallelizable, but one can newly also
--enable-link-serialization=3
etc. which says that at most 3 of the large link commands can run
concurrently.
And finally I've implemented (only if the serialization is enabled) simple
progress bars for the linking.
With --enable-link-serialization and e.g. the 5 large links I have in my
current tree (cc1, cc1plus, f951, lto1 and lto-dump), before the linking it
prints
Linking |==-- | 20%
and after it
Linking |==== | 40%
(each 2 = characters stand for already finished links, each 2 -
characters 2 - characters stand for the link being started).
With --enable-link-serialization=3 it will change the way the start is
printed, one will get:
Linking |-- | 0%
at the start of cc1 link,
Linking |>>-- | 0%
at the start of the second large link and
Linking |>>>>-- | 0%
at the start of the third large link, where the 2 > characters stand for
already pending links. The printing at the end of link command is
the same as with the full serialization, i.e. for the above 3:
Linking |== | 20%
Linking |==== | 40%
Linking |====== | 60%
but one could actually get them in any order depending on which of those 3
finishes first - to get it 100% accurate I'd need to add some directory with
files representing finished links or similar, doesn't seem worth it.
2020-09-08 Jakub Jelinek <jakub@redhat.com>
gcc/
* configure.ac: Add $lang.prev rules, INDEX.$lang and SERIAL_LIST and
SERIAL_COUNT variables to Make-hooks.
(--enable-link-serialization): New configure option.
* Makefile.in (DO_LINK_SERIALIZATION, LINK_PROGRESS): New variables.
* doc/install.texi (--enable-link-serialization): Document.
* configure: Regenerated.
gcc/c/
* Make-lang.in (c.serial): New goal.
(.PHONY): Add c.serial c.prev.
(cc1$(exeext)): Call LINK_PROGRESS.
gcc/cp/
* Make-lang.in (c++.serial): New goal.
(.PHONY): Add c++.serial c++.prev.
(cc1plus$(exeext)): Depend on c++.prev. Call LINK_PROGRESS.
gcc/fortran/
* Make-lang.in (fortran.serial): New goal.
(.PHONY): Add fortran.serial fortran.prev.
(f951$(exeext)): Depend on fortran.prev. Call LINK_PROGRESS.
gcc/lto/
* Make-lang.in (lto, lto1.serial, lto2.serial): New goals.
(.PHONY): Add lto lto1.serial lto1.prev lto2.serial lto2.prev.
(lto.all.cross, lto.start.encap): Remove dependencies.
($(LTO_EXE)): Depend on lto1.prev. Call LINK_PROGRESS.
($(LTO_DUMP_EXE)): Depend on lto2.prev. Call LINK_PROGRESS.
gcc/objc/
* Make-lang.in (objc.serial): New goal.
(.PHONY): Add objc.serial objc.prev.
(cc1obj$(exeext)): Depend on objc.prev. Call LINK_PROGRESS.
gcc/objcp/
* Make-lang.in (obj-c++.serial): New goal.
(.PHONY): Add obj-c++.serial obj-c++.prev.
(cc1objplus$(exeext)): Depend on obj-c++.prev. Call LINK_PROGRESS.
gcc/ada/
* gcc-interface/Make-lang.in (ada.serial): New goal.
(.PHONY): Add ada.serial ada.prev.
(gnat1$(exeext)): Depend on ada.prev. Call LINK_PROGRESS.
gcc/brig/
* Make-lang.in (brig.serial): New goal.
(.PHONY): Add brig.serial brig.prev.
(brig1$(exeext)): Depend on brig.prev. Call LINK_PROGRESS.
gcc/go/
* Make-lang.in (go.serial): New goal.
(.PHONY): Add go.serial go.prev.
(go1$(exeext)): Depend on go.prev. Call LINK_PROGRESS.
gcc/jit/
* Make-lang.in (jit.serial): New goal.
(.PHONY): Add jit.serial jit.prev.
($(LIBGCCJIT_FILENAME)): Depend on jit.prev. Call LINK_PROGRESS.
gcc/d/
* Make-lang.in (d.serial): New goal.
(.PHONY): Add d.serial d.prev.
(d21$(exeext)): Depend on d.prev. Call LINK_PROGRESS.
--- gcc/configure.ac.jj 2020-09-08 12:24:39.199542406 +0200
+++ gcc/configure.ac 2020-09-08 13:00:48.788408222 +0200
@@ -6380,6 +6380,29 @@ else
fi
AC_SUBST(DO_LINK_MUTEX)
+dnl Whether to prevent multiple GCC front-ends from linking at the same time
+
+AC_MSG_CHECKING([whether to serialize linking of multiple front-ends])
+ AC_ARG_ENABLE(link-serialization,
+[AS_HELP_STRING([--enable-link-serialization],
+ [avoid linking multiple GCC front-ends at once using make
+ dependencies to avoid thrashing on the build machine])],
+ do_link_serialization=$enableval,
+ do_link_serialization=no)
+AC_MSG_RESULT($do_link_serialization)
+
+case "$do_link_serialization" in
+ yes)
+ DO_LINK_SERIALIZATION=1;;
+ [[1-9]] | [[1-9]][[0-9]] | [[1-9]][[0-9]][[0-9]])
+ DO_LINK_SERIALIZATION=$do_link_serialization;;
+ no)
+ DO_LINK_SERIALIZATION=;;
+ *)
+ AC_MSG_ERROR(bad value ${do_link_serialization} given for --enable-link-serialization) ;;
+esac
+AC_SUBST(DO_LINK_SERIALIZATION)
+
# --------------
# Language hooks
# --------------
@@ -6540,21 +6563,42 @@ do
echo "lang.$t: $x" >> Make-hooks
done
-echo "ifeq (\$(DO_LINK_SERIALIZATION),true)" >> Make-hooks
-prev=c
+echo "ifeq (\$(DO_LINK_SERIALIZATION),)" >> Make-hooks
+echo "SERIAL_LIST =" >> Make-hooks
+echo else >> Make-hooks
+lang_cnt=0
+lang_list=
+prev=c.serial
+serialization_languages=c
for lang in $all_selected_languages
do
test $lang = c && continue
- echo "$lang.prev: $prev.serial" >> Make-hooks
- prev=$lang
+ if test $lang = lto; then
+ serialization_languages="$serialization_languages lto1 lto2"
+ else
+ serialization_languages="$serialization_languages $lang"
+ fi
done
-echo else >> Make-hooks
-for lang in $all_selected_languages
+for lang in $serialization_languages
do
test $lang = c && continue
- echo "$lang.prev:" >> Make-hooks
+ lang_cnt=`expr $lang_cnt + 1`
+ lang_list=" $prev$lang_list"
+ prev=${lang}.serial
done
+echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
echo endif >> Make-hooks
+echo "SERIAL_COUNT = `expr $lang_cnt + 1`" >> Make-hooks
+echo "INDEX.c = 0" >> Make-hooks
+lang_idx=1
+for lang in $serialization_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+ echo "INDEX.$lang = $lang_idx" >> Make-hooks
+ lang_cnt=`expr $lang_cnt - 1`
+ lang_idx=`expr $lang_idx + 1`
+done
# --------
# Option include files
--- gcc/Makefile.in.jj 2020-09-03 20:10:13.572996335 +0200
+++ gcc/Makefile.in 2020-09-08 14:04:39.765714704 +0200
@@ -1739,6 +1739,8 @@ $(FULL_DRIVER_NAME): ./xgcc$(exeext)
# Otherwise $(SELFTEST_DEPS) is empty when used from <LANG>/Make-lang.in.
SELFTEST_DEPS = $(GCC_PASSES) stmp-int-hdrs $(srcdir)/testsuite/selftests
+DO_LINK_SERIALIZATION = @DO_LINK_SERIALIZATION@
+
# Language makefile fragments.
# The following targets define the interface between us and the languages.
@@ -1756,6 +1758,23 @@ SELFTEST_DEPS = $(GCC_PASSES) stmp-int-h
# language hooks, generated by configure
@language_hooks@
+ifeq ($(DO_LINK_SERIALIZATION),)
+LINK_PROGRESS = :
+else
+LINK_PROGRESS = msg="Linking |"; cnt=0; if test "$(2)" = start; then \
+ idx=0; cnt2=$(DO_LINK_SERIALIZATION); \
+ while test $$cnt2 -le $(1); do msg="$${msg}=="; cnt2=`expr $$cnt2 + 1`; idx=`expr $$idx + 1`; done; \
+ cnt=$$idx; \
+ while test $$cnt -lt $(1); do msg="$${msg}>>"; cnt=`expr $$cnt + 1`; done; \
+ msg="$${msg}--"; cnt=`expr $$cnt + 1`; \
+ else \
+ idx=`expr $(1) + 1`; \
+ while test $$cnt -le $(1); do msg="$${msg}=="; cnt=`expr $$cnt + 1`; done; \
+ fi; \
+ while test $$cnt -lt $(SERIAL_COUNT); do msg="$${msg} "; cnt=`expr $$cnt + 1`; done; \
+ msg="$${msg}| `expr 100 \* $$idx / $(SERIAL_COUNT)`%"; echo "$$msg"
+endif
+
# Wire in install-gnatlib invocation with `make install' for a configuration
# with top-level libada disabled.
gnat_install_lib = @gnat_install_lib@
--- gcc/c/Make-lang.in.jj 2020-09-03 20:10:13.608995812 +0200
+++ gcc/c/Make-lang.in 2020-09-08 13:35:55.907221237 +0200
@@ -37,9 +37,10 @@
#\f
# Define the names for selecting c in LANGUAGES.
c: cc1$(exeext)
+c.serial: c
# Tell GNU make to ignore these if they exist.
-.PHONY: c gcc
+.PHONY: c gcc c.serial
# The C front end driver. This is different from the drivers for other
# front ends, because there is no C language specific driver (i.e. nothing
@@ -82,8 +83,10 @@ cc1-checksum.c : build/genchecksum$(buil
fi
cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS)
+ @$(call LINK_PROGRESS,$(INDEX.c),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \
cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.c),end)
cc1.fda: ../stage1-gcc/cc1$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../stage1-gcc/cc1$(exeext) -gcov cc1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
--- gcc/cp/Make-lang.in.jj 2020-09-03 20:10:13.683994722 +0200
+++ gcc/cp/Make-lang.in 2020-09-08 13:35:15.793814755 +0200
@@ -47,9 +47,10 @@ CP_PLUGIN_HEADERS := cp-tree.h cxx-prett
# into the C++ rule, but that needs a little bit of work
# to do the right thing within all.cross.
c++: cc1plus$(exeext)
+c++.serial: c++
# Tell GNU make to ignore these if they exist.
-.PHONY: c++
+.PHONY: c++ c++.serial c++.prev
CFLAGS-cp/g++spec.o += $(DRIVER_DEFINES)
@@ -116,9 +117,11 @@ cc1plus-checksum.c : build/genchecksum$(
$(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \
fi
-cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1plus$(exeext): $(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBDEPS) c++.prev
+ @$(call LINK_PROGRESS,$(INDEX.c++),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(CXX_OBJS) cc1plus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.c++),end)
ifeq ($(ENABLE_MAINTAINER_RULES), true)
# Special build rule. This is a maintainer rule, that is only
--- gcc/fortran/Make-lang.in.jj 2020-09-03 20:10:13.737993937 +0200
+++ gcc/fortran/Make-lang.in 2020-09-08 13:36:53.382370858 +0200
@@ -72,9 +72,10 @@ fortran_OBJS = $(F95_OBJS) fortran/gfort
#\f
# Define the names for selecting gfortran in LANGUAGES.
fortran: f951$(exeext)
+fortran.serial: fortran
# Tell GNU make to ignore files by these names if they exist.
-.PHONY: fortran
+.PHONY: fortran fortran.serial fortran.prev
CFLAGS-fortran/gfortranspec.o += $(DRIVER_DEFINES)
@@ -92,11 +93,12 @@ gfortran-cross$(exeext): gfortran$(exeex
cp gfortran$(exeext) gfortran-cross$(exeext)
# The compiler itself is called f951.
-f951$(exeext): $(F95_OBJS) \
- $(BACKEND) $(LIBDEPS) attribs.o
+f951$(exeext): $(F95_OBJS) $(BACKEND) $(LIBDEPS) attribs.o fortran.prev
+ @$(call LINK_PROGRESS,$(INDEX.fortran),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(F95_OBJS) $(BACKEND) $(ZLIB) $(LIBS) attribs.o \
$(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.fortran),end)
gt-fortran-trans.h : s-gtype; @true
#\f
--- gcc/lto/Make-lang.in.jj 2020-09-03 20:10:13.759993617 +0200
+++ gcc/lto/Make-lang.in 2020-09-08 13:37:15.038050442 +0200
@@ -41,10 +41,16 @@ lto_dump_OBJS = $(LTO_DUMP_OBJS)
# Rules
+lto: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto1.serial: $(LTO_EXE)
+lto2.serial: $(LTO_DUMP_EXE)
+
+.PHONY: lto lto1.serial lto1.prev lto2.serial lto2.prev
+
# These hooks are used by the main GCC Makefile. Consult that
# Makefile for documentation.
-lto.all.cross: $(LTO_EXE) $(LTO_DUMP_EXE)
-lto.start.encap: $(LTO_EXE) $(LTO_DUMP_EXE)
+lto.all.cross:
+lto.start.encap:
lto.rest.encap:
lto.tags:
lto.install-common: installdirs
@@ -84,13 +90,17 @@ lto.stagefeedback:
# Use strict warnings for this front end.
lto-warn = $(STRICT_WARN)
-$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS) lto1.prev
+ @$(call LINK_PROGRESS,$(INDEX.lto1),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
+ @$(call LINK_PROGRESS,$(INDEX.lto1),end)
-$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS)
+$(LTO_DUMP_EXE): $(LTO_DUMP_OBJS) $(BACKEND) $(LIBDEPS) lto2.prev
+ @$(call LINK_PROGRESS,$(INDEX.lto2),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_DUMP_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
+ @$(call LINK_PROGRESS,$(INDEX.lto2),end)
lto/lto-dump.o: $(LTO_OBJS)
--- gcc/objc/Make-lang.in.jj 2020-09-03 20:10:13.764993545 +0200
+++ gcc/objc/Make-lang.in 2020-09-08 13:36:29.303727116 +0200
@@ -38,9 +38,10 @@
#\f
# Define the names for selecting Objective-C in LANGUAGES.
objc: cc1obj$(exeext)
+objc.serial: objc
# Tell GNU make to ignore these if they exist.
-.PHONY: objc
+.PHONY: objc objc.serial objc.prev
# Use maximal warnings for this front end.
objc-warn = $(STRICT_WARN)
@@ -62,10 +63,13 @@ cc1obj-checksum.c : build/genchecksum$(b
$(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \
$(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c
-cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) $(LIBDEPS)
+cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \
+ $(LIBDEPS) objc.prev
+ @$(call LINK_PROGRESS,$(INDEX.objc),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o \
$(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.objc),end)
objc.srcextra:
--- gcc/objcp/Make-lang.in.jj 2020-09-03 20:10:13.771993443 +0200
+++ gcc/objcp/Make-lang.in 2020-09-08 13:36:40.822556689 +0200
@@ -39,9 +39,10 @@
#\f
# Define the names for selecting Objective-C++ in LANGUAGES.
obj-c++: cc1objplus$(exeext)
+obj-c++.serial: obj-c++
# Tell GNU make to ignore these if they exist.
-.PHONY: obj-c++
+.PHONY: obj-c++ obj-c++.serial obj-c++.prev
# Use maximal warnings for this front end. Also, make ObjC and C++
# headers accessible.
@@ -66,9 +67,12 @@ cc1objplus-checksum.c : build/genchecksu
$(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \
cc1objplus-checksum.c
-cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBDEPS)
+cc1objplus$(exeext): $(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) \
+ $(LIBDEPS) obj-c++.prev
+ @$(call LINK_PROGRESS,$(INDEX.obj-c++),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(OBJCXX_OBJS) cc1objplus-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.obj-c++),end)
# Objective C++ language specific files.
--- gcc/ada/gcc-interface/Make-lang.in.jj 2020-09-03 20:10:13.592996044 +0200
+++ gcc/ada/gcc-interface/Make-lang.in 2020-09-08 13:35:25.692668295 +0200
@@ -146,9 +146,10 @@ endif
# Define the names for selecting Ada in LANGUAGES.
ada: gnat1$(exeext) gnatbind$(exeext)
+ada.serial: gnat1$(exeext)
# Tell GNU Make to ignore these, if they exist.
-.PHONY: ada
+.PHONY: ada ada.serial ada.prev
# Compute the FLAGS to pass for gnattools, now linked with a C++ driver as
# we're linking against at least libcommon which contains C++ compiled code.
@@ -666,10 +667,13 @@ ada/libgnat/s-excmac.adb: $(srcdir)/ada/
# Needs to be built with CC=gcc
# Since the RTL should be built with the latest compiler, remove the
# stamp target in the parent directory whenever gnat1 is rebuilt
-gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a $(LIBDEPS)
+gnat1$(exeext): $(TARGET_ADA_SRCS) $(GNAT1_OBJS) $(ADA_BACKEND) libcommon-target.a \
+ $(LIBDEPS) ada.prev
+ @$(call LINK_PROGRESS,$(INDEX.ada),start)
+$(GCC_LLINK) -o $@ $(GNAT1_OBJS) $(ADA_BACKEND) \
libcommon-target.a $(LIBS) $(SYSLIBS) $(BACKENDLIBS) $(CFLAGS)
$(RM) stamp-gnatlib2-rts stamp-tools
+ @$(call LINK_PROGRESS,$(INDEX.ada),end)
gnatbind$(exeext): ada/b_gnatb.o $(CONFIG_H) $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBDEPS)
+$(GCC_LINK) -o $@ ada/b_gnatb.o $(GNATBIND_OBJS) ggc-none.o libcommon-target.a $(LIBS) $(SYSLIBS) $(CFLAGS)
--- gcc/brig/Make-lang.in.jj 2020-09-03 20:10:13.596995986 +0200
+++ gcc/brig/Make-lang.in 2020-09-08 13:35:37.111499344 +0200
@@ -29,8 +29,9 @@ GCCBRIG_TARGET_INSTALL_NAME := $(target_
# The name for selecting brig in LANGUAGES.
brig: brig1$(exeext)
+brig.serial: brig
-.PHONY: brig
+.PHONY: brig brig.serial brig.prev
CFLAGS-brig/brigspec.o += $(DRIVER_DEFINES)
@@ -81,15 +82,12 @@ BRIG_OBJS = \
brig_OBJS = $(BRIG_OBJS) brig/brigspec.o
-# brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
-# +$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
-# $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
-
-
-brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+brig1$(exeext): $(BRIG_OBJS) attribs.o $(BACKEND) $(LIBDEPS) brig.prev
+ @$(call LINK_PROGRESS,$(INDEX.brig),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(BRIG_OBJS) attribs.o $(BACKEND) $(LIBS) \
$(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.brig),end)
# Documentation.
--- gcc/go/Make-lang.in.jj 2020-09-03 20:10:13.737993937 +0200
+++ gcc/go/Make-lang.in 2020-09-08 13:35:47.201350056 +0200
@@ -27,8 +27,9 @@ GCCGO_TARGET_INSTALL_NAME := $(target_no
# The name for selecting go in LANGUAGES.
go: go1$(exeext)
+go.serial: go
-.PHONY: go
+.PHONY: go go.serial go.prev
CFLAGS-go/gospec.o += $(DRIVER_DEFINES)
@@ -78,9 +79,11 @@ GO_OBJS = \
go_OBJS = $(GO_OBJS) go/gospec.o
-go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+go1$(exeext): $(GO_OBJS) attribs.o $(BACKEND) $(LIBDEPS) go.prev
+ @$(call LINK_PROGRESS,$(INDEX.go),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(GO_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.go),end)
# Documentation.
--- gcc/jit/Make-lang.in.jj 2020-09-03 20:10:13.750993749 +0200
+++ gcc/jit/Make-lang.in 2020-09-08 13:36:12.289978848 +0200
@@ -81,8 +81,10 @@ jit: $(LIBGCCJIT_FILENAME) \
$(FULL_DRIVER_NAME)
endif
+jit.serial: $(LIBGCCJIT_FILENAME)
+
# Tell GNU make to ignore these if they exist.
-.PHONY: jit
+.PHONY: jit jit.serial jit.prev
jit_OBJS = attribs.o \
jit/dummy-frontend.o \
@@ -117,12 +119,14 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) \
$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
- $(EXTRA_GCC_OBJS)
+ $(EXTRA_GCC_OBJS) jit.prev
+ @$(call LINK_PROGRESS,$(INDEX.jit),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
$(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
$(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
$(EXTRA_GCC_OBJS) \
$(LIBGCCJIT_EXTRA_OPTS)
+ @$(call LINK_PROGRESS,$(INDEX.jit),end)
# Create symlinks when not building for Windows
ifeq (,$(findstring mingw,$(target)))
--- gcc/d/Make-lang.in.jj 2020-09-03 20:10:13.696994533 +0200
+++ gcc/d/Make-lang.in 2020-09-08 13:36:21.474842957 +0200
@@ -27,9 +27,10 @@ D_LIBPHOBOS = -DLIBPHOBOS=\"gphobos\"
# The name for selecting d in LANGUAGES.
d: d21$(exeext)
+d.serial: d
# Tell GNU make to ignore these if they exist.
-.PHONY: d
+.PHONY: d d.serial d.prev
# Create the compiler driver for D.
CFLAGS-d/d-spec.o += $(DRIVER_DEFINES) $(D_LIBPHOBOS)
@@ -162,9 +163,11 @@ D_ALL_OBJS = $(D_FRONTEND_OBJS) $(D_GENE
d_OBJS = $(D_ALL_OBJS) d/d-spec.o
-d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS)
+d21$(exeext): $(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBDEPS) d.prev
+ @$(call LINK_PROGRESS,$(INDEX.d),start)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(D_ALL_OBJS) attribs.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
+ @$(call LINK_PROGRESS,$(INDEX.d),end)
# Documentation.
--- gcc/doc/install.texi.jj 2020-09-03 20:10:13.719994200 +0200
+++ gcc/doc/install.texi 2020-09-08 13:13:16.934334012 +0200
@@ -1533,6 +1533,14 @@ When building GCC, use a mutex to avoid
multiple languages at the same time, to avoid thrashing on build
systems with limited free memory. The default is not to use such a mutex.
+@item --enable-link-serialization
+When building GCC, use make dependencies to serialize linking the compilers for
+multiple languages, to avoid thrashing on build
+systems with limited free memory. The default is not to add such
+dependencies and thus with parallel make potentially link different
+compilers concurrently. If the argument is a positive integer, allow
+that number of concurrent link processes for the large binaries.
+
@item --enable-maintainer-mode
The build rules that regenerate the Autoconf and Automake output files as
well as the GCC master message catalog @file{gcc.pot} are normally
--- gcc/configure.jj 2020-09-03 20:10:13.657995099 +0200
+++ gcc/configure 2020-09-08 13:00:52.702350290 +0200
@@ -710,6 +710,7 @@ subdirs
dollar
gcc_tooldir
enable_lto
+DO_LINK_SERIALIZATION
DO_LINK_MUTEX
MAINT
zlibinc
@@ -1012,6 +1013,7 @@ with_gc
with_system_zlib
enable_maintainer_mode
enable_link_mutex
+enable_link_serialization
enable_version_specific_runtime_libs
enable_plugin
enable_host_shared
@@ -1767,6 +1769,10 @@ Optional Features:
sometimes confusing) to the casual installer
--enable-link-mutex avoid linking multiple front-ends at once to avoid
thrashing on the build machine
+ --enable-link-serialization
+ avoid linking multiple GCC front-ends at once using
+ make dependencies to avoid thrashing on the build
+ machine
--enable-version-specific-runtime-libs
specify that runtime libraries should be installed
in a compiler-specific directory
@@ -19013,7 +19019,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 19016 "configure"
+#line 19022 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -19119,7 +19125,7 @@ else
lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
lt_status=$lt_dlunknown
cat > conftest.$ac_ext <<_LT_EOF
-#line 19122 "configure"
+#line 19128 "configure"
#include "confdefs.h"
#if HAVE_DLFCN_H
@@ -30049,6 +30055,31 @@ else
fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to serialize linking of multiple front-ends" >&5
+$as_echo_n "checking whether to serialize linking of multiple front-ends... " >&6; }
+ # Check whether --enable-link-serialization was given.
+if test "${enable_link_serialization+set}" = set; then :
+ enableval=$enable_link_serialization; do_link_serialization=$enableval
+else
+ do_link_serialization=no
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $do_link_serialization" >&5
+$as_echo "$do_link_serialization" >&6; }
+
+case "$do_link_serialization" in
+ yes)
+ DO_LINK_SERIALIZATION=1;;
+ [1-9] | [1-9][0-9] | [1-9][0-9][0-9])
+ DO_LINK_SERIALIZATION=$do_link_serialization;;
+ no)
+ DO_LINK_SERIALIZATION=;;
+ *)
+ as_fn_error $? "bad value ${do_link_serialization} given for --enable-link-serialization" "$LINENO" 5 ;;
+esac
+
+
# --------------
# Language hooks
# --------------
@@ -30209,6 +30240,43 @@ do
echo "lang.$t: $x" >> Make-hooks
done
+echo "ifeq (\$(DO_LINK_SERIALIZATION),)" >> Make-hooks
+echo "SERIAL_LIST =" >> Make-hooks
+echo else >> Make-hooks
+lang_cnt=0
+lang_list=
+prev=c.serial
+serialization_languages=c
+for lang in $all_selected_languages
+do
+ test $lang = c && continue
+ if test $lang = lto; then
+ serialization_languages="$serialization_languages lto1 lto2"
+ else
+ serialization_languages="$serialization_languages $lang"
+ fi
+done
+for lang in $serialization_languages
+do
+ test $lang = c && continue
+ lang_cnt=`expr $lang_cnt + 1`
+ lang_list=" $prev$lang_list"
+ prev=${lang}.serial
+done
+echo "SERIAL_LIST = \$(wordlist \$(DO_LINK_SERIALIZATION),$lang_cnt,$lang_list)" >> Make-hooks
+echo endif >> Make-hooks
+echo "SERIAL_COUNT = `expr $lang_cnt + 1`" >> Make-hooks
+echo "INDEX.c = 0" >> Make-hooks
+lang_idx=1
+for lang in $serialization_languages
+do
+ test $lang = c && continue
+ echo "$lang.prev: \$(word $lang_cnt,\$(SERIAL_LIST))" >> Make-hooks
+ echo "INDEX.$lang = $lang_idx" >> Make-hooks
+ lang_cnt=`expr $lang_cnt - 1`
+ lang_idx=`expr $lang_idx + 1`
+done
+
# --------
# Option include files
# --------
Jakub
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-08 12:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-03 13:36 RFC: --enable-link-serialization support Jakub Jelinek
2020-09-03 13:53 ` Richard Biener
2020-09-03 14:49 ` [PATCH] " Jakub Jelinek
2020-09-04 8:32 ` Jakub Jelinek
2020-09-07 21:58 ` Jason Merrill
2020-09-08 12:28 ` [PATCH v2] " Jakub Jelinek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).