public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
From: "marxin at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>
To: gcc-bugs@gcc.gnu.org
Subject: [Bug bootstrap/84402] [meta] GCC build system: parallelism bottleneck
Date: Wed, 30 Nov 2022 08:23:04 +0000 [thread overview]
Message-ID: <bug-84402-4-LLDbRLA1H6@http.gcc.gnu.org/bugzilla/> (raw)
In-Reply-To: <bug-84402-4@http.gcc.gnu.org/bugzilla/>
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84402
--- Comment #49 from Martin Liška <marxin at gcc dot gnu.org> ---
One more observation I made, apparently we're trying to sort (in Makefile.in)
OBJS with the biggest at the very beginning:
1295 # Language-independent object files.
1296 # We put the *-match.o and insn-*.o files first so that a parallel make
1297 # will build them sooner, because they are large and otherwise tend to
be
1298 # the last objects to finish building.
1299 OBJS = \
1300 gimple-match.o \
1301 generic-match.o \
1302 insn-attrtab.o \
1303 insn-automata.o \
That's fine, plus we introduce dependency for all objects to depend on
generated_files:
4441 # In order for parallel make to really start compiling the expensive
4442 # objects from $(OBJS) as early as possible, build all their
4443 # prerequisites strictly before all objects.
4444 $(ALL_HOST_OBJS) : | $(generated_files)
Using that, we should see gimple-match.o being spawned very soon, but it's not
the case. Imagine you have already built all-host and let's see what happens:
$ rm -f gimple-match.o ; rm -f tree*.o && make -j4 --debug=b libbackend.a 2>&1
| less
...
File 'gimple-match.o' does not exist.
Prerequisite 'cs-bconfig.h' is newer than target 'bconfig.h'.
Must remake target 'bconfig.h'.
Prerequisite 'cstamp-h' is newer than target 'auto-host.h'.
Must remake target 'auto-host.h'.
Prerequisite 's-options' is newer than target
'optionlist'.
Must remake target 'optionlist'.
Prerequisite 's-gtyp-input' is newer than target
'gtyp-input.list'.
Must remake target 'gtyp-input.list'.
Prerequisite 's-bversion' is newer than target
'bversion.h'.
Must remake target 'bversion.h'.
Prerequisite 'cs-config.h' is newer than target 'config.h'.
Must remake target 'config.h'.
...
File 'tree-vrp.o' does not exist.
File 'tree.o' does not exist.
Prerequisite 's-i386-bt' is newer than target 'i386-builtin-types.inc'.
Must remake target 'i386-builtin-types.inc'.
File 'gimple-match.o' does not exist.
Prerequisite 's-modes-h' is newer than target 'insn-modes.h'.
Must remake target 'insn-modes.h'.
Prerequisite 's-modes-inline-h' is newer than target
'insn-modes-inline.h'.
Must remake target 'insn-modes-inline.h'.
Prerequisite 's-version' is newer than target 'version.h'.
Must remake target 'version.h'.
Prerequisite 's-options-h' is newer than target 'options.h'.
Must remake target 'options.h'.
Prerequisite 's-genrtl-h' is newer than target 'genrtl.h'.
Must remake target 'genrtl.h'.
Prerequisite 's-modes-m' is newer than target 'min-insn-modes.cc'.
Must remake target 'min-insn-modes.cc'.
...
File 'gimple-match.o' does not exist.
Prerequisite 's-gtype' is newer than target 'gtype-desc.h'.
Must remake target 'gtype-desc.h'.
Prerequisite 's-constants' is newer than target
'insn-constants.h'.
Must remake target 'insn-constants.h'.
...
Must remake target 'tree-affine.o'.
g++ -fno-PIE -c -g -DIN_GCC -fno-exceptions -fno-rtti
-fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common
-DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc
-I/home/marxin/Programming/gcc/gcc/.
-I/home/marxin/Programming/gcc/gcc/../include
-I/home/marxin/Programming/gcc/gcc/../libcpp/include
-I/home/marxin/Programming/gcc/gcc/../libcody
-I/home/marxin/Programming/gcc/gcc/../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libbacktrace -o tree-affine.o -MT
tree-affine.o -MMD -MP -MF ./.deps/tree-affine.TPo
/home/marxin/Programming/gcc/gcc/tree-affine.cc
File 'tree-call-cdce.o' does not exist.
Must remake target 'tree-call-cdce.o'.
g++ -fno-PIE -c -g -DIN_GCC -fno-exceptions -fno-rtti
-fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common
-DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc
-I/home/marxin/Programming/gcc/gcc/.
-I/home/marxin/Programming/gcc/gcc/../include
-I/home/marxin/Programming/gcc/gcc/../libcpp/include
-I/home/marxin/Programming/gcc/gcc/../libcody
-I/home/marxin/Programming/gcc/gcc/../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libbacktrace -o tree-call-cdce.o -MT
tree-call-cdce.o -MMD -MP -MF ./.deps/tree-call-cdce.TPo
/home/marxin/Programming/gcc/gcc/tree-call-cdce.cc
File 'tree-cfg.o' does not exist.
Must remake target 'tree-cfg.o'.
g++ -fno-PIE -c -g -DIN_GCC -fno-exceptions -fno-rtti
-fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common
-DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc
-I/home/marxin/Programming/gcc/gcc/.
-I/home/marxin/Programming/gcc/gcc/../include
-I/home/marxin/Programming/gcc/gcc/../libcpp/include
-I/home/marxin/Programming/gcc/gcc/../libcody
-I/home/marxin/Programming/gcc/gcc/../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libbacktrace -o tree-cfg.o -MT
tree-cfg.o -MMD -MP -MF ./.deps/tree-cfg.TPo
/home/marxin/Programming/gcc/gcc/tree-cfg.cc
File 'tree-cfgcleanup.o' does not exist.
Must remake target 'tree-cfgcleanup.o'.
So gimple-match.o has a complex dependencies that are somehow under
investigation and that's why it doesn't start early :/
It's likely related to various ' $(STAMP) $name' we use, if I consider
one:
gtyp-input.list: s-gtyp-input ; @true
s-gtyp-input: Makefile
@: $(call write_entries_to_file,$(GTFILES),tmp-gi.list)
$(SHELL) $(srcdir)/../move-if-change tmp-gi.list gtyp-input.list
$(STAMP) s-gtyp-input
Here we touch 's-gtyp-input' later than gtyp-input.list is created and thus
gtyp-input.list always need
to be remade becase it's dependency s-gtyp-input is newer. Similarly for many
other rules:
gimple-match.cc: s-match gimple-match-head.cc ; @true
s-match: build/genmatch$(build_exeext) $(srcdir)/match.pd cfn-operators.pd
$(RUN_GEN) build/genmatch$(build_exeext) --gimple $(srcdir)/match.pd \
> tmp-gimple-match.cc
$(RUN_GEN) build/genmatch$(build_exeext) --generic $(srcdir)/match.pd \
> tmp-generic-match.cc
$(SHELL) $(srcdir)/../move-if-change tmp-gimple-match.cc \
gimple-match.cc
$(SHELL) $(srcdir)/../move-if-change tmp-generic-match.cc \
generic-match.cc
$(STAMP) s-match
Here it's even more complicated, I think s-match should be only updated if
generic-match.cc is touched,
otherwise, we again end up younger s-match than gimple-match.cc.
Can please any GNU make expect judge here? Starting e.g. gimple-match.cc early
would really help
to speed up the build process.
next prev parent reply other threads:[~2022-11-30 8:23 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <bug-84402-4@http.gcc.gnu.org/bugzilla/>
2020-05-07 11:56 ` jakub at gcc dot gnu.org
2020-05-07 22:59 ` egallager at gcc dot gnu.org
2020-07-09 9:44 ` rjiejie at me dot com
2020-07-09 10:04 ` marxin at gcc dot gnu.org
2020-07-09 11:40 ` rguenth at gcc dot gnu.org
2020-07-13 5:51 ` rjiejie at me dot com
2020-07-23 6:51 ` rguenth at gcc dot gnu.org
2021-04-08 12:02 ` rguenth at gcc dot gnu.org
2021-07-19 6:17 ` pinskia at gcc dot gnu.org
2021-07-19 6:19 ` pinskia at gcc dot gnu.org
2021-10-09 12:58 ` egallager at gcc dot gnu.org
2021-10-11 8:01 ` marxin at gcc dot gnu.org
2021-10-11 18:10 ` egallager at gcc dot gnu.org
2021-11-01 4:56 ` egallager at gcc dot gnu.org
2022-05-29 3:44 ` sam at gentoo dot org
2022-06-02 22:05 ` segher at gcc dot gnu.org
2022-11-30 8:13 ` marxin at gcc dot gnu.org
2022-11-30 8:23 ` marxin at gcc dot gnu.org [this message]
2022-11-30 8:25 ` rguenth at gcc dot gnu.org
2022-11-30 8:27 ` rguenth at gcc dot gnu.org
2022-11-30 8:38 ` rguenth at gcc dot gnu.org
2022-11-30 9:10 ` marxin at gcc dot gnu.org
2022-12-01 9:43 ` marxin at gcc dot gnu.org
2022-12-01 10:01 ` marxin at gcc dot gnu.org
2022-12-01 10:03 ` marxin at gcc dot gnu.org
2022-12-01 10:07 ` marxin at gcc dot gnu.org
2023-03-27 14:55 ` andrew.carlotti at arm dot com
2023-03-28 3:01 ` marxin at gcc dot gnu.org
2023-03-28 8:30 ` rguenth at gcc dot gnu.org
2023-03-28 8:45 ` amonakov at gcc dot gnu.org
2023-03-28 8:54 ` jakub at gcc dot gnu.org
2023-03-28 9:05 ` rguenther at suse dot de
2023-03-28 11:31 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:47 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:47 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:47 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:48 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:48 ` cvs-commit at gcc dot gnu.org
2023-05-05 12:48 ` cvs-commit at gcc dot gnu.org
2023-07-07 11:29 ` sjames at gcc dot gnu.org
2023-10-31 12:35 ` cvs-commit at gcc dot gnu.org
2023-10-31 12:48 ` sjames at gcc dot gnu.org
2024-03-04 4:29 ` law at gcc dot gnu.org
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bug-84402-4-LLDbRLA1H6@http.gcc.gnu.org/bugzilla/ \
--to=gcc-bugzilla@gcc.gnu.org \
--cc=gcc-bugs@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).