From: David Edelsohn <dje.gcc@gmail.com>
To: "Martin Liška" <mliska@suse.cz>
Cc: Richard Biener <richard.guenther@gmail.com>,
GCC Mailing List <gcc@gcc.gnu.org>,
GCC Patches <gcc-patches@gcc.gnu.org>,
Segher Boessenkool <segher@kernel.crashing.org>
Subject: Re: [PATCH] Try LTO partial linking. (Was: Speed of compiling gimple-match.c)
Date: Fri, 21 May 2021 08:35:09 -0400 [thread overview]
Message-ID: <CAGWvny=97e5LQeVOMSPnkPD8sq6C9q2v4a-EeKxaZOF4maDTWw@mail.gmail.com> (raw)
In-Reply-To: <03febb0f-16fa-4048-6680-438a63b11dcf@suse.cz>
On Fri, May 21, 2021 at 5:25 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 5/20/21 2:54 PM, Richard Biener wrote:
> > On Thu, May 20, 2021 at 2:34 PM Martin Liška <mliska@suse.cz> wrote:
> >>
> >> Hello.
> >>
> >> I've got a patch candidate that leverages partial linking for a couple of selected object files.
> >>
> >> I'm sending make all-host- jX results for my machine:
> >>
> >> before: 3m18s (user 32m52s)
> >> https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/1dd5eae5001295ba0230a689f7edc67284c9b742/gcc-all-host.svg
> >>
> >> after: 2m57m (user 35m)
> >> https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/d659b2187cf622167841efbbe6bc93cb33855fa9/gcc-all-host-partial-lto.svg
> >>
> >> One can utilize it with:
> >> make -j16 all-host PARTIAL_LTO=1
> >>
> >> @Segher, Andrew: Can you please measure time improvement for your slow bootstrap?
> >> One can also tweak --param=lto-partitions=16 param value.
> >>
> >> Thoughts?
> >
> > You're LTO linking multiple objects here - that's almost as if you
> > were doing this
> > for the whole of libbackend.a ... so $(OBJS)_CLFAGS += -flto and in the
> > libbackend.a rule do a similar partial link trick.
>
> Yeah, apart from that one can't likely do partial linking for an archive:
>
> $ g++ -no-pie -flto=auto --param=lto-partitions=16 -flinker-output=nolto-rel -r libbackend.a
> collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
> compilation terminated.
>
> while ld.bfd immediately finishes.
>
> >
> > That gets you half of a LTO bootstrap then.
> >
> > So why did you go from applying this per-file to multiple files? Does $(LINKER)
> > have a proper rule to pick up a jobserver?
> >
> > When upstreaming in any form you probably have to gate it on bootstrap-lto
> > being not active.
>
> Sure, that's reasonable, we can likely detect a -flto option in $(COMPILE), right?
>
> One more thing I face is broken dependency:
> $ make clean && make -j32 PARTIAL_LTO=1
>
> g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-error=format-diag -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common -Wno-unused -DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc -I/home/marxin/Programming/gcc/gcc/. -I/home/marxin/Programming/gcc/gcc/../include -I/home/marxin/Programming/gcc/gcc/../libcpp/include -I/home/marxin/Programming/gcc/gcc/../libcody -I/home/marxin/Programming/gcc/gcc/../libdecnumber -I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber -I/home/marxin/Programming/gcc/gcc/../libbacktrace -o gimple-match-lto.o -MT gimple-match-lto.o -MMD -MP -MF ./.deps/gimple-match-lto.TPo gimple-match.c -flto
> g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-error=format-diag -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common -Wno-unused -DHAVE_CONFIG_H -I. -I. -I/home/marxin/Programming/gcc/gcc -I/home/marxin/Programming/gcc/gcc/. -I/home/marxin/Programming/gcc/gcc/../include -I/home/marxin/Programming/gcc/gcc/../libcpp/include -I/home/marxin/Programming/gcc/gcc/../libcody -I/home/marxin/Programming/gcc/gcc/../libdecnumber -I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber -I/home/marxin/Programming/gcc/gcc/../libbacktrace -o generic-match-lto.o -MT generic-match-lto.o -MMD -MP -MF ./.deps/generic-match-lto.TPo generic-match.c -flto
>
> In file included from ./tm.h:26,
> from /home/marxin/Programming/gcc/gcc/backend.h:28,
> from /home/marxin/Programming/gcc/gcc/generic-match-head.c:23,
> from generic-match.c:4:
> /home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error: insn-attr-common.h: No such file or directory
> 2286 | #include "insn-attr-common.h"
> | ^~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> make: *** [Makefile:2678: generic-match-lto.o] Error 1
> make: *** Waiting for unfinished jobs....
>
> In file included from ./tm.h:26,
> from /home/marxin/Programming/gcc/gcc/backend.h:28,
> from /home/marxin/Programming/gcc/gcc/gimple-match-head.c:23,
> from gimple-match.c:4:
> /home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error: insn-attr-common.h: No such file or directory
> 2286 | #include "insn-attr-common.h"
> | ^~~~~~~~~~~~~~~~~~~~
>
> I explicitly added:
> gimple-match.o: gimple-match.c $(generated_files)
> generic-match.o: generic-match.c $(generated_files)
>
> But it's not obeyed.
Please remember that not all targets support LTO so a fallback to a
non-partial-LTO build needs to be provided and automatically invoked
for those targets.
Thanks, David
next prev parent reply other threads:[~2021-05-21 12:35 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CA+=Sn1=+tuqG7qu5C6_atWzJp+af1P6oVPvABHxqNxiUmJb1dg@mail.gmail.com>
[not found] ` <CAFiYyc1DC1W==r37RhVX9epv1F5XSQhr1XeZEKCk1QOBObSKaQ@mail.gmail.com>
[not found] ` <20210511145904.GM10366@gate.crashing.org>
[not found] ` <CAFiYyc12G6qAeCevEH0-oE4kX7KN1gafBHN4UNYnQJWQVujffg@mail.gmail.com>
[not found] ` <20210512121248.GU10366@gate.crashing.org>
2021-05-20 12:34 ` Martin Liška
2021-05-20 12:54 ` Richard Biener
2021-05-20 13:06 ` Martin Liška
2021-05-20 13:16 ` Richard Biener
2021-05-20 13:22 ` Richard Biener
2021-05-20 15:55 ` Jan Hubicka
2021-05-21 8:29 ` Martin Liška
2021-06-01 7:31 ` Martin Liška
2021-06-23 13:53 ` Martin Liška
2021-08-16 13:58 ` Martin Liška
2021-08-20 14:54 ` Martin Liška
2021-08-22 13:09 ` Jan Hubicka
2021-06-12 15:55 ` Giuliano Belinassi
2021-05-21 8:43 ` Martin Liška
2021-05-21 12:35 ` David Edelsohn [this message]
2021-05-24 8:07 ` Martin Liška
2021-06-01 7:33 ` Martin Liška
2021-06-01 7:42 ` Richard Biener
2021-06-01 11:25 ` Martin Liška
2021-06-01 13:19 ` Richard Biener
2021-08-20 14:57 ` Martin Liška
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGWvny=97e5LQeVOMSPnkPD8sq6C9q2v4a-EeKxaZOF4maDTWw@mail.gmail.com' \
--to=dje.gcc@gmail.com \
--cc=gcc-patches@gcc.gnu.org \
--cc=gcc@gcc.gnu.org \
--cc=mliska@suse.cz \
--cc=richard.guenther@gmail.com \
--cc=segher@kernel.crashing.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).