Re: Parallelize the compilation using Threads

[Richard Biener <richard.guenther@gmail.com>]

On Wed, Nov 14, 2018 at 10:47 PM Giuliano Augusto Faulin Belinassi
<giuliano.belinassi@usp.br> wrote:
>
> As a brief introduction, I am a graduate student that got interested
>
> in the "Parallelize the compilation using threads"(GSoC 2018 [1]). I
> am a newcommer in GCC, but already have sent some patches, some of
> them have already been accepted [2].
>
> I brought this subject up in IRC, but maybe here is a proper place to
> discuss this topic.
>
> From my point of view, parallelizing GCC itself will only speed up the
> compilation of projects which have a big file that creates a
> bottleneck in the whole project compilation (note: by big, I mean the
> amount of code to generate).

That's true.  During GCC bootstrap there are some of those (see PR84402).

One way to improve parallelism is to use link-time optimization where
even single source files can be split up into multiple link-time units.  But
then there's the serial whole-program analysis part.

> Additionally, I know that GCC must not
> change the project layout, but from the software engineering perspective,
> this may be a bad smell that indicates that the file should be broken
> into smaller files. Finally, the Makefiles will take care of the
> parallelization task.

What do you mean by GCC must not change the project layout?  GCC
happily re-orders functions and link-time optimization will reorder
TUs (well, linking may as well).

> My questions are:
>
>  1. Is there any project compilation that will significantly be improved
> if GCC runs in parallel? Do someone has data about something related
> to that? How about the Linux Kernel? If not, I can try to bring some.

We do not have any data about this apart from experiments with
splitting up source files for PR84402.

>  2. Did I correctly understand the goal of the parallelization? Can
> anyone provide extra details to me?

You may want to search the mailing list archives since we had a
student application (later revoked) for the task with some discussion.

In my view (I proposed the thing) the most interesting parts are
getting GCCs global state documented and reduced.  The parallelization
itself is an interesting experiment but whether there will be any
substantial improvement for builds that can already benefit from make
parallelism remains a question.

> I am willing to turn my master’s thesis on that and also apply to GSoC
> 2019 if it shows to be fruitful.
>
> [1] https://gcc.gnu.org/wiki/SummerOfCode
> [2] https://patchwork.ozlabs.org/project/gcc/list/?submitter=74682
>
>
> Thanks