Re: [PATCH] Properly detect working jobserver in gcc driver.

[Richard Biener <richard.guenther@gmail.com>]

On Fri, Aug 2, 2019 at 11:19 AM Martin Liška <mliska@suse.cz> wrote:
>
> On 8/2/19 11:15 AM, Jan Hubicka wrote:
> >> On Fri, Aug 2, 2019 at 10:50 AM Jakub Jelinek <jakub@redhat.com> wrote:
> >>>
> >>> On Fri, Aug 02, 2019 at 10:47:10AM +0200, Martin Liška wrote:
> >>>>> Can you strace if other fds are opened and not closed in the spot you had it
> >>>>> before?  Advantage of doing it there is that it will not be done for all the
> >>>>> -E/-S/-c compilations when the linker is not spawned.
> >>>>
> >>>> I've used the same trick which you used and I'm attaching the output.
> >>>> I believe it's fine, I can't see any opened fd by GCC.
> >>>
> >>> LGTM.
> >>
>
> Hi.
>
> >> Btw, we discussed yesterday on the phone and the conclusion was to
> >> make -flto auto-detect a job-server (but not fall back to # of threads)
> >> and add -flto=auto to auto-detect a job-server and fall back to # of threads.
> >> That basically makes -flto=jobserver the default behavior which means
> >> we should document -flto=1 as a way to override jobserver detection.
> >
> > And concerning to the yesterday discussion, my preference would still be
> > -flto to first try presence of jobserver and default to number of
> > threads otherwise. It seems like user friendly default to me and other
> > tools with reasonable parallelism support usually behaves this way, too.
>
> I also like the default as Honza defined.
>
> >
> > Sure one can use -flto=auto everywhere but then one needs to use
> > different options for differnt compilers.  It would be nice to make
> > -flto to do right hting for majority of users including those like me
> > who cut&paste command lines from Make output and execute them by hand.
>
> Note that our ambition is to ideally backport the patches to our gcc9 package.
> The auto-detection of job-server with -flto is a behavior change that will happen
> in the gcc9 package.
>
> To be honest I don't like the invention of 'auto' value for -flto command.
> That would be useful just for gcc9 right now :/

Well.  We teached people they need to use -flto=N or -flto=jobserver to
get parallelism.  Like we teached people they need to use -O2 to get
any optimization.

Other compilers behave differently.  So what.

Auto-detecting threads is nice.  Suddenly trashing your system
because you happen to invoke two links in parallel is not.
Which is at least why changing this kind of behavior for 9.2 (or 9.3)
vs. 9.1 is out of the question.  And I'm not sure it is OK for 10.

As with defaulting to use a job-server when present that's more
reasonable.

For cut&pasting you then can use -flto=auto.

But it's just my opinion - I surely can adapt.

Richard.

> Martin
>
> >
> > The fork bombing is IMO relatively rare and it was not what Jakub ran
> > into (it was broken jobserv detection).
> > After all whole distro built with Martin's orriginal approach of
> > -flto=<nthreads> which forkbombs on every possible occasion.
> >
> > Said that, I could live with more conservative defaults.
> > Honza
> >
>