From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 63468 invoked by alias); 2 Aug 2019 09:04:32 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 63455 invoked by uid 89); 2 Aug 2019 09:04:32 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-14.7 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.1 spammy=anybody X-HELO: mail-lj1-f193.google.com Received: from mail-lj1-f193.google.com (HELO mail-lj1-f193.google.com) (209.85.208.193) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 02 Aug 2019 09:04:31 +0000 Received: by mail-lj1-f193.google.com with SMTP id x25so72206749ljh.2 for ; Fri, 02 Aug 2019 02:04:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=tVNBYMQkDk0VDAwF807uj1ngqIclRXyJkhfjxRwLGek=; b=CnE9XebjBVLyAjXzamhUptzht6dwG8q80KQ1kVGHx77DGRP8KYDd4cRjhna9imAXv1 /G4md/asfbz1PixOSxCxzKZwk5OvdX4q7v8MiA3fqK6YJjOYFVyOTxek/yBiAKxsFDO3 4JZMVZGjq0FPbFYuR0pwVRL3NXht/2C1u4wdEV1PpiyeWyKu52K+HH6QFbxEeCpKX6YC 8FGWU5O+vqv7HPXnkyRuftKNwr5/QKbuxNsae4kBzNpIC/XFaW6bcpexMYlOBwPGF+40 d2yxt2W8TjqmM0YROlpOwrkn01DT3YdqNMExoWY7AteWQKEYGI8pDvnIr1OSsavz1ClC 06Mg== MIME-Version: 1.0 References: <20190731085757.GX15878@tucnak> <20190731091215.f5yjjaevmvhiyma4@kam.mff.cuni.cz> <20190801131932.GA2726@tucnak> <089f4b9d-a29a-5031-a272-e005cb5ce78c@suse.cz> <20190801144126.GC2726@tucnak> <20190802074450.GG2726@tucnak> <20190802085001.GK2726@tucnak> In-Reply-To: <20190802085001.GK2726@tucnak> From: Richard Biener Date: Fri, 02 Aug 2019 09:04:00 -0000 Message-ID: Subject: Re: [PATCH] Properly detect working jobserver in gcc driver. To: Jakub Jelinek Cc: =?UTF-8?Q?Martin_Li=C5=A1ka?= , Jan Hubicka , Jeff Law , GCC Patches , Michael Matz Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2019-08/txt/msg00120.txt.bz2 On Fri, Aug 2, 2019 at 10:50 AM Jakub Jelinek wrote: > > On Fri, Aug 02, 2019 at 10:47:10AM +0200, Martin Li=C5=A1ka wrote: > > > Can you strace if other fds are opened and not closed in the spot you= had it > > > before? Advantage of doing it there is that it will not be done for = all the > > > -E/-S/-c compilations when the linker is not spawned. > > > > I've used the same trick which you used and I'm attaching the output. > > I believe it's fine, I can't see any opened fd by GCC. > > LGTM. Btw, we discussed yesterday on the phone and the conclusion was to make -flto auto-detect a job-server (but not fall back to # of threads) and add -flto=3Dauto to auto-detect a job-server and fall back to # of thre= ads. That basically makes -flto=3Djobserver the default behavior which means we should document -flto=3D1 as a way to override jobserver detection. We also discussed carrying distribution-local patches to make GNU make expose the jobserver FD to all jobs, not just those marked to make the new default effective. Does anybody have an idea if there's another common enough make that would benefit from support? That is, do other "jobserver" implementations/APIs exist we could support or is another make used in more than 1% of build systems? Richard. > > gcc/ChangeLog: > > > > 2019-08-02 Martin Liska > > > > * gcc.c (driver::maybe_run_linker): Call detect_jobserver > > to detect working job server. > > (driver::detect_jobserver): Test whether jobserver > > is active from GCC driver. That will prevent situation where > > GCC is invoked from a LD plugin and the linker already uses > > file descriptors suggested by make. That leads to a wrong > > detection. > > * gcc.h (driver): Add detect_jobserver. > > * lto-wrapper.c (jobserver_active_p): Simplify sscanf by > > not scanning for --jobserver-auth prefix. > > --- > > gcc/gcc.c | 42 ++++++++++++++++++++++++++++++++++++++++++ > > gcc/gcc.h | 1 + > > gcc/lto-wrapper.c | 2 +- > > 3 files changed, 44 insertions(+), 1 deletion(-) > > > > diff --git a/gcc/gcc.c b/gcc/gcc.c > > index a4323eb146e..18a07426290 100644 > > --- a/gcc/gcc.c > > +++ b/gcc/gcc.c > > @@ -8268,6 +8268,8 @@ driver::maybe_run_linker (const char *argv0) const > > { > > int tmp =3D execution_count; > > > > + detect_jobserver (); > > + > > if (! have_c) > > { > > #if HAVE_LTO_PLUGIN > 0 > > @@ -8357,6 +8359,46 @@ driver::final_actions () const > > } > > } > > > > +/* Detect whether jobserver is active and working. If not drop > > + --jobserver-auth from MAKEFLAGS. */ > > + > > +void > > +driver::detect_jobserver () const > > +{ > > + /* Detect jobserver and drop it if it's not working. */ > > + const char *makeflags =3D env.get ("MAKEFLAGS"); > > + if (makeflags !=3D NULL) > > + { > > + const char *needle =3D "--jobserver-auth=3D"; > > + const char *n =3D strstr (makeflags, needle); > > + if (n !=3D NULL) > > + { > > + int rfd =3D -1; > > + int wfd =3D -1; > > + > > + bool jobserver > > + =3D (sscanf (n + strlen (needle), "%d,%d", &rfd, &wfd) =3D=3D= 2 > > + && rfd > 0 > > + && wfd > 0 > > + && fcntl (rfd, F_GETFD) >=3D 0 > > + && fcntl (wfd, F_GETFD) >=3D 0); > > + > > + /* Drop the jobserver if it's not working now. */ > > + if (!jobserver) > > + { > > + unsigned offset =3D n - makeflags; > > + char *dup =3D xstrdup (makeflags); > > + dup[offset] =3D '\0'; > > + > > + const char *space =3D strchr (makeflags + offset, ' '); > > + if (space !=3D NULL) > > + strcpy (dup + offset, space); > > + xputenv (concat ("MAKEFLAGS=3D", dup, NULL)); > > + } > > + } > > + } > > +} > > + > > /* Determine what the exit code of the driver should be. */ > > > > int > > diff --git a/gcc/gcc.h b/gcc/gcc.h > > index a0a1d94c6e6..dc77dba67fb 100644 > > --- a/gcc/gcc.h > > +++ b/gcc/gcc.h > > @@ -51,6 +51,7 @@ class driver > > void do_spec_on_infiles () const; > > void maybe_run_linker (const char *argv0) const; > > void final_actions () const; > > + void detect_jobserver () const; > > int get_exit_code () const; > > > > private: > > diff --git a/gcc/lto-wrapper.c b/gcc/lto-wrapper.c > > index 353187c6043..3414adedd26 100644 > > --- a/gcc/lto-wrapper.c > > +++ b/gcc/lto-wrapper.c > > @@ -1234,7 +1234,7 @@ jobserver_active_p (void) > > int rfd =3D -1; > > int wfd =3D -1; > > > > - return ((sscanf(n, "--jobserver-auth=3D%d,%d", &rfd, &wfd) =3D=3D 2) > > + return (sscanf (n + strlen (needle), "%d,%d", &rfd, &wfd) =3D=3D 2 > > && rfd > 0 > > && wfd > 0 > > && fcntl (rfd, F_GETFD) >=3D 0 > > -- > > 2.22.0 > > > > > Jakub