From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by sourceware.org (Postfix) with ESMTPS id 91452386EC23 for ; Wed, 12 May 2021 09:21:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 91452386EC23 Received: by mail-ej1-x629.google.com with SMTP id c22so5835111ejd.12 for ; Wed, 12 May 2021 02:21:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=/cSeWMKu+2R63rG9Esq9a3kVh63gRjc9WRvzwDi4pgU=; b=Gy7tZ9zUviKHNE3FTe7cdf4Bw6dcDaey9XrhG7MSIAmo7xHi8PueKO0tOpy2ddlrQc yi15raqIiN4AKcCb6k5/MkFY3bQXAubOKwMGw+GvhFMv6nUlxpWCi8R47N80IEW9yl0v QoGcUVEkk2fU3vFwpBnoLNr3oa53jzqvyuhEtgQHY+wNo4BNqOQ1oVBdDEVBI3f+lobw 2b/WWbfbp4sK2atkV0emnyiWHZpiajUzcPfYPKpDq9xJwwurgbL/gexI5ER6RJ55Ek/0 sGB7nhNJbcVqxJopz0dP9Rbq8USs51ZkywcEM/4MqlNRPkZgw/gQHZ33G4mCU2pwMXfu H6fw== X-Gm-Message-State: AOAM531VNHjmEGC1B3r3hpzsFdSy4UczQ61tMqTzhgX1IDd97GT6dKgX Kx2HbhWtgPM78y16CVpnjjfWh1DzhU6z7bDfU7TlCzTrxQY= X-Google-Smtp-Source: ABdhPJxJjxnri1PI1UABUIHNZj7xxHVc3fpM6DLdhxFYaSsuTYGTrwHd9jAgHwto8OvCOKEKdhZsvN9iFBor1THif5U= X-Received: by 2002:a17:906:cd27:: with SMTP id oz39mr36447617ejb.129.1620811310678; Wed, 12 May 2021 02:21:50 -0700 (PDT) MIME-Version: 1.0 References: <11a45856-44b3-754d-5a42-19a348d0d59d@suse.cz> <8cc12cd9-fe88-f97b-eec2-d47f5342496d@suse.cz> <73eaf32c-3776-9345-ec5f-57d43e94ecfe@suse.cz> In-Reply-To: From: Richard Biener Date: Wed, 12 May 2021 11:21:39 +0200 Message-ID: Subject: Re: [PATCH] LTO: fallback to -flto=N if -flto=jobserver does not work. To: =?UTF-8?Q?Martin_Li=C5=A1ka?= Cc: GCC Patches Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-8.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2021 09:21:54 -0000 On Wed, May 12, 2021 at 11:10 AM Martin Li=C5=A1ka wrote: > > May I please ping this Richi? OK. Thanks, Richard. > Thanks, > Martin > > On 4/22/21 4:30 PM, Martin Li=C5=A1ka wrote: > > On 4/22/21 2:47 PM, Richard Biener wrote: > >> On Thu, Apr 22, 2021 at 2:21 PM Martin Li=C5=A1ka wro= te: > >>> > >>> On 4/22/21 1:19 PM, Richard Biener wrote: > >>>> On Thu, Apr 22, 2021 at 11:02 AM Martin Li=C5=A1ka = wrote: > >>>>> > >>>>> On 4/22/21 10:04 AM, Richard Biener wrote: > >>>>>> On Wed, Apr 21, 2021 at 3:08 PM Martin Li=C5=A1ka = wrote: > >>>>>>> > >>>>>>> When -flto=3Djobserver is used and we cannot detect job server, t= hen we can > >>>>>>> still fallbackto -flto=3DN mode. > >>>>>>> > >>>>>>> Patch can bootstrap on x86_64-linux-gnu and survives regression t= ests. > >>>>>>> > >>>>>>> Ready to be installed? > >>>>>> > >>>>>> I think this behavior needs to be documented - it falls back to a = less > >>>>>> conservative (possibly system overloading) mode - which IMHO is > >>>>>> non-obvious and IMHO we shouldn't do. > >>>>> > >>>>> Sure, I'm sending corresponding patch. Note that it's quite common = mistake > >>>>> that '+' is missing in Makefile rule. That was motivation for my ch= ange. > >>>> > >>>> Sure, but that change won't get this fixed. > >>> > >>> It will as linker command line will contain (-flto=3Djobserver) and L= TO will fallback to -flto=3DN. > >>> > >>>> IMHO we should eventually > >>>> emit diagnostic like > >>>> > >>>> warning: could not find jobserver, compiling N jobs serially > >>>> > >>>> once N > 1 (or 2?). > >>> > >>> We do that now (for all N): > >>> lto-wrapper: warning: jobserver is not available: =E2=80=98MAKEFLAGS= =E2=80=99 environment variable is unset > >>> > >>> > >>>> Likewise if people just use -flto and auto-detection > >>>> finds nothing: > >>> > >>> -flto !=3D -flto=3Dauto > >>> > >>> Yes, -flto is a serial linking and we can emit a warning. > >> > >> I'd avoid warning if there's just a single ltrans unit. > > > > That's doable and I've just done that in the attached patch. > > Two disadvantages: > > - one needs waiting for the warning after WPA > > - source change (>1 LTRANS) can trigger the warning > > > >> > >>>> warning: using serial compilation of N LTRANS jobs > >>>> note: refer to http://.... for how to use parallel compile > >>>> > >>>> using the URL diagnostics to point to -flto=3D... documentation. > >>> > >>> What about making that a proper warning (-Wlto)? We have diagnostics = infrastructure > >>> that prints URL links. > >> > >> Note that drivers like lto-wrapper do not have fully initialized diagn= ostic > >> machinery and use a "different" set of overloads (likewise gen* progra= ms). > > > > I managed printing the warning: > > > > lto-wrapper: warning: jobserver is not available: =E2=80=98MAKEFLAGS=E2= =80=99 environment variable is unset > > > > lto-wrapper: note: see the =E2=80=98-flto=E2=80=99 option documentation= for more information > > > > > > and > > > > lto-wrapper: warning: using serial compilation of 128 LTRANS jobs > > > > lto-wrapper: note: see the =E2=80=98-flto=E2=80=99 option documentation= for more information > > > > > >> > >>>> > >>>> That is, teach users rather than second-guessing and eventually > >>>> blowing things up. IMHO only the jobserver mode is safe to > >>>> automatically use. > >>> > >>> Well, -flto=3Dauto is also fine and document. I think there is no pos= sibility > >>> auto CPU deduction can fail. So -flto=3Djobserver (with missing make = job server) > >>> and -flto (equal to -flto=3D1) worth emitting a warning. > >>> > >>> What do you think? > >> > >> Yes, that sounds reasonable. I suspect that people might want to see > >> -flto default to -flto=3Dauto but then I don't think that's good becau= se there's > >> no system wide job scheduler limiting things (systemd-jobserver anyone= ?) > > > > Done that. > > > > Thoughts? > > Martin > > > >> > >> Richard. > >> > >>> Martin > >>> > >>>> > >>>> Richard. > >>>> > >>>>> Martin > >>>>> > >>>>>> > >>>>>> Richard. > >>>>>> > >>>>>>> Thanks, > >>>>>>> Martin > >>>>>>> > >>>>>>> gcc/ChangeLog: > >>>>>>> > >>>>>>> * lto-wrapper.c (run_gcc): When -flto=3Djobserver is use= d, but the > >>>>>>> makeserver cannot be detected, then use -flto=3DN fallba= ck. > >>>>>>> --- > >>>>>>> gcc/lto-wrapper.c | 3 ++- > >>>>>>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>>>>>> > >>>>>>> diff --git a/gcc/lto-wrapper.c b/gcc/lto-wrapper.c > >>>>>>> index 03a5922f8ea..0b626d7c811 100644 > >>>>>>> --- a/gcc/lto-wrapper.c > >>>>>>> +++ b/gcc/lto-wrapper.c > >>>>>>> @@ -1585,8 +1585,9 @@ run_gcc (unsigned argc, char *argv[]) > >>>>>>> if (jobserver && jobserver_error !=3D NULL) > >>>>>>> { > >>>>>>> warning (0, jobserver_error); > >>>>>>> - parallel =3D 0; > >>>>>>> + /* Fall back to auto parallelism. */ > >>>>>>> jobserver =3D 0; > >>>>>>> + auto_parallel =3D 1; > >>>>>>> } > >>>>>>> else if (!jobserver && jobserver_error =3D=3D NULL) > >>>>>>> { > >>>>>>> -- > >>>>>>> 2.31.1 > >>>>>>> > >>>>> > >>> > > >