From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 60517 invoked by alias); 31 Jul 2019 07:20:45 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 60400 invoked by uid 89); 31 Jul 2019 07:20:45 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-17.4 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KAM_SHORT,SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mx1.suse.de Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 31 Jul 2019 07:20:43 +0000 Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 5D7BFAF4C; Wed, 31 Jul 2019 07:20:41 +0000 (UTC) Subject: Re: [PATCH] Deduce automatically number of cores for -flto option. To: Jakub Jelinek Cc: Jeff Law , Jan Hubicka , gcc-patches@gcc.gnu.org, Michael Matz , Richard Biener References: <95f20c0a-81a3-2319-326b-3c5baf71e2d1@suse.cz> <20190723092040.czs64os2qymax7sq@kam.mff.cuni.cz> <69d3f527-4086-da6c-e789-469dc389d9eb@redhat.com> <20190730233230.GS15878@tucnak> From: =?UTF-8?Q?Martin_Li=c5=a1ka?= Message-ID: <87d4b006-bb01-a608-7abe-680dffcc937a@suse.cz> Date: Wed, 31 Jul 2019 07:24:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190730233230.GS15878@tucnak> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2019-07/txt/msg01841.txt.bz2 On 7/31/19 1:32 AM, Jakub Jelinek wrote: > This broke a lot of tests. Whoops. > The logs show > spawn -ignore SIGHUP /home/jakub/src/gcc/obj31/gcc/xgcc -B/home/jakub/src/gcc/obj31/gcc/ c_lto_pr83954_0.o c_lto_pr83954_1.o -fno-diagnostics-show- > caret -fno-diagnostics-show-line-numbers -fdiagnostics-color=never -O2 -flto -flto-partition=1to1 -o gcc-dg-lto-pr83954-31.exe > make[4]: *** write jobserver: Bad file descriptor. Stop. > make[4]: *** Waiting for unfinished jobs.... > make[4]: *** write jobserver: Bad file descriptor. Stop. > lto-wrapper: fatal error: make returned 2 exit status > compilation terminated. > collect2: fatal error: lto-wrapper returned 1 exit status > compilation terminated. > compiler exited with status 1 > FAIL: gcc.dg/lto/pr83954 c_lto_pr83954_0.o-c_lto_pr83954_1.o link, -O2 -flto -flto-partition=1to1 > and similar, e.g. for x86_64-linux it was following regressions, on > i686-linux also some tests in libgomp etc. > Is -flto now really using all available CPUs for each compilation? Yes, I can confirm that linking happens in N processed for each LTO test now. It's caused by fact that current Dejagnu machinery does not pass a make jobserver to gcc command invocations. On the other hand, our LTO tests are so tiny that we should have always very low number of partitions. > Without > jobserver that would like a fork-bomb, say if I have a CPU with 32 threads > and do make check -j32, does that mean there are 1024 lto1s? > Judging from http://gcc.gnu.org/ml/gcc-testresults/2019-07/msg03610.html > I'm not alone. One possible solution will be to adjust lto.exp: set LTO_OPTIONS [list \ {-O0 -flto -flto-partition=none -fuse-linker-plugin} \ {-O2 -flto -flto-partition=none -fuse-linker-plugin -fno-fat-lto-objects } \ {-O0 -flto -flto-partition=1to1 -fno-use-linker-plugin } \ {-O2 -flto -flto-partition=1to1 -fno-use-linker-plugin } \ and replace all -flto with -flto=1. But still, many individual tests set -flto by themselves. Another solution would be to disable the auto-detection with an environment variable: diff --git a/gcc/lto-wrapper.c b/gcc/lto-wrapper.c index 353187c6043..bb6fb2b53ff 100644 --- a/gcc/lto-wrapper.c +++ b/gcc/lto-wrapper.c @@ -1423,7 +1423,7 @@ run_gcc (unsigned argc, char *argv[]) auto_parallel = 0; parallel = 0; } - else if (!jobserver && parallel) + else if (!jobserver && parallel && !getenv ("LTO_NO_AUTO_PARALLEL")) { /* If there's no explicit usage of jobserver and parallel is enabled, then automatically detect diff --git a/gcc/testsuite/lib/lto.exp b/gcc/testsuite/lib/lto.exp index 25c934731df..e303894e0b0 100644 --- a/gcc/testsuite/lib/lto.exp +++ b/gcc/testsuite/lib/lto.exp @@ -209,6 +209,8 @@ proc lto_init { args } { ] } } + + setenv LTO_NO_AUTO_PARALLEL 1 } # Can you Jakub test the patch or the s/-flto/-flto=1 solutions please? Thanks, Martin