From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15313 invoked by alias); 31 Jul 2019 07:49:15 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 15304 invoked by uid 89); 31 Jul 2019 07:49:14 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-4.9 required=5.0 tests=AWL,BAYES_00,SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mx1.suse.de Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 31 Jul 2019 07:49:13 +0000 Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 3DA23AE5C; Wed, 31 Jul 2019 07:49:11 +0000 (UTC) Subject: Re: [PATCH] Deduce automatically number of cores for -flto option. To: Jan Hubicka , Jakub Jelinek Cc: Jeff Law , gcc-patches@gcc.gnu.org, Michael Matz , Richard Biener References: <95f20c0a-81a3-2319-326b-3c5baf71e2d1@suse.cz> <20190723092040.czs64os2qymax7sq@kam.mff.cuni.cz> <69d3f527-4086-da6c-e789-469dc389d9eb@redhat.com> <20190730233230.GS15878@tucnak> <87d4b006-bb01-a608-7abe-680dffcc937a@suse.cz> <20190731073407.GU15878@tucnak> <20190731074019.b4l6pwcpeh2q2zai@kam.mff.cuni.cz> From: =?UTF-8?Q?Martin_Li=c5=a1ka?= Message-ID: <217ef7a3-7726-04fa-3bd5-ca5b10844091@suse.cz> Date: Wed, 31 Jul 2019 07:50:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190731074019.b4l6pwcpeh2q2zai@kam.mff.cuni.cz> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2019-07/txt/msg01846.txt.bz2 On 7/31/19 9:40 AM, Jan Hubicka wrote: >> Neither will work very well, we have thousands of -flto tests outside of >> lto.exp, e.g. dg-torture.exp (or libgomp and others) cycle through various >> options including -flto etc. >> >> Some env var would be useful I guess, though shouldn't it have GCC in the >> name? I mean, if we run into these fork-bomb problems in gcc, won't other >> projects run into those as well? >> >> Why doesn't the jobserver work in the tests? Is that because of missing + >> somewhere in the Makefiles or is something unsetting MFLAGS or MAKEFLAGS >> env vars? > > Main trouble with make's jobserver is that it works by > 1) defining environment variable saying which file descriptior to > connect to > 2) keeping the file descriptor open upon invoking "+" prefixed lines > > Adding "+" to GCC invocation is wrong since it breaks dry run (we do not > want to link at that time) but it is only way to access the jobserver. > If "+" is not present, make will keep the environment vairable but will > close file descriptors prior exec. > > Make developers said that this is because some old prorams misbehave > when you exec them with more than 3 file descriptors open. I tried to > negotiate for named pipe which would solve this and it would make it > easy to connect to outermost jobserver from anything invoked form > toplevel make, but they was worried about systems w/o named pipes. Yes a more generic approach would be welcome as other build systems would also be able to utilize it. > > I wonder why we do not detect jobserv as unavailable in this case and do > not default to -flto=? We do not detect jobserver because of Dejagnu is not using it. And yes, we default to -flto= in LTO tests now. That's what is causing issues right now. > Is it because dejagnu machinery actually opens some other file > descriptor that gets same ID and executes us with it? > > Or does LTO wrapper open something prior accessing jobserver? > >> >> Though, as I said on IRC, I think we might run out of filedescriptors when >> using jobserver too, say if on 64 thread machine one does make -j64 -k check >> and each test simultaneously tries to create 64 partitions, that would be >> 4096 connections to the jobserver, right? > > Only WPA process connects to jobserver (which is 1 per linker > invokation), so I think this should be safe. Yes and it's only about a quick fcntl. But as mentioned, Dejagnu is not passing us jobserver, so we don't do it right now. Martin > > Honza >> >> Jakub