From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 25132 invoked by alias); 18 Sep 2018 04:31:24 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 25109 invoked by uid 89); 18 Sep 2018 04:31:23 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=0.1 required=5.0 tests=AWL,BAYES_00,KAM_LAZY_DOMAIN_SECURITY,RDNS_DYNAMIC,TVD_RCVD_IP autolearn=no version=3.3.2 spammy=H*Ad:D*google.com X-HELO: brightrain.aerifal.cx Date: Tue, 18 Sep 2018 04:31:00 -0000 From: Rich Felker To: Adhemerval Zanella Cc: Martin Buchholz , Zack Weinberg , GNU C Library Subject: Re: system and popen fail in case of big application Message-ID: <20180918043117.GT17995@brightrain.aerifal.cx> References: <0caab9ea-5c2c-fe37-385b-090b21dd3cbb@linaro.org> <01ffd458-794b-6d05-0635-f13437bec096@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: Rich Felker X-SW-Source: 2018-09/txt/msg00270.txt.bz2 On Mon, Sep 17, 2018 at 06:58:56PM -0700, Adhemerval Zanella wrote: > >> For former I see there is no easy way to provide a similar closefrom > >> function without either being racy (as *BSD and openjdk implementations) > >> or to remove some scalability by adding serialization (for open* syscall > >> to avoid create new FD while closeall is closing them). A better alternative > >> would to request for some kernel helper, but I am not convinced that > >> current way of explicit open all file descriptors with O_CLOEXEC is not > >> the better option. Unfortunately it does not help run external code > >> (through shared libraries) that still call open in default mode. > > > > It seems like traditional methods to create file descriptors will > > never set the CLOEXEC flag by default, and most programmers will not > > be aware of the problem, so those of us implementing subprocess > > creation cannot ever rely on all the file descriptors having the flag > > set. > > But from a QoI standpoint O_CLOEXEC is still the better option. And f we > aim to add such extension on posix_spawn, I will expect we also push for > some kernel support. There are reports where trying to interacting over > all possible FD takes a lot of time [1]. > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=837033 While closeall is an awful operation that should not be done (fix stuff to use O_CLOEXEC), looping over the whole fd range and making a close syscall for each probably-already-closed one is the most awful possible way to implement it. You can make it fast by calling poll with 1000 or more fds at a time in the pollfd set, checking revents for POLLNVAL to determine which ones are open, and only close those. Rich