From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from re-prd-fep-042.btinternet.com (mailomta14-re.btinternet.com [213.120.69.107]) by sourceware.org (Postfix) with ESMTPS id 8F3EB3858CDA for ; Thu, 30 Mar 2023 20:34:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8F3EB3858CDA Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=dronecode.org.uk Authentication-Results: sourceware.org; spf=none smtp.mailfrom=dronecode.org.uk Received: from re-prd-rgout-002.btmx-prd.synchronoss.net ([10.2.54.5]) by re-prd-fep-042.btinternet.com with ESMTP id <20230330203448.VVSB30996.re-prd-fep-042.btinternet.com@re-prd-rgout-002.btmx-prd.synchronoss.net>; Thu, 30 Mar 2023 21:34:48 +0100 Authentication-Results: btinternet.com; auth=pass (PLAIN) smtp.auth=jonturney@btinternet.com; bimi=skipped X-SNCR-Rigid: 63FE976D0370A36D X-Originating-IP: [81.153.98.181] X-OWM-Source-IP: 81.153.98.181 (GB) X-OWM-Env-Sender: jonturney@btinternet.com X-VadeSecure-score: verdict=clean score=0/300, class=clean X-RazorGate-Vade: gggruggvucftvghtrhhoucdtuddrgedvhedrvdehledgheelucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuueftkffvkffujffvgffngfevqffopdfqfgfvnecuuegrihhlohhuthemuceftddunecunecujfgurhepkfffgggfuffvfhfhjggtgfesthekredttdefjeenucfhrhhomheplfhonhcuvfhurhhnvgihuceojhhonhdrthhurhhnvgihsegurhhonhgvtghouggvrdhorhhgrdhukheqnecuggftrfgrthhtvghrnhepvddtteffkeevveejgeehgeelhfdtgefgieelgffgudetudefvdeggfeiiefftdevnecukfhppeekuddrudehfedrleekrddukedunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehhvghloheplgduledvrdduieekrddurddutdeingdpihhnvghtpeekuddrudehfedrleekrddukedupdhmrghilhhfrhhomhepjhhonhdrthhurhhnvgihsegurhhonhgvtghouggvrdhorhhgrdhukhdpnhgspghrtghpthhtohepvddprhgtphhtthhopefuthhrohhmvghkohesnhgvgihgohdruggvpdhrtghpthhtoheptgihghifihhnqdgrphhpshestgihghifihhnrdgtohhmpdhrvghvkffrpehhohhsthekuddqudehfedqleekqddukedurdhrrghnghgvkeduqdduheefrdgsthgtvghnthhrrghlphhluhhsrdgtohhmpdgruhhthhgpuhhsvghrpehjohhnthhurhhnvgihsegsthhinhhtvghrnhgvthdrtghomhdpghgvohfkrfepifeupdfovfetjfhoshhtpehrvgdq phhrugdqrhhgohhuthdqtddtvd X-RazorGate-Vade-Verdict: clean 0 X-RazorGate-Vade-Classification: clean Received: from [192.168.1.106] (81.153.98.181) by re-prd-rgout-002.btmx-prd.synchronoss.net (5.8.814) (authenticated as jonturney@btinternet.com) id 63FE976D0370A36D; Thu, 30 Mar 2023 21:34:48 +0100 Message-ID: <197d05e2-2308-3a18-abbd-f6ced1c123aa@dronecode.org.uk> Date: Thu, 30 Mar 2023 21:34:47 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [PATCH cygport] lib/src_postinst.cygpart: parallelize __prepstrip Content-Language: en-GB To: Achim Gratz , "cygwin-apps@cygwin.com" Newsgroups: gmane.os.cygwin.applications References: <822a414c-be34-319e-c8d9-810cf7383e7e@dronecode.org.uk> <878rfec94o.fsf@Rainer.invalid> From: Jon Turney In-Reply-To: <878rfec94o.fsf@Rainer.invalid> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1189.7 required=5.0 tests=BAYES_00,BODY_8BITS,FORGED_SPF_HELO,KAM_DMARC_STATUS,KAM_LAZY_DOMAIN_SECURITY,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_NONE,TXREP autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 30/03/2023 20:49, Achim Gratz via Cygwin-apps wrote: > Jon Turney via Cygwin-apps writes: >> On 26/03/2023 19:12, Jon Turney via Cygwin-apps wrote: >>>> -        usr/lib/gcc/*/lib*|usr/lib/gcc/*/*.o) >>>> +        usr/lib/gcc/*/*.o) >>> Why this change? > > It looks like a mistake that I didn't catch. > >>> >>>> +    local nproc=$(nproc) >>> This limit should probably be taken from the --jobs command line >>> parameter, if specified > > Yes, although one could argue that it should actually oversubscribe > since the CPU load per process is expected to be significantly less than > 100% per process. > >> Looking at this a bit more, a couple of perhaps more serious problems: >> >> * The parallel invocations of __prepstrip_one are all appending to >> ${T}/.dbgsrc.out >> >> I don't see what makes that safe against interleaving of the output. > > Line-buffering and the line being shorter than the buffer should. and what causes line-buffering be on? >> It's probably possible to have each instance write to a separate file >> and collect them together in __prepdebugsrc > > Nah, if you insist on making it _really_ safe regardless of line lenght > and buffer size vagaries I'll do file locking on the output file. If you lock the file while objdump is running, you've basically serialized this again... >> * This patch causes several failures in the testsuite, e.g. with >> autotools/c testcase. > > Which? The full list of failing tests is: 21/54 autotools/c FAIL 63.16s exit status 1 22/54 autotools/gnome FAIL 170.49s exit status 1 23/54 autotools/gtkmm FAIL 177.54s exit status 1 24/54 autotools/mate FAIL 131.24s exit status 1 25/54 autotools/xfce FAIL 85.64s exit status 1 26/54 cmake/c FAIL 13.75s exit status 1 28/54 cmake/qt5 FAIL 94.44s exit status 1 37/54 lua/all FAIL 10.92s exit status 1 38/54 httpd/apxs FAIL 31.98s exit status 1 44/54 perl/Module-Build FAIL 16.09s exit status 1 46/54 php/pecl FAIL 48.13s exit status 1 49/54 qmake/qt5 FAIL 46.84s exit status 1 > >> On a brief attempt at debugging, it this looks like it's due to not >> waiting for all the __prepstrip_one to complete before moving on, but >> I think the final wait should prevent that, so idk. > > I've seen an indication that the final wait doesn't work, but that is > fixable by a sleep apparently. Did You see that the process number > limiting doesn't work? No, but I haven't been trying to test it. >> I'm not clear that invoking 'jobs', is actually doing anything, if >> job-control is turned off in a non-interactive shell? > > No, "jobs" shouldn't do anything, but wait should still work I think > (the manpage talks about jobs, but it really means "children"). But > then again, a bit of googling tells me that the bashism "wait -n" > actually needs job control to be enabled, natch.