From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by sourceware.org (Postfix) with ESMTPS id A36813858D1E for ; Tue, 20 Feb 2024 03:42:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A36813858D1E Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A36813858D1E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::32c ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1708400565; cv=none; b=PofUyW8cubX5K5wfhwkghZ08by4RgePpq8pikOYMWfC5oWWV8afwLNe445KWOQptoAgzLGOY0k0HIDZKbyy+olvOjXhPILjLr/9/EW7VjY3GNji32yUryA1noIalAmJajlbQJaIDQjXMRgFCoCOw4SsHSTVo0q+tRFPl8ZRSFOs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1708400565; c=relaxed/simple; bh=XQSRtQZNb1k1hIgli3WvYV1ShOcXfGFly/bLzThtCdQ=; h=DKIM-Signature:Message-ID:Date:MIME-Version:Subject:To:From; b=IOh2+Arl6HBi2ofH0WeTPHeaGUEU4H0VSblAhJSEaCZkddexGI8epe1uCKnFzE98o1voW0O9HJzVu6PRFwmY5MyRkZpQ5gRNxuCGHxalPnW2YrEoF6U/9GEFLHsnuW65SqgcgTmzBQE6eJ5c7VVHbJbBSuFvzhWsATS5m5X1tcs= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-4126fda8ef8so408305e9.2 for ; Mon, 19 Feb 2024 19:42:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708400561; x=1709005361; darn=cygwin.com; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=Dz8WrzgqAxKL9ZMuyIOJFRS3f/QMMwJCpgszfc/ZDRc=; b=kuUQ5llkon6NWOj6FSqdX8Iu8wFaH73R65l+BUP5xR25WZnRhobMkRMxLCTDSdCHpC u1dlVkz8eqh5OBPSr3Fs0ajaEkkYge4x44prEiW8BtN+0Nd6hB6jODhnSN+YJY+Th3Dx 5pH9oOVGJ82mC4BObcgRECcyYVUXj1wXKDfVW4RVODK5057Njk8xfI9e6V6cy1QJXTqQ vKYXD70npofiFMZOJ1Td0F2Gi+sYRfwVmM6wpLoEYXSU7Z2JblvWmV7O2sgckwRyZJ+X DwHQYhwRRQ+iQaBboiuBPv/KZMetKu8skPNT4JLtIhccsUBkFaz7JDwumDjS2mmWiHDy cxsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708400561; x=1709005361; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Dz8WrzgqAxKL9ZMuyIOJFRS3f/QMMwJCpgszfc/ZDRc=; b=DkCdB1zXkXuyMq2aKSaHcFww6HeEwZjV+3YQOpjJgZcLlc1ExZuZ8OPy7Sd+9E2O+K Z48vVs8k2+l80h+n/92rFPMrdb4l5YYRF6NXRLMLUNdP42NfKeFYThrzXj8LZ1TVdXqo n5GBXDn4XnzE23Pu41snM1iFW6G3wbhB4KA/k65wfBJMtz8l4P9rUIj+LqMXvqHD18c/ 6ONPRhX59agPHqkCtQ9iiMv4L4hy5TVbrx2I33h1kIVxvXToK6GZsVbZsUzI3YorC1I1 ezBwS/67VuLd93Wp9IbTvZTuFPGw8i0t5EzVAtQ9f1MGUwF3FH7WHs7SorbqHubakiiz HnuA== X-Gm-Message-State: AOJu0Yzvsrv3pPRKfrar19FMH016OGQn53L557yl+icJW2EDDwW8Kt5e AOC30iGFPlAf8YtObYih9+5Hhy0Zxk99shhoERt3ZdAyb+30JJNJlar/ktC0 X-Google-Smtp-Source: AGHT+IECMAKfPvpNF8xTjLVZbAdktTjC8znaD2Tpmld+YwuBOnhCKdME65SBJ7JzM9GfFLL4AjdzkA== X-Received: by 2002:a05:600c:3ba6:b0:411:d89d:d7ba with SMTP id n38-20020a05600c3ba600b00411d89dd7bamr12933706wms.7.1708400561117; Mon, 19 Feb 2024 19:42:41 -0800 (PST) Received: from ?IPV6:2001:a61:1224:9301:55bc:2362:1d8:3b72? ([2001:a61:1224:9301:55bc:2362:1d8:3b72]) by smtp.gmail.com with ESMTPSA id q2-20020a056000136200b0033d12895cfdsm12177792wrz.61.2024.02.19.19.42.40 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 19 Feb 2024 19:42:40 -0800 (PST) Message-ID: <2088b3bf-496e-4b71-8228-531955fe6298@gmail.com> Date: Tue, 20 Feb 2024 04:42:40 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [cygport] enabling a replacement for "objdump -d -l" Content-Language: en-US To: cygwin-apps@cygwin.com References: <87a5nx5z5e.fsf@Gerda.invalid> From: Marco Atzeri In-Reply-To: <87a5nx5z5e.fsf@Gerda.invalid> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 18/02/2024 20:51, ASSI via Cygwin-apps wrote: > > Cygport uses "objdump -d -l" to extract the list of source files that > need to be copied into the debuginfo package. This operation triggers > some O(N²) or even higher complexity and in addition has been getting > slower in recent binutils releases due to more and more information > being put into the object files. For gcc-11 extracting the debug source > files takes up to 45 minutes per executable (up from about 15 minutes > until 2.39) and for gcc-13 (with about 1.5 times the number of lines to > extract) it is already taking more than two hours. So if you just > package gcc-13 using a single thread you'd be looking on the order of 20 > hours wall clock time, which is unacceptable. > > The deassembly implied by the "-d" (which is not the part that has the > superlinear complexity btw, but produces a baseline of 2 hours single > thread runtime all by itself) is also unnecessary to extract just the > filenames of the source files as we throw away the location information > anyway and so I've written a small parser that works on the DWARF dump > instead (which can be produced in linear time with a very small scaling > factor, so practically constant time even for very large executables). > Unfortunately binutils does not yet offer a machine readable format for > these dumps, but parsing the text is not too difficult even though the > format is undocumented. The DWARF-5 documentation isn't the most > enjoyable read, but it was helpful enough to figure it all out. I've > also integrated the filtering of unrelated source file information (from > system headers and external libraries). The end result is the same > runtime as before on small object files, a factor up to 100 speedup for > medium sized object files and speedups in the several thousands range > for large sized ones (or a total single-thread runtime of less than 20 > seconds for gcc-13). > > > Integration into cygport is made configurable via a variable to be set > in .cygportrc for instance in order to easily revert back to the > original objdump invocation if necessary. I've been producing packages > with that setup for a while now and have not noticed any errors. In > principle the new parser actually produces more complete output as there > can be multiple line number statements and hence source files per > location, but objdump only lists one of them in the disassembly (at > least sometimes). In practise I haven't found a package until now where > the final list (after filtering) is different. > if works should not be the default ? Reducing that time is very interesting for the big stuff > > Regards, > Achim. Thanks Marco