From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 72A053857378 for ; Fri, 15 Apr 2022 13:11:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 72A053857378 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from tarox.wildebeest.org (83-87-18-245.cable.dynamic.v4.ziggo.nl [83.87.18.245]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 2B04730002FC; Fri, 15 Apr 2022 15:11:56 +0200 (CEST) Received: by tarox.wildebeest.org (Postfix, from userid 1000) id 3529A413CD15; Fri, 15 Apr 2022 15:11:55 +0200 (CEST) Message-ID: <3ca917c7b98423ad5bc711900a7d46753fffc48a.camel@klomp.org> Subject: Re: parallel downloads of multiple debuginfo files From: Mark Wielaard To: "Frank Ch. Eigler" , Milian Wolff Cc: elfutils-devel@sourceware.org Date: Fri, 15 Apr 2022 15:11:55 +0200 In-Reply-To: <20220408213120.GE23295@redhat.com> References: <6158828.BCo7LnDCpQ@milian-workstation> <20220408205435.GD23295@redhat.com> <10231549.TdVgQxHH2d@milian-workstation> <20220408213120.GE23295@redhat.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Evolution 3.28.5 (3.28.5-10.el7) Mime-Version: 1.0 X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Apr 2022 13:12:00 -0000 Hi, On Fri, 2022-04-08 at 17:31 -0400, Frank Ch. Eigler via Elfutils-devel wrote: > Hi - >=20 > > But once again - isn't this a problem that everyone using dwfl is > > going to=20 > > encounter? Should we not try to find a general solution to this > > problem and=20 > > fix it for all consumers of the API? >=20 > I suspect not many apps are going to have a complete list of files > they know they'll need a priori. A live profiler tends to find out > about target files gradually. A debugger uses debuginfo to help > enumerate other debuginfo. DWZ files can only be even known of when > the base DWARF file is fetched. A "two stage" profiler like perf, which first just records build-ids, addresses and stack dumps, then afterwards does address/symbol/dwarf resolution and backtracing, can know before the second stage all the needed debuginfo for all the build-ids recorded. And a debugger can be "greedy" and try to make sure all debuginfo for a core file or live process is available up front. I believe gdb for example does try to generate symbol tables for everything on startup in a background thread so everything is available quickly once the user is really starting to debug. > So yeah I'm not sure there exists a > widespread general problem for which multithreading is not a workable > solution. That said, I am not sure multithreading is a real solution for the above scenarios. Doing parallel downloads is often not that efficient. It is better to reuse the same debuginfo_client handle to download things serially, that way you can reuse the existing connection instead of having to set it up for each download separately. Cheers, Mark