From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id AB7D33952487 for ; Fri, 13 Jan 2023 00:50:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AB7D33952487 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673571011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=P7TCek2yIMmCLrHGbDaF+FAnAvfb0u3coAh+nk7Z88c=; b=ba7hBTjgecL8/FoqA9GGjMZeJUIYESrzrsAO+HN9W0AyUEwYgyN81gjerjOWr0Lu99vUTm FgOuRkxOiWmbF1UXjqpI5IBqxNWZ2TU4/QDFLXrO+EINqjxKbv5eVvCx7/+pzCnb1JqE09 AhlNzXcvt6LbiOWL5ucem/qC4pyDVmo= Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-270-oWES3Y6WMWK97Q5gOWEqFA-1; Thu, 12 Jan 2023 19:50:10 -0500 X-MC-Unique: oWES3Y6WMWK97Q5gOWEqFA-1 Received: by mail-pf1-f197.google.com with SMTP id dc11-20020a056a0035cb00b00589a6a97519so6287209pfb.8 for ; Thu, 12 Jan 2023 16:50:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P7TCek2yIMmCLrHGbDaF+FAnAvfb0u3coAh+nk7Z88c=; b=eJ8C5eJWC2qkCS5j1c7pC4EGz+RzqSL70J21j+JSermvHAxX8wIo0yiUm9JbCQIQA7 pUdWOhfeHQ2JzgMXzrAaUOBIsR38yfA4Ndmt01UoTtknv16rAcF5f8ZABrfRt/tNda23 GiLcFHdOKfk5MgmgI1mYM1P6Is1XMlZFvJQJQprr50Y9C7j3Hj6YT+Ps6hMb8BewuFWO 7Yx86SELlM7f5DtF2Iy/UTL97M/M/Fad1gazUsayQzfzjg8WtmpygYD509ijLUxfuQ5V H7VsuU9Ffutbdm7t0gNmfm4IeG9//P9g1VlNQv7irNclToDu3+NiR7v+2KcYWa13t6n7 oqFA== X-Gm-Message-State: AFqh2kq37kUnvPnlqH2Fble2y+P/Fre6GqvoSn+PCQtnog6FMIxc0YjE AXLGV4uUsew1GuhcL9whPvnVGRg23NJHyqfI+EggrPZ7THlYuisAuPgMwtm80cPp2srzaKYMcOP +fy1Ce8IB1mVg0MF9FNy8Orue9hZSOAXO66HT X-Received: by 2002:a17:90b:3c06:b0:229:2826:d5ae with SMTP id pb6-20020a17090b3c0600b002292826d5aemr23359pjb.113.1673571008407; Thu, 12 Jan 2023 16:50:08 -0800 (PST) X-Google-Smtp-Source: AMrXdXvPeKN2k3bAWS4O6lKMe7navXnrQM2tLY0Gvdmv+eBq1PHChIypgs2HPaZ3/LTCWMhQLM0jENiiGGYooDAEBHg= X-Received: by 2002:a17:90b:3c06:b0:229:2826:d5ae with SMTP id pb6-20020a17090b3c0600b002292826d5aemr23353pjb.113.1673571007929; Thu, 12 Jan 2023 16:50:07 -0800 (PST) MIME-Version: 1.0 References: <20221006022424.399932-1-amerey@redhat.com> <87y1q869wz.fsf@tromey.com> In-Reply-To: <87y1q869wz.fsf@tromey.com> From: Aaron Merey Date: Thu, 12 Jan 2023 19:49:56 -0500 Message-ID: Subject: Re: [RFC][PATCH] gdb/debuginfod: Support on-demand downloading of debuginfo To: Tom Tromey Cc: Aaron Merey via Gdb-patches X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi Tom, On Wed, Jan 11, 2023 at 4:25 PM Tom Tromey wrote: > >>>>> "Aaron" == Aaron Merey via Gdb-patches writes: > > Aaron> This patch helps address the issue by adding support for on-demand > Aaron> downloading and reading of debuginfo. The basic approach it takes is > Aaron> to use debuginfod to download just the .gdb_index of a debuginfo file > Aaron> as soon as the corresponding library is linked. GDB then relies on the > Aaron> information in the index for as long as possible. When the index isn't > Aaron> enough then debuginfo is downloaded and read. This helps avoid unnecessary > Aaron> downloads. > > Makes sense to me. > > Aaron> Although this patch specifically uses .gdb_index, other indexes such > Aaron> as .debug_names could be supported in much the same way. > > I mentioned this in bugzilla, but .debug_names will be worse here > because it requires .debug_str and .debug_aranges to be downloaded as > well, and the former in particular will contain a lot of data that's not > immediately needed. In the case of libxul, the combined size of its .debug_names, .debug_str and .debug_aranges is about 72% larger than the size of its .gdb_index (690 MB vs. 400 MB). At least for this library we are much better off sticking with .gdb_index. > Aaron> This patch adds a command 'set debuginfod enabled lazy' which enables > Aaron> on-demand debuginfo downloading/reading. If this 'lazy' mode is enabled > Aaron> and a solib's debuginfo cannot be found locally, the new function > Aaron> dwarf2_has_separate_index is called in elf_symfile_read. This function > Aaron> queries debuginfod servers for the .gdb_index matching the build-id > Aaron> of the solib. If it's found, a new objfile is created to hold the .gdb_index > Aaron> information. The new objfile flag OBJF_INDEX_READLATER is used to indicate > Aaron> that the objfile contains quick_symbols_functions for an index has deferred > Aaron> debuginfo reading. > > It seems to me that OBJF_INDEX_READLATER should not be needed. > > Instead, what I'd propose is creating the separate debuginfo objfile and > providing it with an instance of dwarf2_gdb_index -- but one that > arranges to download the remaining data when necessary. > > It's fine to add new virtual methods to dwarf2_base_index_functions or > wherever if this helps. E.g, dw2_instantiate_symtab could be a virtual > method in the base class, and then the subclass could override it to > first download the remaining sections. Sure. > You'd have to consider how to handle errors in this situation, but > probably just printing a warning (once per file) and carrying on would > be good enough. The idea behind warn-and-carry-on is that the user > asked for this lazy behavior, if their network goes down then the kind > of asked for it; and anyway maybe they can rescue the situation with > "noshared" or something. > > Maybe you tried something like this already and ran into problems? I'm > handwaving a lot, maybe there are hidden difficulties. Currently if the index downloads successfully but the debuginfo download fails, the usual "Download failed" message is printed and it indicates that gdb will continue without that library's debuginfo. So far I haven't seen any issues in these cases but making use of "noshared" is a good idea. > > TBH the current gdb design where separate debug files are their own > objfile seems like a mistake to me. I haven't ever looked at changing > this though. But, maybe it doesn't have to really be changed, > dwarf2/read.c could just provide one of these subclasses attached to the > parent objfile. > > Aaron> Also with this patch, GDB will not build with the latest release of > Aaron> debuginfod, hence the RFC. The debuginfod ELF/DWARF section query > > Is it possible to detect the non-existence of debuginfod_find_section at > compile time so that we can avoid bumping the minimum requirement? Yes I will change this. debuginfod_find_section is supported as of elfutils 0.188 but it's still a good idea to avoid bumping the minimum requirement. Thanks, Aaron