From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id B98003857400 for ; Thu, 15 Jun 2023 13:45:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B98003857400 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686836708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V5YDV9kogv4Oi7FtCkhkOiFHtdKvjgGV+gYhrRozkT0=; b=FKgO0VjJvOSoXd6/wdf6nJCOdwFOrhqcTsDJozGgwoOH7v4/taUdkMBWuOqFfKRqMKZBdO avDUEMKj8wrFXgyJy83Tj5Ol7KSApkuCCr8Gpm7U3pJNoGv2XlQsFd9cxqiZB/lrUiGdg2 pPbsEf05KPN1MKsdDRE6V6AAQszXMnE= Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-651-1mUWQzVHNfOOnDQVGiMl6w-1; Thu, 15 Jun 2023 09:45:06 -0400 X-MC-Unique: 1mUWQzVHNfOOnDQVGiMl6w-1 Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-25e8a81fd14so785016a91.3 for ; Thu, 15 Jun 2023 06:45:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686836705; x=1689428705; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V5YDV9kogv4Oi7FtCkhkOiFHtdKvjgGV+gYhrRozkT0=; b=e8x5L3diX2FBtGLZrA+oicOkcwYm56FEFNLYT8GsZMHh2WruOGuNfaoocmnscx2wjg 5K6+9YAv47vWidIokEDdsjZk7F25lLNF7UTm2XgjUYqLMP2I7SMrcxHC0wjqoN8QMQNQ Nkzj+8QW6G/uJAYB/yL9nYtgwKrCTrUpdl2NWxsKMUR58zGYCBH9gpc7a2LVON2gXDn0 G5lEtwWsbzFRMArIE1ihDKBKU81JqFc18YcKAXFt1okfNYi0kiDcuABOD1BBIyHVWSzJ fyMPHhUKJx2+vMGCbQGU4sG5+vUfiI7yDaugoSq2XNKLWzn464PfmpkvM2wQcckQHgsH 8PMg== X-Gm-Message-State: AC+VfDxA4B50ZUv/dK0iCkplznqvgKyTW7qjyfRIfb61JAHV8oDne3DW jhGBDPSXxE7HJc9LjmFp6In6X8AqzRBuXl7tF3o2wVIR9DcF43VR0OZOTq58snfOaQTKwVeZLwM FT/ShTK7SLPSJXQS1RrkyX85hhwYWjD0/L9JwEfC4Uhbs X-Received: by 2002:a17:90a:6b03:b0:25e:8f12:a74d with SMTP id v3-20020a17090a6b0300b0025e8f12a74dmr1988599pjj.44.1686836704477; Thu, 15 Jun 2023 06:45:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6dXuYdKXKGtaftetWumgixuyPuWmzujbAmPa9m6k9/R+hAYdrE02kkDqikVKAahpGU5URNzq1IPiV7G/bBePk= X-Received: by 2002:a17:90a:6b03:b0:25e:8f12:a74d with SMTP id v3-20020a17090a6b0300b0025e8f12a74dmr1988565pjj.44.1686836703659; Thu, 15 Jun 2023 06:45:03 -0700 (PDT) MIME-Version: 1.0 References: <20230601014347.3367489-1-amerey@redhat.com> <20230601014347.3367489-6-amerey@redhat.com> In-Reply-To: <20230601014347.3367489-6-amerey@redhat.com> From: Aaron Merey Date: Thu, 15 Jun 2023 09:44:52 -0400 Message-ID: Subject: Re: [PATCH 5/6 v3] gdb/debuginfod: Support on-demand debuginfo downloading To: gdb-patches@sourceware.org Cc: aburgess@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,KAM_SHORT,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Ping Thanks, Aaron On Wed, May 31, 2023 at 9:44=E2=80=AFPM Aaron Merey wro= te: > > v2.2: https://sourceware.org/pipermail/gdb-patches/2023-May/199326.html > > v3 includes testcases instead of adding them in a separate patch. > Tests related to section downloading are now included in > testsuite/gdb.debuginfod/section.exp. > > Commit message: > > At the beginning of a session, gdb may attempt to download debuginfo > for all shared libraries associated with the process or core file > being debugged. This can be a waste of time and storage space when much > of the debuginfo ends up not being used during the session. > > To reduce the gdb's startup latency and to download only the debuginfo > that is really needed, this patch adds on-demand downloading of debuginfo= . > > 'set debuginfo enabled on' now causes gdb to attempt to download a .gdb_i= ndex > for each shared library instead of its full debuginfo. Each correspondin= g > separate debuginfo will be deferred until gdb needs to expand symtabs > associated with the debuginfo's index. > > Because these indices are significantly smaller than their corresponding > debuginfo, this generally reduces the total amount of data gdb downloads. > Reductions of 80%-95% have been observed when debugging large GUI program= s. > > (gdb) set debuginfod enabled on > (gdb) start > Downloading section .gdb_index for /lib64/libcurl.so.4 > [...] > 1826 client->server_mhandle =3D curl_multi_init (); > (gdb) step > Downloading separate debug info for /lib64/libcurl.so.4 > Downloading separate debug info for [libcurl dwz] > Downloading source file /usr/src/debug/curl-7.85.0-6.fc37.x86_64/buil= d-full/lib/../../lib/multi.c > curl_multi_init () at ../../lib/multi.c:457 > 457 { > (gdb) > > Some of the key functions below include dwarf2_has_separate_index which > downloads the separate .gdb_index. If successful, the shared library > objfile owns the index until the separate debug objfile is downloaded > or confirmed to not be available. > > read_full_dwarf_from_debuginfod downloads the full debuginfo and > initializes the separate debug objfile. It is called by functions > such as dwarf2_gdb_index::expand_symtabs_matching and > dwarf2_base_index_functions::find_pc_sect_compunit_symtab when symtab > expansion is required. > --- > gdb/dwarf2/frame.c | 13 ++ > gdb/dwarf2/frame.h | 4 + > gdb/dwarf2/index-cache.c | 33 ++++ > gdb/dwarf2/index-cache.h | 13 ++ > gdb/dwarf2/public.h | 7 + > gdb/dwarf2/read-gdb-index.c | 156 ++++++++++++++++-- > gdb/dwarf2/read.c | 146 ++++++++++++++++- > gdb/dwarf2/read.h | 10 ++ > gdb/dwarf2/section.c | 3 +- > gdb/elfread.c | 2 +- > gdb/objfile-flags.h | 4 + > gdb/objfiles.h | 17 +- > gdb/quick-symbol.h | 4 + > gdb/symfile.c | 13 +- > gdb/symtab.c | 18 +- > gdb/testsuite/gdb.debuginfod/libsection1.c | 40 +++++ > gdb/testsuite/gdb.debuginfod/libsection2.c | 37 +++++ > gdb/testsuite/gdb.debuginfod/section.c | 29 ++++ > gdb/testsuite/gdb.debuginfod/section.exp | 181 +++++++++++++++++++++ > gdb/testsuite/lib/debuginfod-support.exp | 30 ++++ > 20 files changed, 734 insertions(+), 26 deletions(-) > create mode 100644 gdb/testsuite/gdb.debuginfod/libsection1.c > create mode 100644 gdb/testsuite/gdb.debuginfod/libsection2.c > create mode 100644 gdb/testsuite/gdb.debuginfod/section.c > create mode 100644 gdb/testsuite/gdb.debuginfod/section.exp > > diff --git a/gdb/dwarf2/frame.c b/gdb/dwarf2/frame.c > index a561aaf3100..3613f8252a7 100644 > --- a/gdb/dwarf2/frame.c > +++ b/gdb/dwarf2/frame.c > @@ -1609,6 +1609,19 @@ set_comp_unit (struct objfile *objfile, struct com= p_unit *unit) > return dwarf2_frame_bfd_data.set (abfd, unit); > } > > +/* See frame.h. */ > + > +void > +dwarf2_clear_frame_data (struct objfile *objfile) > +{ > + bfd *abfd =3D objfile->obfd.get (); > + > + if (gdb_bfd_requires_relocations (abfd)) > + dwarf2_frame_objfile_data.clear (objfile); > + else > + dwarf2_frame_bfd_data.clear (abfd); > +} > + > /* Find the FDE for *PC. Return a pointer to the FDE, and store the > initial location associated with it into *PC. */ > > diff --git a/gdb/dwarf2/frame.h b/gdb/dwarf2/frame.h > index 5643e557513..2391e313e7c 100644 > --- a/gdb/dwarf2/frame.h > +++ b/gdb/dwarf2/frame.h > @@ -238,6 +238,10 @@ void dwarf2_append_unwinders (struct gdbarch *gdbarc= h); > extern const struct frame_base * > dwarf2_frame_base_sniffer (frame_info_ptr this_frame); > > +/* Delete OBJFILEs comp_unit. */ > + > +extern void dwarf2_clear_frame_data (struct objfile * objfile); > + > /* Compute the DWARF CFA for a frame. */ > > CORE_ADDR dwarf2_frame_cfa (frame_info_ptr this_frame); > diff --git a/gdb/dwarf2/index-cache.c b/gdb/dwarf2/index-cache.c > index 79ab706ee9d..bbafcd321b2 100644 > --- a/gdb/dwarf2/index-cache.c > +++ b/gdb/dwarf2/index-cache.c > @@ -216,6 +216,33 @@ index_cache::lookup_gdb_index (const bfd_build_id *b= uild_id, > return {}; > } > > +/* See index-cache.h. */ > + > +gdb::array_view > +index_cache::lookup_gdb_index_debuginfod (const char *index_path, > + std::unique_ptr *resource) > +{ > + try > + { > + /* Try to map that file. */ > + index_cache_resource_mmap *mmap_resource > + =3D new index_cache_resource_mmap (index_path); > + > + /* Hand the resource to the caller. */ > + resource->reset (mmap_resource); > + > + return gdb::array_view > + ((const gdb_byte *) mmap_resource->mapping.get (), > + mmap_resource->mapping.size ()); > + } > + catch (const gdb_exception_error &except) > + { > + warning (_("Unable to read %s: %s"), index_path, except.what ()); > + } > + > + return {}; > +} > + > #else /* !HAVE_SYS_MMAN_H */ > > /* See dwarf-index-cache.h. This is a no-op on unsupported systems. */ > @@ -227,6 +254,12 @@ index_cache::lookup_gdb_index (const bfd_build_id *b= uild_id, > return {}; > } > > +gdb::array_view > +index_cache::lookup_gdb_index_debuginfod (const char *index_path, > + std::unique_ptr *resource) > +{ > + return {}; > +} > #endif > > /* See dwarf-index-cache.h. */ > diff --git a/gdb/dwarf2/index-cache.h b/gdb/dwarf2/index-cache.h > index 1efff17049f..e400afd5123 100644 > --- a/gdb/dwarf2/index-cache.h > +++ b/gdb/dwarf2/index-cache.h > @@ -67,6 +67,19 @@ class index_cache > lookup_gdb_index (const bfd_build_id *build_id, > std::unique_ptr *resource); > > + /* Look for an index file located at INDEX_PATH in the debuginfod cach= e. > + Unlike lookup_gdb_index, this function does not exit early if the > + index cache has not been enabled. > + > + If found, return the contents as an array_view and store the underl= ying > + resources (allocated memory, mapped file, etc) in RESOURCE. The re= turned > + array_view is valid as long as RESOURCE is not destroyed. > + > + If no matching index file is found, return an empty array view. */ > + gdb::array_view > + lookup_gdb_index_debuginfod (const char *index_path, > + std::unique_ptr *res= ource); > + > /* Return the number of cache hits. */ > unsigned int n_hits () const > { return m_n_hits; } > diff --git a/gdb/dwarf2/public.h b/gdb/dwarf2/public.h > index 0e74857eb1a..4a44cdbc223 100644 > --- a/gdb/dwarf2/public.h > +++ b/gdb/dwarf2/public.h > @@ -40,4 +40,11 @@ extern void dwarf2_initialize_objfile (struct objfile = *objfile); > > extern void dwarf2_build_frame_info (struct objfile *); > > +/* Query debuginfod for the .gdb_index associated with OBJFILE. If > + successful, create an objfile to hold the .gdb_index information > + and act as a placeholder until the full debuginfo needs to be > + downloaded. */ > + > +extern bool dwarf2_has_separate_index (struct objfile *); > + > #endif /* DWARF2_PUBLIC_H */ > diff --git a/gdb/dwarf2/read-gdb-index.c b/gdb/dwarf2/read-gdb-index.c > index 1006386cb2d..d3516e92361 100644 > --- a/gdb/dwarf2/read-gdb-index.c > +++ b/gdb/dwarf2/read-gdb-index.c > @@ -136,6 +136,7 @@ struct dwarf2_gdb_index : public dwarf2_base_index_fu= nctions > gdb.dwarf2/gdb-index.exp testcase. */ > void dump (struct objfile *objfile) override; > > + /* Calls do_expand_matching_symbols and downloads debuginfo if necessa= ry. */ > void expand_matching_symbols > (struct objfile *, > const lookup_name_info &lookup_name, > @@ -143,6 +144,14 @@ struct dwarf2_gdb_index : public dwarf2_base_index_f= unctions > int global, > symbol_compare_ftype *ordered_compare) override; > > + void do_expand_matching_symbols > + (struct objfile *, > + const lookup_name_info &lookup_name, > + domain_enum domain, > + int global, > + symbol_compare_ftype *ordered_compare); > + > + /* Calls do_expand_symtabs_matching and downloads debuginfo if necessa= ry. */ > bool expand_symtabs_matching > (struct objfile *objfile, > gdb::function_view file_matcher, > @@ -152,8 +161,59 @@ struct dwarf2_gdb_index : public dwarf2_base_index_f= unctions > block_search_flags search_flags, > domain_enum domain, > enum search_domain kind) override; > + > + bool do_expand_symtabs_matching > + (struct objfile *objfile, > + gdb::function_view file_matcher, > + const lookup_name_info *lookup_name, > + gdb::function_view symbol_matc= her, > + gdb::function_view expansion_notif= y, > + block_search_flags search_flags, > + domain_enum domain, > + enum search_domain kind); > + > + /* Calls dwarf2_base_index_functions::expand_all_symtabs and downloads > + debuginfo if necessary. */ > + void expand_all_symtabs (struct objfile *objfile) override; > + > + /* Calls dwarf2_base_index_functions::find_last_source_symtab and down= loads > + debuginfo if necessary. */ > + struct symtab *find_last_source_symtab (struct objfile *objfile) overr= ide; > }; > > +void > +dwarf2_gdb_index::expand_all_symtabs (struct objfile *objfile) > +{ > + try > + { > + dwarf2_base_index_functions::expand_all_symtabs (objfile); > + } > + catch (gdb_exception e) > + { > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + exception_print (gdb_stderr, e); > + else > + read_full_dwarf_from_debuginfod (objfile, this); > + } > +} > + > +struct symtab * > +dwarf2_gdb_index::find_last_source_symtab (struct objfile *objfile) > +{ > + try > + { > + return dwarf2_base_index_functions::find_last_source_symtab (objfi= le); > + } > + catch (gdb_exception e) > + { > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + exception_print (gdb_stderr, e); > + else > + read_full_dwarf_from_debuginfod (objfile, this); > + return nullptr; > + } > +} > + > /* This dumps minimal information about the index. > It is called via "mt print objfiles". > One use is to verify .gdb_index has been loaded by the > @@ -315,7 +375,7 @@ dw2_symtab_iter_next (struct dw2_symtab_iterator *ite= r, > } > > void > -dwarf2_gdb_index::expand_matching_symbols > +dwarf2_gdb_index::do_expand_matching_symbols > (struct objfile *objfile, > const lookup_name_info &name, domain_enum domain, > int global, > @@ -353,6 +413,29 @@ dwarf2_gdb_index::expand_matching_symbols > }, per_objfile); > } > > +void > +dwarf2_gdb_index::expand_matching_symbols > + (struct objfile *objfile, > + const lookup_name_info &lookup_name, > + domain_enum domain, > + int global, > + symbol_compare_ftype *ordered_compare) > +{ > + try > + { > + do_expand_matching_symbols (objfile, lookup_name, domain, > + global, ordered_compare); > + } > + catch (gdb_exception e) > + { > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + exception_print (gdb_stderr, e); > + else > + read_full_dwarf_from_debuginfod (objfile, this); > + return; > + } > +} > + > /* Helper for dw2_expand_matching symtabs. Called on each symbol > matched, to expand corresponding CUs that were marked. IDX is the > index of the symbol name that matched. */ > @@ -455,7 +538,7 @@ dw2_expand_marked_cus > } > > bool > -dwarf2_gdb_index::expand_symtabs_matching > +dwarf2_gdb_index::do_expand_symtabs_matching > (struct objfile *objfile, > gdb::function_view file_matcher, > const lookup_name_info *lookup_name, > @@ -504,6 +587,39 @@ dwarf2_gdb_index::expand_symtabs_matching > return result; > } > > +bool > +dwarf2_gdb_index::expand_symtabs_matching > + (struct objfile *objfile, > + gdb::function_view file_matcher, > + const lookup_name_info *lookup_name, > + gdb::function_view symbol_matc= her, > + gdb::function_view expansion_notif= y, > + block_search_flags search_flags, > + domain_enum domain, > + enum search_domain kind) > +{ > + if (objfile->flags & OBJF_READNEVER) > + return false; > + > + try > + { > + return do_expand_symtabs_matching (objfile, file_matcher, lookup_n= ame, > + symbol_matcher, expansion_notify= , > + search_flags, domain, kind); > + } > + catch (gdb_exception e) > + { > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + { > + exception_print (gdb_stderr, e); > + return false; > + } > + > + read_full_dwarf_from_debuginfod (objfile, this); > + return true; > + } > +} > + > quick_symbol_functions_up > mapped_gdb_index::make_quick_functions () const > { > @@ -797,28 +913,32 @@ dwarf2_read_gdb_index > > /* If there is a .dwz file, read it so we can get its CU list as > well. */ > - dwz =3D dwarf2_get_dwz_file (per_bfd); > - if (dwz !=3D NULL) > + if (get_gdb_index_contents_dwz !=3D nullptr) > { > mapped_gdb_index dwz_map; > const gdb_byte *dwz_types_ignore; > offset_type dwz_types_elements_ignore; > + dwz =3D dwarf2_get_dwz_file (per_bfd); > > - gdb::array_view dwz_index_content > - =3D get_gdb_index_contents_dwz (objfile, dwz); > - > - if (dwz_index_content.empty ()) > - return 0; > - > - if (!read_gdb_index_from_buffer (bfd_get_filename (dwz->dwz_bfd.ge= t ()), > - 1, dwz_index_content, &dwz_map, > - &dwz_list, &dwz_list_elements, > - &dwz_types_ignore, > - &dwz_types_elements_ignore)) > + if (dwz !=3D nullptr) > { > - warning (_("could not read '.gdb_index' section from %s; skippi= ng"), > - bfd_get_filename (dwz->dwz_bfd.get ())); > - return 0; > + gdb::array_view dwz_index_content > + =3D get_gdb_index_contents_dwz (objfile, dwz); > + > + if (dwz_index_content.empty ()) > + return 0; > + > + if (!read_gdb_index_from_buffer (bfd_get_filename > + (dwz->dwz_bfd.get ()), > + 1, dwz_index_content, &dwz_map= , > + &dwz_list, &dwz_list_elements, > + &dwz_types_ignore, > + &dwz_types_elements_ignore)) > + { > + warning (_("could not read '.gdb_index' section from %s; sk= ipping"), > + bfd_get_filename (dwz->dwz_bfd.get ())); > + return 0; > + } > } > } > > diff --git a/gdb/dwarf2/read.c b/gdb/dwarf2/read.c > index 4828409222c..96d1ff53d91 100644 > --- a/gdb/dwarf2/read.c > +++ b/gdb/dwarf2/read.c > @@ -34,6 +34,7 @@ > #include "dwarf2/attribute.h" > #include "dwarf2/comp-unit-head.h" > #include "dwarf2/cu.h" > +#include "dwarf2/frame.h" > #include "dwarf2/index-cache.h" > #include "dwarf2/index-common.h" > #include "dwarf2/leb.h" > @@ -95,6 +96,8 @@ > #include "split-name.h" > #include "gdbsupport/parallel-for.h" > #include "gdbsupport/thread-pool.h" > +#include "inferior.h" > +#include "debuginfod-support.h" > > /* When =3D=3D 1, print basic high level tracing messages. > When > 1, be more verbose. > @@ -3168,7 +3171,7 @@ dwarf2_base_index_functions::find_per_cu (dwarf2_pe= r_bfd *per_bfd, > } > > struct compunit_symtab * > -dwarf2_base_index_functions::find_pc_sect_compunit_symtab > +dwarf2_base_index_functions::do_find_pc_sect_compunit_symtab > (struct objfile *objfile, > struct bound_minimal_symbol msymbol, > CORE_ADDR pc, > @@ -3199,6 +3202,32 @@ dwarf2_base_index_functions::find_pc_sect_compunit= _symtab > return result; > } > > +struct compunit_symtab * > +dwarf2_base_index_functions::find_pc_sect_compunit_symtab > + (struct objfile *objfile, > + struct bound_minimal_symbol msymbol, > + CORE_ADDR pc, > + struct obj_section *section, > + int warn_if_readin) > +{ > + if (objfile->flags & OBJF_READNEVER) > + return nullptr; > + > + try > + { > + return do_find_pc_sect_compunit_symtab (objfile, msymbol, pc, > + section, warn_if_readin); > + } > + catch (gdb_exception e) > + { > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + exception_print (gdb_stderr, e); > + else > + read_full_dwarf_from_debuginfod (objfile, this); > + return nullptr; > + } > +} > + > void > dwarf2_base_index_functions::map_symbol_filenames > (struct objfile *objfile, > @@ -3355,6 +3384,29 @@ get_gdb_index_contents_from_cache_dwz (objfile *ob= j, dwz_file *dwz) > return global_index_cache.lookup_gdb_index (build_id, &dwz->index_cach= e_res); > } > > +/* Query debuginfod for the .gdb_index matching OBJFILE's build-id. Ret= urn the > + contents if successful. */ > + > +static gdb::array_view > +get_gdb_index_contents_from_debuginfod (objfile *objfile, dwarf2_per_bfd= *per_bfd) > +{ > + const bfd_build_id *build_id =3D build_id_bfd_get (objfile->obfd.get (= )); > + if (build_id =3D=3D nullptr) > + return {}; > + > + gdb::unique_xmalloc_ptr index_path; > + scoped_fd fd =3D debuginfod_section_query (build_id->data, build_id->s= ize, > + bfd_get_filename > + (objfile->obfd.get ()), > + ".gdb_index", > + &index_path); > + if (fd.get () < 0) > + return {}; > + > + return global_index_cache.lookup_gdb_index_debuginfod > + (index_path.get (), &per_bfd->index_cache_res); > +} > + > static quick_symbol_functions_up make_cooked_index_funcs (); > > /* See dwarf2/public.h. */ > @@ -3420,10 +3472,102 @@ dwarf2_initialize_objfile (struct objfile *objfi= le) > return; > } > > + if ((objfile->flags & OBJF_DOWNLOAD_DEFERRED) > + && dwarf2_read_gdb_index (per_objfile, > + get_gdb_index_contents_from_debuginfod, > + nullptr)) > + { > + dwarf_read_debug_printf ("found .gdb_index from debuginfod"); > + objfile->qf.push_front (per_bfd->index_table->make_quick_functions= ()); > + objfile->qf.begin ()->get ()->from_separate_index =3D true; > + return; > + } > + > global_index_cache.miss (); > objfile->qf.push_front (make_cooked_index_funcs ()); > } > > +/* See read.h. */ > + > +void > +read_full_dwarf_from_debuginfod (struct objfile *objfile, > + dwarf2_base_index_functions *fncs) > +{ > + gdb_assert (objfile->flags & OBJF_DOWNLOAD_DEFERRED); > + > + const struct bfd_build_id *build_id =3D build_id_bfd_get (objfile->obf= d.get ()); > + const char *filename; > + gdb_bfd_ref_ptr debug_bfd; > + gdb::unique_xmalloc_ptr symfile_path; > + scoped_fd fd; > + > + if (build_id =3D=3D nullptr) > + goto unset; > + > + filename =3D bfd_get_filename (objfile->obfd.get ()); > + fd =3D debuginfod_debuginfo_query (build_id->data, build_id->size, > + filename, &symfile_path); > + if (fd.get () < 0) > + goto unset; > + > + /* Separate debuginfo successfully retrieved from server. */ > + debug_bfd =3D symfile_bfd_open (symfile_path.get ()); > + if (debug_bfd =3D=3D nullptr > + || !build_id_verify (debug_bfd.get (), build_id->size, build_id->d= ata)) > + { > + warning (_("File \"%s\" from debuginfod cannot be opened as bfd"), > + filename); > + goto unset; > + } > + > + /* Clear frame data so it can be recalculated using DWARF. */ > + dwarf2_clear_frame_data (objfile); > + > + /* This may also trigger a dwz download. */ > + symbol_file_add_separate (debug_bfd, symfile_path.get (), > + current_inferior ()->symfile_flags, objfile); > + > +unset: > + objfile->remove_deferred_status (); > +} > + > +/* See public.h. */ > + > +bool > +dwarf2_has_separate_index (struct objfile *objfile) > +{ > + if (objfile->flags & OBJF_DOWNLOAD_DEFERRED) > + return true; > + if (objfile->flags & OBJF_MAINLINE) > + return false; > + if (!IS_DIR_SEPARATOR (*objfile_filename (objfile))) > + return false; > + > + gdb::unique_xmalloc_ptr index_path; > + const bfd_build_id *build_id =3D build_id_bfd_get (objfile->obfd.get (= )); > + > + if (build_id =3D=3D nullptr) > + return false; > + > + scoped_fd fd =3D debuginfod_section_query (build_id->data, > + build_id->size, > + bfd_get_filename > + (objfile->obfd.get ()), > + ".gdb_index", > + &index_path); > + > + if (fd.get () < 0) > + return false; > + > + /* We found a separate .gdb_index file so a separate debuginfo file > + should exist, but we don't want to download it until necessary. > + Attach the index to this objfile and defer the debuginfo download > + until gdb needs to expand symtabs referenced by the index. */ > + objfile->flags |=3D OBJF_DOWNLOAD_DEFERRED; > + dwarf2_initialize_objfile (objfile); > + return true; > +} > + > > > /* Build a partial symbol table. */ > diff --git a/gdb/dwarf2/read.h b/gdb/dwarf2/read.h > index 37023a20709..e3131693b81 100644 > --- a/gdb/dwarf2/read.h > +++ b/gdb/dwarf2/read.h > @@ -866,6 +866,10 @@ struct dwarf2_base_index_functions : public quick_sy= mbol_functions > CORE_ADDR pc, struct obj_section *section, int warn_if_readin) > override final; > > + struct compunit_symtab *do_find_pc_sect_compunit_symtab > + (struct objfile *objfile, struct bound_minimal_symbol msymbol, > + CORE_ADDR pc, struct obj_section *section, int warn_if_readin); > + > struct compunit_symtab *find_compunit_symtab_by_address > (struct objfile *objfile, CORE_ADDR address) override > { > @@ -942,4 +946,10 @@ extern bool read_addrmap_from_aranges (dwarf2_per_ob= jfile *per_objfile, > dwarf2_section_info *section, > addrmap *mutable_map); > > +/* If OBJFILE contains information from a separately downloaded .gdb_ind= ex, > + attempt to download the full debuginfo. */ > + > +extern void read_full_dwarf_from_debuginfod (struct objfile *, > + dwarf2_base_index_functions = *); > + > #endif /* DWARF2READ_H */ > diff --git a/gdb/dwarf2/section.c b/gdb/dwarf2/section.c > index c9ef41893ee..8cb09e3381a 100644 > --- a/gdb/dwarf2/section.c > +++ b/gdb/dwarf2/section.c > @@ -54,7 +54,8 @@ dwarf2_section_info::get_bfd_owner () const > section =3D get_containing_section (); > gdb_assert (!section->is_virtual); > } > - gdb_assert (section->s.section !=3D nullptr); > + if (section->s.section =3D=3D nullptr) > + error (_("Can't find owner of DWARF section.")); > return section->s.section->owner; > } > > diff --git a/gdb/elfread.c b/gdb/elfread.c > index 799e3b914f8..133341ea615 100644 > --- a/gdb/elfread.c > +++ b/gdb/elfread.c > @@ -1242,7 +1242,7 @@ elf_symfile_read_dwarf2 (struct objfile *objfile, > symbol_file_add_separate (debug_bfd, debugfile.c_str (), > symfile_flags, objfile); > } > - else > + else if (!dwarf2_has_separate_index (objfile)) > { > has_dwarf2 =3D false; > const struct bfd_build_id *build_id > diff --git a/gdb/objfile-flags.h b/gdb/objfile-flags.h > index 9dee2ee51a0..fb3f741c899 100644 > --- a/gdb/objfile-flags.h > +++ b/gdb/objfile-flags.h > @@ -60,6 +60,10 @@ enum objfile_flag : unsigned > /* User requested that we do not read this objfile's symbolic > information. */ > OBJF_READNEVER =3D 1 << 6, > + > + /* A separate .gdb_index has been downloaded for this objfile. > + Debuginfo for this objfile can be downloaded when required. */ > + OBJF_DOWNLOAD_DEFERRED =3D 1 << 7, > }; > > DEF_ENUM_FLAGS_TYPE (enum objfile_flag, objfile_flags); > diff --git a/gdb/objfiles.h b/gdb/objfiles.h > index bb7b0a4579d..57bc1d45965 100644 > --- a/gdb/objfiles.h > +++ b/gdb/objfiles.h > @@ -613,11 +613,22 @@ struct objfile > /* See quick_symbol_functions. */ > void require_partial_symbols (bool verbose); > > - /* Remove TARGET from this objfile's collection of quick_symbol_functi= ons. */ > - void remove_partial_symbol (quick_symbol_functions *target) > + /* Indicate that the aquisition of this objfile's separate debug objf= ile > + is no longer deferred. Used when the debug objfile has been aquir= ed > + or could not be found. */ > + void remove_deferred_status () > { > + flags &=3D ~OBJF_DOWNLOAD_DEFERRED; > + > + /* Remove quick_symbol_functions derived from a separately downloaded > + index. If available the separate debug objfile's index will be us= ed > + instead, since that objfile actually contains the symbols and CUs > + referenced in the index. > + > + No more than one element of qf should have from_separate_index set > + to true. */ > for (quick_symbol_functions_up &qf_up : qf) > - if (qf_up.get () =3D=3D target) > + if (qf_up->from_separate_index) > { > qf.remove (qf_up); > return; > diff --git a/gdb/quick-symbol.h b/gdb/quick-symbol.h > index a7fea2ccb49..e7163503e39 100644 > --- a/gdb/quick-symbol.h > +++ b/gdb/quick-symbol.h > @@ -225,6 +225,10 @@ struct quick_symbol_functions > virtual void read_partial_symbols (struct objfile *objfile) > { > } > + > + /* True if this quick_symbol_functions is derived from a separately > + downloaded index. */ > + bool from_separate_index =3D false; > }; > > typedef std::unique_ptr quick_symbol_functions_u= p; > diff --git a/gdb/symfile.c b/gdb/symfile.c > index 96239679c77..c476196184a 100644 > --- a/gdb/symfile.c > +++ b/gdb/symfile.c > @@ -991,6 +991,10 @@ syms_from_objfile (struct objfile *objfile, > static void > finish_new_objfile (struct objfile *objfile, symfile_add_flags add_flags= ) > { > + struct objfile *parent =3D objfile->separate_debug_objfile_backlink; > + bool was_deferred > + =3D (parent !=3D nullptr) && (parent->flags & OBJF_DOWNLOAD_DEFERRED= ); > + > /* If this is the main symbol file we have to clean up all users of th= e > old main symbol file. Otherwise it is sufficient to fixup all the > breakpoints that may have been redefined by this symbol file. */ > @@ -1001,7 +1005,8 @@ finish_new_objfile (struct objfile *objfile, symfil= e_add_flags add_flags) > > clear_symtab_users (add_flags); > } > - else if ((add_flags & SYMFILE_DEFER_BP_RESET) =3D=3D 0) > + else if ((add_flags & SYMFILE_DEFER_BP_RESET) =3D=3D 0 > + && !was_deferred) > { > breakpoint_re_set (); > } > @@ -1127,6 +1132,12 @@ symbol_file_add_with_addrs (const gdb_bfd_ref_ptr = &abfd, const char *name, > > finish_new_objfile (objfile, add_flags); > > + /* Remove deferred status now in case any observers trigger symtab > + expansion. Otherwise gdb might try to read parent for psymbols > + when it should read the separate debug objfile instead. */ > + if (parent !=3D nullptr && (parent->flags & OBJF_DOWNLOAD_DEFERRED)) > + parent->remove_deferred_status (); > + > gdb::observers::new_objfile.notify (objfile); > > bfd_cache_close_all (); > diff --git a/gdb/symtab.c b/gdb/symtab.c > index 5e1b9d91879..2408725fa73 100644 > --- a/gdb/symtab.c > +++ b/gdb/symtab.c > @@ -2897,14 +2897,30 @@ find_pc_sect_compunit_symtab (CORE_ADDR pc, struc= t obj_section *section) > if (best_cust !=3D NULL) > return best_cust; > > + int warn_if_readin =3D 1; > + > /* Not found in symtabs, search the "quick" symtabs (e.g. psymtabs). = */ > > for (objfile *objf : current_program_space->objfiles ()) > { > + bool was_deferred =3D objf->flags & OBJF_DOWNLOAD_DEFERRED; > + > struct compunit_symtab *result > - =3D objf->find_pc_sect_compunit_symtab (msymbol, pc, section, 1); > + =3D objf->find_pc_sect_compunit_symtab (msymbol, pc, section, > + warn_if_readin); > + > if (result !=3D NULL) > return result; > + > + /* If objf's separate debug info was just acquired, disable > + warn_if_readin for the next iteration of this loop. This preven= ts > + a spurious warning in case an observer already triggered expansi= on > + of the separate debug objfile's symtabs. */ > + if (was_deferred && objf->separate_debug_objfile !=3D nullptr > + && (objf->flags & OBJF_DOWNLOAD_DEFERRED) =3D=3D 0) > + warn_if_readin =3D 0; > + else if (warn_if_readin =3D=3D 0) > + warn_if_readin =3D 1; > } > > return NULL; > diff --git a/gdb/testsuite/gdb.debuginfod/libsection1.c b/gdb/testsuite/g= db.debuginfod/libsection1.c > new file mode 100644 > index 00000000000..60824b415c6 > --- /dev/null > +++ b/gdb/testsuite/gdb.debuginfod/libsection1.c > @@ -0,0 +1,40 @@ > +/* This testcase is part of GDB, the GNU debugger. > + > + Copyright 2023 Free Software Foundation, Inc. > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; either version 3 of the License, or > + (at your option) any later version. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program. If not, see .= */ > + > +#include > +#include > +#include > + > +extern void libsection2_test (); > +extern void *libsection2_thread_test (void *); > + > +void > +libsection1_test () > +{ > + pthread_t thr; > + > + printf ("In libsection1\n"); > + libsection2_test (); > + > + pthread_create (&thr, NULL, libsection2_thread_test, NULL); > + > + /* Give the new thread a chance to actually enter libsection2_thread_t= est. */ > + sleep (3); > + printf ("Cancelling thread\n"); > + > + pthread_cancel (thr); > +} > diff --git a/gdb/testsuite/gdb.debuginfod/libsection2.c b/gdb/testsuite/g= db.debuginfod/libsection2.c > new file mode 100644 > index 00000000000..629a67f94a5 > --- /dev/null > +++ b/gdb/testsuite/gdb.debuginfod/libsection2.c > @@ -0,0 +1,37 @@ > +/* This testcase is part of GDB, the GNU debugger. > + > + Copyright 2023 Free Software Foundation, Inc. > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; either version 3 of the License, or > + (at your option) any later version. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program. If not, see .= */ > + > +#include > + > +void > +libsection2_test () > +{ > + printf ("In libsection2\n"); > +} > + > +void * > +libsection2_thread_test (void *arg) > +{ > + (void) arg; > + > + printf ("In thread test\n"); > + > + while (1) > + ; > + > + return NULL; > +} > diff --git a/gdb/testsuite/gdb.debuginfod/section.c b/gdb/testsuite/gdb.d= ebuginfod/section.c > new file mode 100644 > index 00000000000..d391a8f898e > --- /dev/null > +++ b/gdb/testsuite/gdb.debuginfod/section.c > @@ -0,0 +1,29 @@ > +/* This testcase is part of GDB, the GNU debugger. > + > + Copyright 2023 Free Software Foundation, Inc. > + > + This program is free software; you can redistribute it and/or modify > + it under the terms of the GNU General Public License as published by > + the Free Software Foundation; either version 3 of the License, or > + (at your option) any later version. > + > + This program is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + GNU General Public License for more details. > + > + You should have received a copy of the GNU General Public License > + along with this program. If not, see .= */ > + > +#include > + > +extern void libsection1_test (); > + > +int > +main() > +{ > + libsection1_test (); > + printf ("in section exec\n"); > + > + return 0; > +} > diff --git a/gdb/testsuite/gdb.debuginfod/section.exp b/gdb/testsuite/gdb= .debuginfod/section.exp > new file mode 100644 > index 00000000000..96e9750cd38 > --- /dev/null > +++ b/gdb/testsuite/gdb.debuginfod/section.exp > @@ -0,0 +1,181 @@ > +# Copyright 2023 Free Software Foundation, Inc. > + > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 3 of the License, or > +# (at your option) any later version. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see . > + > +# Test debuginfod functionality > + > +standard_testfile > + > +load_lib debuginfod-support.exp > + > +require allow_debuginfod_tests > + > +set sourcetmp [standard_output_file tmp-${srcfile}] > +set outputdir [standard_output_file {}] > + > +# SECTEXEC is an executable which calls a function from LIB_SL1. > +set sectfile "section" > +set sectsrc $srcdir/$subdir/section.c > +set sectexec [standard_output_file $sectfile] > + > +# Solib LIB_SL1 calls functions from LIB_SL2. > +set libfile1 "libsection1" > +set libsrc1 $srcdir/$subdir/$libfile1.c > +set lib_sl1 [standard_output_file $libfile1.sl] > + > +set libfile2 "libsection2" > +set libsrc2 $srcdir/$subdir/$libfile2.c > +set lib_sl2 [standard_output_file $libfile2.sl] > + > +set lib_opts1 [list debug build-id shlib=3D$lib_sl2] > +set lib_opts2 [list debug build-id] > +set exec_opts [list debug build-id shlib=3D$lib_sl1 shlib=3D$lib_sl2] > + > +clean_restart > + > +if {[enable_section_downloads] =3D=3D 0} { > + untested "GDB does not support debuginfod section downloads" > + return -1 > +} > + > +# Compile SECTEXEC, LIB_SL1 and LIB_SL2. > +if { [gdb_compile_shlib $libsrc2 $lib_sl2 $lib_opts2] !=3D "" } { > + untested "failed to compile $libfile2" > + return -1 > +} > + > +if { [gdb_compile_shlib_pthreads $libsrc1 $lib_sl1 $lib_opts1] !=3D "" }= { > + untested "failed to compile $libfile1" > + return -1 > +} > + > +if { [gdb_compile $sectsrc $sectexec executable $exec_opts] !=3D "" } { > + untested "failed to compile $sectfile" > + return -1 > +} > + > +# Add .gdb_index to solibs. > +if { [have_index $lib_sl1] !=3D "gdb_index" > + && [add_gdb_index $lib_sl1] =3D=3D 0 } { > + untested "failed to add .gdb_index to $libfile1" > + return -1 > +} > + > +if { [have_index $lib_sl2] !=3D "gdb_index" > + && [add_gdb_index $lib_sl2] =3D=3D 0 } { > + untested "failed to add .gdb_index to $libfile2" > + return -1 > +} > + > +# Strip solib debuginfo into separate files. > +if { [gdb_gnu_strip_debug $lib_sl1 ""] !=3D 0} { > + fail "strip $lib_sl1 debuginfo" > + return -1 > +} > + > +if { [gdb_gnu_strip_debug $lib_sl2 ""] !=3D 0} { > + fail "strip $lib_sl2 debuginfo" > + return -1 > +} > + > +# Move debuginfo files into directory that debuginfod will serve from. > +set debugdir [standard_output_file "debug"] > +set debuginfo_sl1 [standard_output_file $libfile1.sl.debug] > +set debuginfo_sl2 [standard_output_file $libfile2.sl.debug] > + > +file mkdir $debugdir > +file rename -force $debuginfo_sl1 $debugdir > +file rename -force $debuginfo_sl2 $debugdir > + > +# Restart GDB and clear the debuginfod client cache. Then load BINFILE i= nto > +# GDB and start running it. Match output with pattern RES and use TESTN= AME > +# as the test name. > +proc_with_prefix clean_restart_with_prompt { binfile res testname } { > + global cache > + > + clean_restart > + > + # Delete client cache so debuginfo downloads again. > + file delete -force $cache > + > + gdb_test "file $binfile" "" "file [file tail $binfile] file $testnam= e" > + gdb_test "start" $res "file [file tail $binfile] start $testname" \ > + ".*Enable debuginfod.*" "y" > +} > + > +# Tests with no debuginfod server running. > +proc_with_prefix no_url { } { > + global sectexec libfile1 libfile2 > + > + # Check that no section is downloaded and no debuginfo is found. > + gdb_test "file $sectexec" "" "file [file tail $sectexec] file no url= " > + gdb_test "start" "" "file [file tail $sectexec] start no url" > + gdb_test "info sharedlibrary" ".*Yes \\(\\*\\).*$libfile1.*" \ > + "file [file tail $libfile1] found no url" > + gdb_test "info sharedlibrary" ".*Yes \\(\\*\\).*$libfile2.*" \ > + "file [file tail $libfile2] found no url" > +} > + > +# Tests with a debuginfod server running. > +proc_with_prefix local_url { } { > + global sectexec > + global libsrc1 lib_sl1 libfile1 > + global libsrc2 lib_sl2 libfile2 > + global debugdir db > + > + set url [start_debuginfod $db $debugdir] > + if { $url =3D=3D "" } { > + unresolved "failed to start debuginfod server" > + return > + } > + > + # Point GDB to the server. > + setenv DEBUGINFOD_URLS $url > + > + # Download .gdb_index for solibs. > + set res ".*section \.gdb_index for $lib_sl1.*\ > + section \.gdb_index for $lib_sl2.*" > + clean_restart_with_prompt $sectexec $res "index" > + > + # Download debuginfo when stepping into a function. > + set res ".*separate debug info for $lib_sl1.*\"In ${libfile1}\\\\n\"= .*" > + gdb_test "step" $res "file [file tail $lib_sl1] step" > + > + clean_restart_with_prompt $sectexec "" "break" > + > + # Download debuginfo when setting a breakpoint. > + set res "Download.*separate debug info for $lib_sl2.*" > + gdb_test "br libsection2_test" $res "file [file tail $sectexec] brea= k set" > + > + # Hit the breakpoint. > + set res ".*Breakpoint 2, libsection2_test.*\"In ${libfile2}\\\\n\".*= " > + gdb_test "c" $res "file [file tail $sectexec] break continue" > + > + # Check that download progress message is correctly formatted > + # during backtrace. > + set res ".* separate debug info for $lib_sl1.*#0 libsection2_test\ > + \\(\\) at.*" > + set res "Download.*debug info.*$lib_sl1.*#0 libsection2_test \\(\\)= at.*" > + gdb_test "bt" $res "file [file tail $sectexec] break backtrace" > +} > + > +# Create CACHE and DB directories ready for debuginfod to use. > +prepare_for_debuginfod cache db > + > +with_debuginfod_env $cache { > + no_url > + local_url > +} > + > +stop_debuginfod > diff --git a/gdb/testsuite/lib/debuginfod-support.exp b/gdb/testsuite/lib= /debuginfod-support.exp > index 50a8b512a4a..6368c27e9d0 100644 > --- a/gdb/testsuite/lib/debuginfod-support.exp > +++ b/gdb/testsuite/lib/debuginfod-support.exp > @@ -194,3 +194,33 @@ proc stop_debuginfod { } { > unset debuginfod_spawn_id > } > } > + > +# Return 1 if gdb is configured to download ELF/DWARF sections from > +# debuginfod servers. Otherwise return 0. > +proc enable_section_downloads { } { > + global gdb_prompt > + > + set cmd "maint set debuginfod download-sections on" > + set msg "enable section downloads" > + > + gdb_test_multiple $cmd $msg { > + -re ".*Undefined maintenance.*" { > + perror "Undefined command: \"$cmd\"" > + return 0 > + } > + -re ".*not compiled into GDB.*" { > + perror "Unsupported command: \"$cmd\"" > + return 0 > + } > + -re "\r\n${gdb_prompt} $" { > + return 1 > + } > + timeout { > + perror "timeout: \"$cmd\"" > + return 0 > + } > + } > + > + perror "Unexpected output for \"$cmd\"" > + return 0 > +} > -- > 2.40.1 >