public inbox for gdb-prs@sourceware.org
help / color / mirror / Atom feed
* [Bug symtab/29942] New: Read some DWARF purely in background
@ 2022-12-25 21:21 tromey at sourceware dot org
2022-12-25 21:21 ` [Bug symtab/29942] " tromey at sourceware dot org
` (11 more replies)
0 siblings, 12 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2022-12-25 21:21 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Bug ID: 29942
Summary: Read some DWARF purely in background
Product: gdb
Version: HEAD
Status: NEW
Severity: normal
Priority: P2
Component: symtab
Assignee: unassigned at sourceware dot org
Reporter: tromey at sourceware dot org
Target Milestone: ---
Currently, dwarf2_build_psymtabs_hard works synchronously: it sends
some work to worker threads, but it waits for all work to be
completed before proceeding.
However, it doesn't have to work this way. Instead, it could start
reading immediately; and then after starting the background tasks,
it could store a future that would then be waited on by
cooked_index_functions::read_partial_symbols.
I think the main benefit to doing this would be that "attach"
might appear to be faster. In particular work like downloading
from debuginfod could be done while other threads are scanning
the DWARF.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
@ 2022-12-25 21:21 ` tromey at sourceware dot org
2022-12-25 21:22 ` tromey at sourceware dot org
` (10 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2022-12-25 21:21 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Tom Tromey <tromey at sourceware dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks| |29366
Referenced Bugs:
https://sourceware.org/bugzilla/show_bug.cgi?id=29366
[Bug 29366] [meta] New DWARF indexer meta bug
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
2022-12-25 21:21 ` [Bug symtab/29942] " tromey at sourceware dot org
@ 2022-12-25 21:22 ` tromey at sourceware dot org
2022-12-26 5:53 ` tromey at sourceware dot org
` (9 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2022-12-25 21:22 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #1 from Tom Tromey <tromey at sourceware dot org> ---
FWIW I have a WIP patch for this.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
2022-12-25 21:21 ` [Bug symtab/29942] " tromey at sourceware dot org
2022-12-25 21:22 ` tromey at sourceware dot org
@ 2022-12-26 5:53 ` tromey at sourceware dot org
2022-12-27 16:30 ` tromey at sourceware dot org
` (8 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2022-12-26 5:53 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #2 from Tom Tromey <tromey at sourceware dot org> ---
It occurs to me that maybe everything in dwarf2_build_psymtabs_hard
after the sections are read could be done in a worker thread.
(This kind of thing would maybe be easier with C++20 coroutines.)
> I think the main benefit to doing this would be that "attach"
> might appear to be faster.
The theory behind this is that with "attach", breakpoint re-sets
are deferred until all the shared libraries are read, providing
a little extra time for the background threads to work.
It might be interesting to see if minsym reading can also be
done in a worker.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (2 preceding siblings ...)
2022-12-26 5:53 ` tromey at sourceware dot org
@ 2022-12-27 16:30 ` tromey at sourceware dot org
2023-08-13 15:37 ` tromey at sourceware dot org
` (7 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2022-12-27 16:30 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #3 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Tom Tromey from comment #2)
> It occurs to me that maybe everything in dwarf2_build_psymtabs_hard
> after the sections are read could be done in a worker thread.
I haven't tested 'attach' yet, but this doesn't improve things
in the ordinary 'file' case, mainly I think due to the handling
of 'main'.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (3 preceding siblings ...)
2022-12-27 16:30 ` tromey at sourceware dot org
@ 2023-08-13 15:37 ` tromey at sourceware dot org
2023-08-18 17:49 ` tromey at sourceware dot org
` (6 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2023-08-13 15:37 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #4 from Tom Tromey <tromey at sourceware dot org> ---
I had a patch to add some thread-safety to BFD on this branch,
but I belatedly realized it can't really work, due to the way
the BFD fd cache works -- we'd have to introduce some way to
lock an fd while a function is using it.
Instead I'm going to try external locking and avoid modifying BFD at all.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (4 preceding siblings ...)
2023-08-13 15:37 ` tromey at sourceware dot org
@ 2023-08-18 17:49 ` tromey at sourceware dot org
2023-08-19 7:11 ` sam at gentoo dot org
` (5 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2023-08-18 17:49 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #5 from Tom Tromey <tromey at sourceware dot org> ---
I also looked at wrapping the cache iovec in a locked variant.
However, this code in bfd_cache_close prevents that approach:
/* Don't remove this test. bfd_reinit depends on it. */
if (abfd->iovec != &cache_iovec)
return true;
I think I'll just end up wrapping all "suspect" BFD calls in
a global lock. This may not be too bad, especially if we switch
back to eager mapping of DWARF sections.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (5 preceding siblings ...)
2023-08-18 17:49 ` tromey at sourceware dot org
@ 2023-08-19 7:11 ` sam at gentoo dot org
2023-08-23 20:02 ` tromey at sourceware dot org
` (4 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: sam at gentoo dot org @ 2023-08-19 7:11 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Sam James <sam at gentoo dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |sam at gentoo dot org
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (6 preceding siblings ...)
2023-08-19 7:11 ` sam at gentoo dot org
@ 2023-08-23 20:02 ` tromey at sourceware dot org
2023-10-04 12:47 ` tromey at sourceware dot org
` (3 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2023-08-23 20:02 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #6 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Tom Tromey from comment #5)
> I also looked at wrapping the cache iovec in a locked variant.
> However, this code in bfd_cache_close prevents that approach:
>
> /* Don't remove this test. bfd_reinit depends on it. */
> if (abfd->iovec != &cache_iovec)
> return true;
>
> I think I'll just end up wrapping all "suspect" BFD calls in
> a global lock. This may not be too bad, especially if we switch
> back to eager mapping of DWARF sections.
There are 135 separate bfd_* functions called by gdb.
So, auditing this is a pain and probably error-prone.
And then some subset have to be wrapped in a lock, including
any that set the BFD error.
Coming back around to maybe trying to make the BFD error be thread-local.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (7 preceding siblings ...)
2023-08-23 20:02 ` tromey at sourceware dot org
@ 2023-10-04 12:47 ` tromey at sourceware dot org
2023-11-12 23:27 ` tromey at sourceware dot org
` (2 subsequent siblings)
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2023-10-04 12:47 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Tom Tromey <tromey at sourceware dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|unassigned at sourceware dot org |tromey at sourceware dot org
--- Comment #7 from Tom Tromey <tromey at sourceware dot org> ---
I've been working on this.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (8 preceding siblings ...)
2023-10-04 12:47 ` tromey at sourceware dot org
@ 2023-11-12 23:27 ` tromey at sourceware dot org
2024-01-09 1:46 ` cvs-commit at gcc dot gnu.org
2024-01-09 1:48 ` tromey at sourceware dot org
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2023-11-12 23:27 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #8 from Tom Tromey <tromey at sourceware dot org> ---
https://sourceware.org/pipermail/gdb-patches/2023-November/203984.html
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (9 preceding siblings ...)
2023-11-12 23:27 ` tromey at sourceware dot org
@ 2024-01-09 1:46 ` cvs-commit at gcc dot gnu.org
2024-01-09 1:48 ` tromey at sourceware dot org
11 siblings, 0 replies; 13+ messages in thread
From: cvs-commit at gcc dot gnu.org @ 2024-01-09 1:46 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
--- Comment #9 from Sourceware Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom Tromey <tromey@sourceware.org>:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=33c6eaaefcedd45e86d564d014f14cce2620f933
commit 33c6eaaefcedd45e86d564d014f14cce2620f933
Author: Tom Tromey <tom@tromey.com>
Date: Fri Mar 24 23:35:02 2023 -0600
Do more DWARF reading in the background
This patch rearranges the DWARF reader so that more work is done in
the background. This is PR symtab/29942.
The idea here is that there is only a small amount of work that must
be done on the main thread when scanning DWARF -- before the main
scan, the only part is mapping the section data.
Currently, the DWARF reader uses the quick_symbol_functions "lazy"
functionality to defer even starting to read. This patch instead
changes the reader to start reading immediately, but doing more in
worker tasks.
Before this patch, "file" on my machine:
(gdb) file /tmp/gdb
2023-10-23 12:29:56.885 - command started
Reading symbols from /tmp/gdb...
2023-10-23 12:29:58.047 - command finished
Command execution time: 5.867228 (cpu), 1.162444 (wall)
After the patch, more work is done in the background and so this takes
a bit less time:
(gdb) file /tmp/gdb
2023-10-23 13:25:51.391 - command started
Reading symbols from /tmp/gdb...
2023-10-23 13:25:51.712 - command finished
Command execution time: 1.894500 (cpu), 0.320306 (wall)
I think this could be further sped up by using the shared library load
map to avoid objfile loops like the one in expand_symtab_containing_pc
-- it seems like the correct objfile could be chosen more directly.
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30174
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Bug symtab/29942] Read some DWARF purely in background
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
` (10 preceding siblings ...)
2024-01-09 1:46 ` cvs-commit at gcc dot gnu.org
@ 2024-01-09 1:48 ` tromey at sourceware dot org
11 siblings, 0 replies; 13+ messages in thread
From: tromey at sourceware dot org @ 2024-01-09 1:48 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=29942
Tom Tromey <tromey at sourceware dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|NEW |RESOLVED
Target Milestone|--- |15.1
--- Comment #10 from Tom Tromey <tromey at sourceware dot org> ---
Fixed.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2024-01-09 1:48 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-25 21:21 [Bug symtab/29942] New: Read some DWARF purely in background tromey at sourceware dot org
2022-12-25 21:21 ` [Bug symtab/29942] " tromey at sourceware dot org
2022-12-25 21:22 ` tromey at sourceware dot org
2022-12-26 5:53 ` tromey at sourceware dot org
2022-12-27 16:30 ` tromey at sourceware dot org
2023-08-13 15:37 ` tromey at sourceware dot org
2023-08-18 17:49 ` tromey at sourceware dot org
2023-08-19 7:11 ` sam at gentoo dot org
2023-08-23 20:02 ` tromey at sourceware dot org
2023-10-04 12:47 ` tromey at sourceware dot org
2023-11-12 23:27 ` tromey at sourceware dot org
2024-01-09 1:46 ` cvs-commit at gcc dot gnu.org
2024-01-09 1:48 ` tromey at sourceware dot org
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).