[Bug symtab/29391] New: [gdb/symtab] Parallelize process_queue

[Bug symtab/29391] New: [gdb/symtab] Parallelize process_queue

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29391

            Bug ID: 29391
           Summary: [gdb/symtab] Parallelize process_queue
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: symtab
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

I've pushed a WIP branch here (
https://sourceware.org/git/?p=binutils-gdb.git;a=shortlog;h=refs/heads/users/vries/process-queue-parallel
).

More details in the COVER-LETTER (
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=b4045c876dbd39612141bad2eb3cedb73f6835c2
).

Any comments welcome.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/29391] [gdb/symtab] Parallelize process_queue

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29391

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/29391] [gdb/symtab] Parallelize process_queue

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29391

--- Comment #1 from Tom Tromey <tromey at sourceware dot org> ---
It's an interesting idea but as you found there are some issues.

The main issue behind a lot of the allocation problems is that
gdb has a few per-objfile data structures that can't easily be
used from multiple threads: the obstack but also the bcache
and the demangled hash table.

Maybe these problems could all be solved by sharding, or maybe
by heap allocation.  Also I think a couple patches in the series
introduce locks where something like compare-and-swap would work
just as well.

However, I tend to think there's a better approach overall.

The way I see it, there are two main issues with CU expansion.

One is that sometimes gdb decides to expand too many CUs in
response to a request.  This is maybe covered by one of the 
dependencies of bug #29366.  I am not sure yet (haven't looked
in detail) but I suspect the fix will be something like
short-circuiting expansion for certain kinds of queries.
Like, if gdb is looking for a type, just expand the first CU
that matches.

The second problem is that CU expansion can be slow.  Here I think
gdb could do a lot better, the basic idea being lazy CU expansion.
In response to a CU expansion request, the DWARF reader would
create the symtab / compunit_symtab structures and it would also
create some "outline" struct symbols -- one for each cooked_index_entry.
Then when some attribute of a symbol is needed (say, the type),
the DWARF reader would read the rest of the symbol that that moment.

The major advantage of this approach is that most data in a CU
is not needed at all.  So, much less work would need to be done in
general.  I think it would be possible to avoid reading every DIE.

A secondary advantage is that, because the symbols are created directly
from the cooked index, we would avoid the situation where the
two readers could diverge.  That would no longer be possible at all.

There are some downsides.  It's more complex, and it is complicated to
implement and test.  Also I think it would require fixing the .debug_names
bug, and also probably removing .gdb_index support.  Finally, we'd have
to change the blockvector to be expandable.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/29391] [gdb/symtab] Parallelize process_queue

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29391

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #1)
> The second problem is that CU expansion can be slow.  Here I think
> gdb could do a lot better, the basic idea being lazy CU expansion.

Filed as PR29398 - [gdb/symtab] lazy CU expansion.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/29391] [gdb/symtab] Parallelize process_queue

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=29391

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |29366


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=29366
[Bug 29366] [meta] New DWARF indexer meta bug
-- 
You are receiving this mail because:
You are on the CC list for the bug.