[Bug symtab/16998] New: perf improvements for searching GLOBAL_BLOCK

[Bug symtab/16998] New: perf improvements for searching GLOBAL_BLOCK

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

            Bug ID: 16998
           Summary: perf improvements for searching GLOBAL_BLOCK
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: symtab
          Assignee: unassigned at sourceware dot org
          Reporter: dje at google dot com

This pr is a corollary to pr 16994, and is for improving searching
GLOBAL_BLOCK.
[I'm trying to break down the entirety of gdb symbol problems into pieces -
there's just so many of them it's easier for me to keep track of them this way.
Still need one more bit of information to tie them together: I was thinking of
adding a "gdb-performance" "keyword".]

When .gdb_index was added, it was added in a way to let it easily sit along
side the psymtab implementation.  Which is totally reasonable as a first pass.

However, there is, I think, a performance cost.
The way gdb looks up global symbols (static ones too, but this bug is about
global symbols), is that it first searches already expanded symtabs and only if
not found there does gdb try the "quick_symbol_functions" API.

E.g.,

static int
lookup_symbol_global_iterator_cb (struct objfile *objfile,
                                  void *cb_data)
{
  struct global_sym_lookup_data *data =
    (struct global_sym_lookup_data *) cb_data;

  gdb_assert (data->result == NULL);

=>data->result = lookup_symbol_aux_objfile (objfile, GLOBAL_BLOCK,
                                            data->name, data->domain);
  if (data->result == NULL)
    data->result = lookup_symbol_aux_quick (objfile, GLOBAL_BLOCK,
                                            data->name, data->domain);

  /* If we found a match, tell the iterator to stop.  Otherwise,                
     keep going.  */
  return (data->result != NULL);
}

lookup_symbol_aux_objfile does this:

  ALL_OBJFILE_PRIMARY_SYMTABS (objfile, s)
    {
      bv = BLOCKVECTOR (s);
      block = BLOCKVECTOR_BLOCK (bv, block_index);
=>    sym = lookup_block_symbol (block, name, domain);
      if (sym)
        {
          block_found = block;
          return fixup_symbol_section (sym, objfile);
        }
    }

Over time symtabs get expanded, so the list to iterate over here grows.
But .gdb_index knows exactly which symtab(s) to look in.
[for global symbols there is only one, hence the "(s)" suffix is a bit of a
misnomer, but I'm leaving that aside for now]
So why not just always use .gdb_index for GLOBAL_BLOCK lookups?
[That's an honest question, maybe I'm missing something.
Still digging.]

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |performance

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

--- Comment #1 from dje at google dot com ---
I haven't looked into the details, but I wanted to write this down while it's
on my mind.  Iterating over symtabs hurts the non - .gdb_index (psymtabs) case
too.
While we're slurping in psyms, can we build enough of a .gdb_index work-alike
so that both the .gdb_index and psymtab cases can still use the same basic
algorithm for symbol lookup (look in index for list of symtabs to search in).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

--- Comment #2 from Tom Tromey <tromey at sourceware dot org> ---
> But .gdb_index knows exactly which symtab(s) to look in.
...
> So why not just always use .gdb_index for GLOBAL_BLOCK lookups?

It makes sense to me.
Basically we'd want to extend the "quick" functionality
to some symtab-level operations as well, or maybe just
change the lookup code and ensure that the quick methods
also know to iterate over already-expanded symtabs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

--- Comment #3 from Tom Tromey <tromey at sourceware dot org> ---
One subtlety here is that now that we have readnow_functions,
there is a situation where there really is no index.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at sourceware dot org   |tromey at sourceware dot org

--- Comment #4 from Tom Tromey <tromey at sourceware dot org> ---
I have patches for this but they have uncovered some other bugs
that must be fixed first.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |32482


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=32482
[Bug 32482] DWARF index doesn't follow strcmp_iw
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |32510


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=32510
[Bug 32510] Inconsistent treatment of template parameters in symbol lookup
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |32511


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=32511
[Bug 32511] Ada import symbols not created by DWARF indexer
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=16994

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

--- Comment #5 from Tom Tromey <tromey at sourceware dot org> ---
The new indexer doesn't create an entry for "t::t"
(among other things) in the anon-struct.exp test.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |32519


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=32519
[Bug 32519] DWARF index doesn't have entries for anon-struct.exp
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |32520


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=32520
[Bug 32520] Add maint command to check index integrity
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

--- Comment #6 from Tom Tromey <tromey at sourceware dot org> ---
One idea might be to land the initial patches and just not
the one that causes the "t::t" thing to be an issue.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Depends on|                            |27930


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=27930
[Bug 27930] gdb index does not handle DWARF 5 type units
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998
Bug 16998 depends on bug 32482, which changed state.

Bug 32482 Summary: DWARF index doesn't follow strcmp_iw
https://sourceware.org/bugzilla/show_bug.cgi?id=32482

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

--- Comment #7 from Tom Tromey <tromey at sourceware dot org> ---
I've been sending the independent bits of this series recently.
The main series itself is mostly done.  It is down to
6 regressions when using cc-with-gdb-index.  So hopefully
not too much longer.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998
Bug 16998 depends on bug 32510, which changed state.

Bug 32510 Summary: Inconsistent treatment of template parameters in symbol lookup
https://sourceware.org/bugzilla/show_bug.cgi?id=32510

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=25736

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/16998] perf improvements for searching GLOBAL_BLOCK

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=16998

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.