[Bug symtab/30520] New: Slow lookup_symbol_in_objfile

[Bug symtab/30520] New: Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

            Bug ID: 30520
           Summary: Slow lookup_symbol_in_objfile
           Product: gdb
           Version: 13.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: symtab
          Assignee: unassigned at sourceware dot org
          Reporter: dmitry.neverov at jetbrains dot com
  Target Milestone: ---

Created attachment 14919
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14919&action=edit
gdb.log

I'm facing a performance problem in `-var-list-children` gdb/mi command in gdb
13 built on commit d05c047a71374f533ed9261c6f44707285f1b302 but also earlier
gdb 13 versions. It seems like it has to do with slow lookup_symbol_in_objfile.
Most lookups are fast, but one takes around a minute to finish for 449Mb shared
object:

2023-06-06 10:54:51,286 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxObjectDataPtr,
VAR_DOMAIN)\n"
2023-06-06 10:55:50,418 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

Even though a lookup for another symbol in the same session before was fast:

2023-06-06 10:54:36,910 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxWindow,
VAR_DOMAIN)\n"
2023-06-06 10:54:36,910 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

What could cause such a slowdown? Are there any way to work that around, e.g.
by trading memory for speed?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #1 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
Created attachment 14920
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14920&action=edit
gdb-12.1.log

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #2 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
There seems to be no 1-minute delay in gdb-12.1.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

--- Comment #3 from Tom Tromey <tromey at sourceware dot org> ---
It's hard to know for sure what causes the problem
without more investigation.

A typical cause is too much symtab expansion.
Maybe retrying with "set debug symtab-create 1" would show
the problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #4 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
With 'set debug symtab-create 1' gdb produces ~450Mb of logs between these 2
entries:

2023-06-06 10:54:51,286 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxObjectDataPtr,
VAR_DOMAIN)\n"
2023-06-06 10:55:50,418 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

All log entries are of the form like these:

<&"    [symtab-create] start_subfile: name =
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h, name_for_id =
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h\n"
<&"    [symtab-create] start_subfile: found existing symtab with name_for_id
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h\n"

There are 1134332 '[symtab-create] start_subfile: name = ...' entries total,
707543 of them result in 'found existing symtab', 426789 don't find an existing
symtab.

There are 3519 unique names for symtab-create.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #5 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
If needed I think I can share all the logs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #6 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
I investigated it a bit further and the slowness seems to be
indeed caused by too much symtab expansion:

https://sourceware.org/pipermail/gdb/2023-October/050976.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |29366


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=29366
[Bug 29366] [meta] New DWARF indexer meta bug
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2023-12-01

--- Comment #7 from Tom Tromey <tromey at sourceware dot org> ---
First, thank you for your investigation.  That's very helpful.

This seems pretty similar to bug#31010 in that the cooked-index
code is expanding too many CUs for pretty much the same reason.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #8 from Tom Tromey <tromey at sourceware dot org> ---
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=31010

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[josh.cottingham at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Josh Cottingham <josh.cottingham at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |josh.cottingham at linaro dot org

--- Comment #9 from Josh Cottingham <josh.cottingham at linaro dot org> ---
Not sure if this is helpful but worth mentioning that bug#31010 does appear to
be fixed with
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[josh.cottingham at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #10 from Josh Cottingham <josh.cottingham at linaro dot org> ---
Not sure if this is helpful but worth mentioning that bug#31010 does appear to
be fixed with
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #11 from Tom Tromey <tromey at sourceware dot org> ---
*** Bug 31010 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Build|                            |2023-12-01 0:00
   Target Milestone|---                         |15.1

--- Comment #12 from Tom Tromey <tromey at sourceware dot org> ---
We have a patch series for this pending copyright assignment
(and some minor formatting); I'd like to get this in gdb 15,
so I'm setting the target milestone.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[brobecker at gnat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Joel Brobecker <brobecker at gnat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |brobecker at gnat dot com

--- Comment #13 from Joel Brobecker <brobecker at gnat dot com> ---
FTR: v3 patch series at:
https://sourceware.org/pipermail/gdb-patches/2024-May/209010.html

Reviewed by Tom who said they look reasonable (so I'm assuming they are
approved).

I will ping the FSF copyright office.

-- 
You are receiving this mail because:
You are on the CC list for the bug.