[Bug symtab/30520] New: Slow lookup_symbol_in_objfile

[Bug symtab/30520] New: Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

            Bug ID: 30520
           Summary: Slow lookup_symbol_in_objfile
           Product: gdb
           Version: 13.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: symtab
          Assignee: unassigned at sourceware dot org
          Reporter: dmitry.neverov at jetbrains dot com
  Target Milestone: ---

Created attachment 14919
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14919&action=edit
gdb.log

I'm facing a performance problem in `-var-list-children` gdb/mi command in gdb
13 built on commit d05c047a71374f533ed9261c6f44707285f1b302 but also earlier
gdb 13 versions. It seems like it has to do with slow lookup_symbol_in_objfile.
Most lookups are fast, but one takes around a minute to finish for 449Mb shared
object:

2023-06-06 10:54:51,286 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxObjectDataPtr,
VAR_DOMAIN)\n"
2023-06-06 10:55:50,418 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

Even though a lookup for another symbol in the same session before was fast:

2023-06-06 10:54:36,910 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxWindow,
VAR_DOMAIN)\n"
2023-06-06 10:54:36,910 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

What could cause such a slowdown? Are there any way to work that around, e.g.
by trading memory for speed?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #1 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
Created attachment 14920
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14920&action=edit
gdb-12.1.log

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #2 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
There seems to be no 1-minute delay in gdb-12.1.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

--- Comment #3 from Tom Tromey <tromey at sourceware dot org> ---
It's hard to know for sure what causes the problem
without more investigation.

A typical cause is too much symtab expansion.
Maybe retrying with "set debug symtab-create 1" would show
the problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sam at gentoo dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #4 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
With 'set debug symtab-create 1' gdb produces ~450Mb of logs between these 2
entries:

2023-06-06 10:54:51,286 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (_pcbnew.kiface, GLOBAL_BLOCK, wxObjectDataPtr,
VAR_DOMAIN)\n"
2023-06-06 10:55:50,418 <&"    [symbol-lookup] lookup_symbol_in_objfile:
lookup_symbol_in_objfile (...) = NULL\n"

All log entries are of the form like these:

<&"    [symtab-create] start_subfile: name =
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h, name_for_id =
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h\n"
<&"    [symtab-create] start_subfile: found existing symtab with name_for_id
/usr/include/x86_64-linux-gnu/c++/12/bits/c++config.h\n"

There are 1134332 '[symtab-create] start_subfile: name = ...' entries total,
707543 of them result in 'found existing symtab', 426789 don't find an existing
symtab.

There are 3519 unique names for symtab-create.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #5 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
If needed I think I can share all the logs.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[dmitry.neverov at jetbrains dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #6 from Dmitry Neverov <dmitry.neverov at jetbrains dot com> ---
I investigated it a bit further and the slowness seems to be
indeed caused by too much symtab expansion:

https://sourceware.org/pipermail/gdb/2023-October/050976.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |29366


Referenced Bugs:

https://sourceware.org/bugzilla/show_bug.cgi?id=29366
[Bug 29366] [meta] New DWARF indexer meta bug
-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2023-12-01

--- Comment #7 from Tom Tromey <tromey at sourceware dot org> ---
First, thank you for your investigation.  That's very helpful.

This seems pretty similar to bug#31010 in that the cooked-index
code is expanding too many CUs for pretty much the same reason.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #8 from Tom Tromey <tromey at sourceware dot org> ---
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[sam at gentoo dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Sam James <sam at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://sourceware.org/bugz
                   |                            |illa/show_bug.cgi?id=31010

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[josh.cottingham at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Josh Cottingham <josh.cottingham at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |josh.cottingham at linaro dot org

--- Comment #9 from Josh Cottingham <josh.cottingham at linaro dot org> ---
Not sure if this is helpful but worth mentioning that bug#31010 does appear to
be fixed with
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[josh.cottingham at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #10 from Josh Cottingham <josh.cottingham at linaro dot org> ---
Not sure if this is helpful but worth mentioning that bug#31010 does appear to
be fixed with
https://sourceware.org/pipermail/gdb-patches/2024-January/205924.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #11 from Tom Tromey <tromey at sourceware dot org> ---
*** Bug 31010 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Build|                            |2023-12-01 0:00
   Target Milestone|---                         |15.1

--- Comment #12 from Tom Tromey <tromey at sourceware dot org> ---
We have a patch series for this pending copyright assignment
(and some minor formatting); I'd like to get this in gdb 15,
so I'm setting the target milestone.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[brobecker at gnat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Joel Brobecker <brobecker at gnat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |brobecker at gnat dot com

--- Comment #13 from Joel Brobecker <brobecker at gnat dot com> ---
FTR: v3 patch series at:
https://sourceware.org/pipermail/gdb-patches/2024-May/209010.html

Reviewed by Tom who said they look reasonable (so I'm assuming they are
approved).

I will ping the FSF copyright office.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

--- Comment #14 from Sourceware Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom Tromey <tromey@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=3a0fae312983989a33608d924ff902d7b78e8ec1

commit 3a0fae312983989a33608d924ff902d7b78e8ec1
Author: Dmitry.Neverov <dmitry.neverov@jetbrains.com>
Date:   Mon May 6 17:09:17 2024 +0200

    gdb/symtab: check name matches before expanding a CU

    The added check fixes the case when an unqualified lookup
    name without template arguments causes expansion of many CUs
    which contain the name with template arguments.

    This is similar to what dw2_expand_symtabs_matching_symbol does
    before expanding the CU.

    In the referenced issue the lookup name was wxObjectDataPtr and many
    CUs had names like wxObjectDataPtr<wxBitmapBundleImpl>. This caused
    their expansion and the lookup took around a minute. The added check
    helps to avoid the expansion and makes the symbol lookup to return in
    a second or so.

    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=30520

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug symtab/30520] Slow lookup_symbol_in_objfile

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=30520

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #15 from Tom Tromey <tromey at sourceware dot org> ---
Should be fixed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.