[Bug c++/18141] New: massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[Bug c++/18141] New: massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

            Bug ID: 18141
           Summary: massive perf issue trying to lookup namespaces in
                    STRUCT_DOMAIN with gdb_index
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: c++
          Assignee: unassigned at sourceware dot org
          Reporter: dje at google dot com

I found another case of the classic ".gdb_index says symbol xyz exists but then
lookup not finding it" perf issue.  With a few shared libraries the issue isn't
noticeable, but with 4500 shared libraries gdb is currently consuming 20GB of
memory, spent ~4 minutes cpu time (plus commensurately more wall time) and it's
still not done trying to do "print foo".

In this case gdb is doing:

#14 0x00000000008140d0 in cp_lookup_rtti_type (
    name=name@entry=0x1b57ec3 "std::foo<bar>", block=block@entry=0x0)
    at ../../gdb-7.9.x/gdb/cp-support.c:1446
  rtti_sym = lookup_symbol (name, block, STRUCT_DOMAIN, NULL);

which later results in this call:

cp_search_static_and_baseclasses:
  /* Lookup a class named KLASS.  If none is found, there is nothing            
     more that can be done.  */
  klass_sym = lookup_global_symbol (klass, block, domain);

with klass being "std".

However, namespaces are in VAR_DOMAIN.

Since namespaces are recorded in .gdb_index as a type,
dw2_symtab_iter_next will find it, causing the CU that defines
"std" to be expanded, which is good, but then subsequent lookup here:

dw2_lookup_symbol:
          sym = block_lookup_symbol (block, name, domain);

will fail because domain is STRUCT_DOMAIN and "std" is in VAR_DOMAIN.
The end result is that we expand one CU in each of the shared libraries that
has a .gdb_index entry for "std".

Ah, the print has now completed since I started writing this:

Command execution time: 297.609072 (cpu), 761.008252 (wall)
Space used: 25874649088 (+20380094464 for this command)
#symtabs: 622313 (+621693), #compunits: 8782 (+8774), #blocks: 4377936
(+4373024)

[I suspect the effective doubling in the number of expanded CUs (+8774) is due
to type units, but no matter.]

Here are the times with the proposed patch:

Command execution time: 0.605912 (cpu), 0.971609 (wall)
Space used: 5538639872 (+42561536 for this command)
#symtabs: 924 (+304), #compunits: 10 (+2), #blocks: 8950 (+4038)

diff --git a/gdb/cp-namespace.c b/gdb/cp-namespace.c
index 4a00cb6..d925a3b 100644
--- a/gdb/cp-namespace.c
+++ b/gdb/cp-namespace.c
@@ -355,8 +355,9 @@ cp_search_static_and_baseclasses (const char *name,
   make_cleanup (xfree, nested);

   /* Lookup a class named KLASS.  If none is found, there is nothing
-     more that can be done.  */
-  klass_sym = lookup_global_symbol (klass, block, domain);
+     more that can be done.  KLASS could be a namespace, so always look
+     in VAR_DOMAIN.  */
+  klass_sym = lookup_global_symbol (klass, block, VAR_DOMAIN);
   if (klass_sym == NULL)
     {
       do_cleanups (cleanup);

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |performance

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

--- Comment #1 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Doug Evans <devans@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=13ce92227425999aa2666f4d55286193df7d09ca

commit 13ce92227425999aa2666f4d55286193df7d09ca
Author: Doug Evans <dje@google.com>
Date:   Mon Mar 30 16:41:05 2015 -0700

    PR c++/18141

    gdb/ChangeLog:

        PR c++/18141
        * cp-namespace.c (cp_search_static_and_baseclasses): Always look for
        klass in VAR_DOMAIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #2 from dje at google dot com ---
Patch committed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|FIXED                       |---

--- Comment #3 from dje at google dot com ---
Found another related perf issue, this time with this line in
cp_lookup_rtti_type:

rtti_sym = lookup_symbol (name, block, STRUCT_DOMAIN, NULL);

classes are in VAR_DOMAIN, not STRUCT_DOMAIN

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

--- Comment #4 from dje at google dot com ---
(In reply to dje from comment #3)
> Found another related perf issue, this time with this line in
> cp_lookup_rtti_type:
> 
> rtti_sym = lookup_symbol (name, block, STRUCT_DOMAIN, NULL);
> 
> classes are in VAR_DOMAIN, not STRUCT_DOMAIN

Sorry, worded that poorly.
In this case NAME is a typedef, which live in VAR_DOMAIN.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

--- Comment #5 from dje at google dot com ---
See also PR 18417.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

--- Comment #6 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Doug Evans <devans@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=82c7be3106bbbf753f441f8a8113f2cac5e7cba8

commit 82c7be3106bbbf753f441f8a8113f2cac5e7cba8
Author: Doug Evans <dje@google.com>
Date:   Tue May 26 17:20:49 2015 -0700

    PR c++/18141, c++/18417.

    gdb/ChangeLog:

        * cp-support.c (cp_lookup_rtti_type): Handle the case of NAME being
        a typedef.

    gdb/testsuite/ChangeLog:

        * gdb.cp/iostream.cc: New file.
        * gdb.cp/iostream.exp: New file.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug c++/18141] massive perf issue trying to lookup namespaces in STRUCT_DOMAIN with gdb_index

[dje at google dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18141

dje at google dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #7 from dje at google dot com ---
patch committed

-- 
You are receiving this mail because:
You are on the CC list for the bug.