[Bug gdb/25703] New: set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[Bug gdb/25703] New: set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25703

            Bug ID: 25703
           Summary: set dwarf max-cache-age default of 5 is slow for
                    inter-CU-reference binaries
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: gdb
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

In gdb/dwarf2/read.c, we find the default setting for "set dwarf
max-cache-age":
...
/* Loaded secondary compilation units are kept in memory until they             
   have not been referenced for the processing of this many                     
   compilation units.  Set this to zero to disable caching.  Cache              
   sizes of up to at least twenty will improve startup time for                 
   typical inter-CU-reference binaries, at an obvious memory cost.  */
static int dwarf_max_cache_age = 5;
...

See PR23710 comment 5 for an example where indeed increasing this setting for
an inter-cu-reference binary reducing loading time with 16.5%.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/25703] set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25703

--- Comment #1 from Tom de Vries <vries at gcc dot gnu.org> ---
Another data point, using system gdb 8.3.1 on openSUSE Leap 15.1:
...
$ time.sh gdb -batch -iex "maint set dwarf max-cache-age $n"
usr/lib/debug/usr/lib64/firefox/libxul.so-74.0-1.1.x86_64.debug
...

n=5 (default)
maxmem: 7084448
real: 362.42
user: 360.25
system: 2.13


n=10
maxmem: 7421056
real: 310.57
user: 308.35
system: 2.21

n=32
maxmem: 8273816
real: 250.26
user: 247.74
system: 2.47


n=100
maxmem: 9010380
real: 192.34
user: 189.75
system: 2.58

n=316
maxmem: 9637176
real: 143.71
user: 140.78
system: 2.91

n=1000
maxmem: 10504788
real: 114.22
user: 111.25
system: 2.96

n=1316
maxmem: 10502420
real: 119.02
user: 115.66
system: 3.34

So, increasing the value from 5 to 1000 reduces execution time with a factor 3
from 6 minutes to 2 minutes, at the cost of roughly 1.5 times more memory.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/25703] set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25703

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #1)
> Another data point, using system gdb 8.3.1 on openSUSE Leap 15.1:
> ...
> $ time.sh gdb -batch -iex "maint set dwarf max-cache-age $n"
> usr/lib/debug/usr/lib64/firefox/libxul.so-74.0-1.1.x86_64.debug
> ...

Number of CUs:
...
$ readelf -wi usr/lib/debug/usr/lib64/firefox/libxul.so-74.0-1.1.x86_64.debug |
grep -c "Compilation Unit"
2650
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/25703] set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25703

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

--- Comment #3 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Tom de Vries from comment #1)

> So, increasing the value from 5 to 1000 reduces execution time with a factor
> 3 from 6 minutes to 2 minutes, at the cost of roughly 1.5 times more memory.

This seems like a good tradeoff to me.

I was worried a little about people debugging programs that cause gdb
to use all the memory they have available.  However, it seems to me
that 1000 still implies some upper bound to the memory that is
(temporarily) used by gdb.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/25703] set dwarf max-cache-age default of 5 is slow for inter-CU-reference binaries

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=25703

--- Comment #4 from Tom Tromey <tromey at sourceware dot org> ---
I had some other thoughts about this recently:

1. It seems to me that if a CU ever refers to a DIE in
   some PU, then gdb could process the abbrev table for
   that PU and keep it for the duration of psymtab reading.
   The idea here is that if there's one reference to a PU, there
   are probably more.  Also, dwz tries to share abbrev
   tables, so caching like this could be a win that way as well.

2. In combination with 1, instead of reading all the partial
   DIEs for a PU when seeing a cross-unit reference, we
   could instead just read the exact DIE that is referred to.
   This might or might not be faster, depending on what all
   is referenced.

-- 
You are receiving this mail because:
You are on the CC list for the bug.