* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
@ 2020-03-08 11:09 ` vries at gcc dot gnu.org
2020-03-08 13:14 ` jan.kratochvil at redhat dot com
` (19 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2020-03-08 11:09 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #11 from Tom de Vries <vries at gcc dot gnu.org> ---
Comparison, cc1 vs cc1.dwz (produced using dwz build from current master
branch):
...
$ diff.sh cc1 cc1.dwz
.debug_info red: 49.30% 97418416 49399513
.debug_abbrev red: 42.04% 1699940 985372
.debug_str red: 0% 6344030 6344030
total red: 46.21% 105462386 56728915
...
lldb roughly uses same amount of memory, that is: cc1.dwz uses 5.7% less:
...
$ time.sh lldb -batch cc1 -o "b do_rpo_vn"
(lldb) target create "cc1"
Current executable set to 'cc1' (x86_64).
(lldb) b do_rpo_vn
Breakpoint 1: 3 locations.
maxmem: 519116
real: 2.63
user: 4.21
system: 0.14
$ time.sh lldb -batch cc1.dwz -o "b do_rpo_vn"
(lldb) target create "cc1.dwz"
Current executable set to 'cc1.dwz' (x86_64).
(lldb) b do_rpo_vn
Breakpoint 1: 3 locations.
maxmem: 489596
real: 2.78
user: 4.01
system: 0.10
...
With gdb, the difference is a reduction of 51.9%:
...
$ time.sh gdb cc1 -batch -iex "set language c++" -iex "maint set dwarf
max-cache-age 316" -ex "b do_rpo_vn"
Breakpoint 1 at 0xd40e30: do_rpo_vn. (2 locations)
maxmem: 999404
real: 7.03
user: 6.81
system: 0.25
$ time.sh gdb cc1.dwz -batch -iex "set language c++" -iex "maint set dwarf
max-cache-age 316" -ex "b do_rpo_vn"
Breakpoint 1 at 0xd40e30: do_rpo_vn. (2 locations)
maxmem: 481152
real: 6.15
user: 6.09
system: 0.12
...
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
2020-03-08 11:09 ` [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC vries at gcc dot gnu.org
@ 2020-03-08 13:14 ` jan.kratochvil at redhat dot com
2020-03-17 7:56 ` cvs-commit at gcc dot gnu.org
` (18 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: jan.kratochvil at redhat dot com @ 2020-03-08 13:14 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #12 from Jan Kratochvil <jan.kratochvil at redhat dot com> ---
(In reply to Tom de Vries from comment #11)
> $ time.sh lldb -batch cc1.dwz -o "b do_rpo_vn"
FYI upstream LLDB does not yet support DWZ to make such measurement valid.
There is an off-trunk patchset for it:
git clone -b dwz git://git.jankratochvil.net/lldb
It works but it is still being refactored to get it accepted upstream.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
2020-03-08 11:09 ` [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC vries at gcc dot gnu.org
2020-03-08 13:14 ` jan.kratochvil at redhat dot com
@ 2020-03-17 7:56 ` cvs-commit at gcc dot gnu.org
2020-03-20 9:06 ` vries at gcc dot gnu.org
` (17 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: cvs-commit at gcc dot gnu.org @ 2020-03-17 7:56 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #13 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom de Vries <vries@sourceware.org>:
https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=589902954da0d1dd140b33e578954746c9bfc374
commit 589902954da0d1dd140b33e578954746c9bfc374
Author: Tom de Vries <tdevries@suse.de>
Date: Tue Mar 17 08:56:36 2020 +0100
[gdb] Skip imports of c++ CUs
The DWARF standard appendix E.1 describes techniques that can be used for
compression and deduplication: DIEs can be factored out into a new
compilation
unit, and referenced using DW_FORM_ref_addr.
Such a new compilation unit can either use a DW_TAG_compile_unit or
DW_TAG_partial_unit. If a DW_TAG_compile_unit is used, its contents is
evaluated by consumers as though it were an ordinary compilation unit. If
a
DW_TAG_partial_unit is used, it's only considered by consumers in the
context
of a DW_TAG_imported_unit.
An example of when DW_TAG_partial_unit is required is when the factored out
DIEs are not top-level, f.i. because they were children of a namespace. In
such a case the corresponding DW_TAG_imported_unit will occur as child of
the
namespace.
In the case of factoring out DIEs from c++ compilation units, we can factor
out into a new DW_TAG_compile_unit, and no DW_TAG_imported_unit is
required.
This begs the question how to interpret a top-level DW_TAG_imported_unit of
a
c++ DW_TAG_compile_unit compilation unit. The semantics of
DW_TAG_imported_unit describe that the imported unit logically appears at
the
point of the DW_TAG_imported_unit entry. But it's not clear what the
effect
should be in this case, since all the imported DIEs are already globally
visible anyway, due to the use of DW_TAG_compile_unit.
So, skip top-level imports of c++ DW_TAG_compile_unit compilation units in
process_imported_unit_die.
Using the cc1 binary from PR23710 comment 1 and setting a breakpoint on
do_rpo_vn:
...
$ gdb \
-batch \
-iex "maint set dwarf max-cache-age 316" \
-iex "set language c++" \
-ex "b do_rpo_vn" \
cc1
...
we get a 8.1% reduction in execution time, due to reducing the number of
partial symtabs expanded into full symtabs from 212 to 175.
Build and reg-tested on x86_64-linux.
gdb/ChangeLog:
2020-03-17 Tom de Vries <tdevries@suse.de>
PR gdb/23710
* dwarf2/read.h (struct dwarf2_per_cu_data): Add unit_type and lang
fields.
* dwarf2/read.c (process_psymtab_comp_unit): Initialize unit_type
and lang
fields.
(process_imported_unit_die): Skip import of c++ CUs.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (2 preceding siblings ...)
2020-03-17 7:56 ` cvs-commit at gcc dot gnu.org
@ 2020-03-20 9:06 ` vries at gcc dot gnu.org
2020-04-02 12:39 ` vries at gcc dot gnu.org
` (16 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2020-03-20 9:06 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #14 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #5)
> (In reply to Richard Biener from comment #0)
> > gdb takes ~10s to process a LTO bootstrapped cc1 binary and another two
> > seconds
> > when setting the first breakpoint. It also has allocated 1.6GB memory at
> > that point (compared to ~200MB for a non-LTO binary).
>
> There's a setting "maint set/show dwarf max-cache-age" which defaults to 5.
>
> Using a higher setting, I get the following reduction in real execution time:
> - 10 : 1.5%
> - 100 : 12.5%
> - 316 : 16.5%
> - 1000 : 16.5%
> - 10000 : 16.5%
> - 100000: 15.5%
>
> Note: adding the setting to the gdb command line using -iex to make sure it
> gets set _before_ loading the exec):
> ...
> $ gdb -q -nw -nx -batch -iex "maint set dwarf max-cache-age $n" -ex "b
> do_rpo_vn" cc1
> ...
>
> Conversely, disabling the cache by setting the value to 0 causes a real
> execution time increase of 46%.
Filed PR25703 - "set dwarf max-cache-age default of 5 is slow for
inter-CU-reference binaries".
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (3 preceding siblings ...)
2020-03-20 9:06 ` vries at gcc dot gnu.org
@ 2020-04-02 12:39 ` vries at gcc dot gnu.org
2020-07-15 9:43 ` rdiezmail-binutils at yahoo dot de
` (15 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2020-04-02 12:39 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #15 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #8)
> (In reply to Tom de Vries from comment #6)
> > (In reply to Richard Biener from comment #0)
> > > "Mirror" of https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87432 where we try
> > > to improve the GCC side. Input from gdb folks is needed here.
> > >
> > > gdb takes ~10s to process a LTO bootstrapped cc1 binary and another two
> > > seconds
> > > when setting the first breakpoint. It also has allocated 1.6GB memory at
> > > that point (compared to ~200MB for a non-LTO binary).
> >
> > The proposed patch at
> > https://sourceware.org/ml/gdb-patches/2020-02/msg00974.html allows a speedup
> > when manually specifying the source language before loading, which takes 17%
> > off the execution time of loading and setting the first breakpoint.
>
> That patch has been accepted.
>
> I've submitted a RFC patch that automates this workaround:
> https://sourceware.org/ml/gdb-patches/2020-03/msg00009.html .
Committed:
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=d3214198119c1a2f9a6a2b8fcc56d8c324e1a245
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (4 preceding siblings ...)
2020-04-02 12:39 ` vries at gcc dot gnu.org
@ 2020-07-15 9:43 ` rdiezmail-binutils at yahoo dot de
2020-07-16 12:51 ` rdiezmail-binutils at yahoo dot de
` (14 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: rdiezmail-binutils at yahoo dot de @ 2020-07-15 9:43 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
rdiezmail-binutils at yahoo dot de changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |rdiezmail-binutils at yahoo dot de
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (5 preceding siblings ...)
2020-07-15 9:43 ` rdiezmail-binutils at yahoo dot de
@ 2020-07-16 12:51 ` rdiezmail-binutils at yahoo dot de
2020-11-24 11:03 ` rdiezmail-binutils at yahoo dot de
` (13 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: rdiezmail-binutils at yahoo dot de @ 2020-07-16 12:51 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #16 from rdiezmail-binutils at yahoo dot de ---
I develop a relative small firmware for ARM Cortex-M4F since some years ago,
and I did not notice any slowdown up to and including GCC 8.3 and GDB 8.3.
In the past months, I upgraded my toolchain to GCC 9.3 and GDB 9.2, and then I
started noticing a big slowdown of several seconds on the first "hbreak
myfunction" command. I guess that is the first time that GDB loads the symbols.
The slowdown only happens with release builds compiled with LTO. With debug
builds (with asserts and without LTO), GDB start-up is instantaneous.
These are the GDB RAM usage stats I collected:
Debug firmware build: VSZ: 101 MiB, RSS 33 MiB.
Release firmware build: VSZ: 430 MiB, RSS 362 MiB.
I do not understand why this difference, because it is exactly the same
firmware. The LTO build is smaller and faster, but it has the same symbols (or
less, because asserts etc. are per "#ifndef NDEBUG" no longer there).
I hope the patches above fix this issue. But I would say that the GDB's
handling of LTO builds would not need a 30 % speed increase, but more like a 10
fold improvement.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (6 preceding siblings ...)
2020-07-16 12:51 ` rdiezmail-binutils at yahoo dot de
@ 2020-11-24 11:03 ` rdiezmail-binutils at yahoo dot de
2021-03-09 19:08 ` hi-angel at yandex dot ru
` (12 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: rdiezmail-binutils at yahoo dot de @ 2020-11-24 11:03 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #17 from rdiezmail-binutils at yahoo dot de ---
I did some more tests after GDB 10.1 was released. I am now using GCC 10.2 .
GDB 10.1 is faster, but still pauses around 1.5 seconds. My firmware generates
an LTO release build .elf file that weighs 10.4 MiB, and GDB uses a lot of RAM:
VSZ 620 MiB, RSS 286 MiB.
The debug build .elf weighs 8.9 MiB, which is less even though it has more code
because of all the asserts. GDB does not pause at all and uses less RAM: VSZ
372 MiB, RSS 38 MiB.
I tried setting "maintenance set worker-threads 1", because the default has
changed with GDB 10.1, but it did not seem to make much of a difference, so it
is probably an unrelated setting.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (7 preceding siblings ...)
2020-11-24 11:03 ` rdiezmail-binutils at yahoo dot de
@ 2021-03-09 19:08 ` hi-angel at yandex dot ru
2021-04-07 7:25 ` rdiezmail-binutils at yahoo dot de
` (11 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: hi-angel at yandex dot ru @ 2021-03-09 19:08 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
Hi-Angel <hi-angel at yandex dot ru> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |hi-angel at yandex dot ru
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (8 preceding siblings ...)
2021-03-09 19:08 ` hi-angel at yandex dot ru
@ 2021-04-07 7:25 ` rdiezmail-binutils at yahoo dot de
2021-05-31 12:14 ` vries at gcc dot gnu.org
` (10 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: rdiezmail-binutils at yahoo dot de @ 2021-04-07 7:25 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #18 from rdiezmail-binutils at yahoo dot de ---
Could someone clarify the status of this issue? Is a fix going to land soon?
I would also like to know what the last version of GDB is without this issue,
if any. And whether it would still work with newer GCC versions.
Yesterday, GDB used too much memory on my PC. It even managed to freeze my
Linux system, to the point that not even the mouse cursor was moving. I had to
pull the power cord. I don't have a swap file, but the OOM killer did not
trigger.
I am using a toolchain similar to this one:
https://github.com/rdiez/JtagDue/blob/master/Toolchain/Makefile
But I am targetting an embedded ARM Cortex-M4 at the moment.
The toolchain component versions I was using were:
BINUTILS_VERSION := 2.35.1
GMP_VERSION := 6.2.1
MPFR_VERSION := 4.1.0
MPC_VERSION := 1.2.1
GCC_VERSION := 10.2.0
NEWLIB_VERSION := 4.0.0
GDB_VERSION := 10.1
GDB version 10.1 was sluggish with LTO release builds, but because I normally
use non-LTO debug builds, I have put up with this issue for a long time.
However, I recently upgraded these toolchain component versions and rebuilt the
toolchain:
BINUTILS_VERSION := 2.36.1
NEWLIB_VERSION := 4.1.0
After the upgrade, GDB started using gigabytes of memory when programming the
firmware.
The embedded firmware I am building has changed too, but it is very similar, so
it is hard to say if any particular change, for example #including more files
(which could bring in more debug symbols), has triggered the problem.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (9 preceding siblings ...)
2021-04-07 7:25 ` rdiezmail-binutils at yahoo dot de
@ 2021-05-31 12:14 ` vries at gcc dot gnu.org
2021-06-10 18:35 ` tromey at sourceware dot org
` (9 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2021-05-31 12:14 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #19 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #7)
> Breakpoint 1 at 0xd40e30: do_rpo_vn. (2 locations)
Starting commit 77f2120b200 "Don't drop static function bp locations w/o debug
info", we have 3 locations, which is caused by PR26096.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (10 preceding siblings ...)
2021-05-31 12:14 ` vries at gcc dot gnu.org
@ 2021-06-10 18:35 ` tromey at sourceware dot org
2021-06-10 22:59 ` vries at gcc dot gnu.org
` (8 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: tromey at sourceware dot org @ 2021-06-10 18:35 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
Tom Tromey <tromey at sourceware dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |tromey at sourceware dot org
--- Comment #20 from Tom Tromey <tromey at sourceware dot org> ---
Tom --
See https://sourceware.org/pipermail/gdb-patches/2021-June/179765.html
I found out that if the test case from this patch is changed
to use DW_LANG_C, it will fail. (Of course I thinko'd that message
and wrote C++, but the test already uses C++...)
I also don't understand why the DWARF reader check is specific to C++.
It seems like any import of a CU could be skipped, since that CU
will be scanned separately anyway.
Furthermore, the skipping should probably also be done in the psymtab
reader, not just the full reader.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (11 preceding siblings ...)
2021-06-10 18:35 ` tromey at sourceware dot org
@ 2021-06-10 22:59 ` vries at gcc dot gnu.org
2021-06-11 15:02 ` tromey at sourceware dot org
` (7 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2021-06-10 22:59 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #21 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #20)
> Tom --
>
> See https://sourceware.org/pipermail/gdb-patches/2021-June/179765.html
> I found out that if the test case from this patch is changed
> to use DW_LANG_C, it will fail. (Of course I thinko'd that message
> and wrote C++, but the test already uses C++...)
>
You mean gdb.dwarf2/imported-unit-bp.exp? That test-case uses C, right?
> I also don't understand why the DWARF reader check is specific to C++.
> It seems like any import of a CU could be skipped, since that CU
> will be scanned separately anyway.
>
Tried to explain here (
https://sourceware.org/pipermail/gdb-patches/2021-June/179804.html ). Also, I
tried to explain this in the commit log here (
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=589902954da0d1dd140b33e578954746c9bfc374
).
> Furthermore, the skipping should probably also be done in the psymtab
> reader, not just the full reader.
Maybe, not sure yet.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (12 preceding siblings ...)
2021-06-10 22:59 ` vries at gcc dot gnu.org
@ 2021-06-11 15:02 ` tromey at sourceware dot org
2021-06-22 13:24 ` vries at gcc dot gnu.org
` (6 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: tromey at sourceware dot org @ 2021-06-11 15:02 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #22 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Tom de Vries from comment #21)
> (In reply to Tom Tromey from comment #20)
> > Tom --
> >
> > See https://sourceware.org/pipermail/gdb-patches/2021-June/179765.html
> > I found out that if the test case from this patch is changed
> > to use DW_LANG_C, it will fail. (Of course I thinko'd that message
> > and wrote C++, but the test already uses C++...)
> >
>
> You mean gdb.dwarf2/imported-unit-bp.exp? That test-case uses C, right?
Yeah,sorry. I double confused myself I guess.
If you convert that test to use DW_LANG_C_plus_plus, then run it, it will fail.
So it seems to me that this patch had some unintended consequence.
I haven't looked into why, and TBH it doesn't really make sense to me.
> > Furthermore, the skipping should probably also be done in the psymtab
> > reader, not just the full reader.
>
> Maybe, not sure yet.
Normally the rule is that the psymtab reader and the full symtab reader
must agree. Now, this case is a bit weird in that nothing really checks
whether a psymtab dependency is really read in. Though, the above failure
seems to indicate that it may matter.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (13 preceding siblings ...)
2021-06-11 15:02 ` tromey at sourceware dot org
@ 2021-06-22 13:24 ` vries at gcc dot gnu.org
2022-06-17 19:06 ` jsm28 at gcc dot gnu.org
` (5 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: vries at gcc dot gnu.org @ 2021-06-22 13:24 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #23 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #22)
> (In reply to Tom de Vries from comment #21)
> > (In reply to Tom Tromey from comment #20)
> > > Tom --
> > >
> > > See https://sourceware.org/pipermail/gdb-patches/2021-June/179765.html
> > > I found out that if the test case from this patch is changed
> > > to use DW_LANG_C, it will fail. (Of course I thinko'd that message
> > > and wrote C++, but the test already uses C++...)
> > >
> >
> > You mean gdb.dwarf2/imported-unit-bp.exp? That test-case uses C, right?
>
> Yeah,sorry. I double confused myself I guess.
>
> If you convert that test to use DW_LANG_C_plus_plus, then run it, it will
> fail.
> So it seems to me that this patch had some unintended consequence.
> I haven't looked into why, and TBH it doesn't really make sense to me.
>
> > > Furthermore, the skipping should probably also be done in the psymtab
> > > reader, not just the full reader.
> >
> > Maybe, not sure yet.
>
> Normally the rule is that the psymtab reader and the full symtab reader
> must agree. Now, this case is a bit weird in that nothing really checks
> whether a psymtab dependency is really read in. Though, the above failure
> seems to indicate that it may matter.
Submitted patch to fix this:
https://sourceware.org/pipermail/gdb-patches/2021-June/180229.html
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (14 preceding siblings ...)
2021-06-22 13:24 ` vries at gcc dot gnu.org
@ 2022-06-17 19:06 ` jsm28 at gcc dot gnu.org
2022-06-17 19:12 ` dpmendenhall at gmail dot com
` (4 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: jsm28 at gcc dot gnu.org @ 2022-06-17 19:06 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
Joseph Myers <jsm28 at gcc dot gnu.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jsm28 at gcc dot gnu.org
--- Comment #24 from Joseph Myers <jsm28 at gcc dot gnu.org> ---
I've observed a case of GDB slowness on LTO code, still present with current
GDB (testing here with GDB as of commit
2d9cf99d9a6c701de912d3e95ea3ffa134af4c62), that looks a bit different from the
cases discussed here.
The customer test case has about 10 MB of text and about 1 GB of debug info in
the main C++ application (there are also lots of shared libraries involved).
Using GDB to examine a core dump (with about 300 threads), either "info
threads" or "thread apply all bt" is both very slow on a binary built with LTO
(maybe 10 times slower than on a non-LTO binary) and consumes much more memory.
For the LTO binary and core dump, GDB loads the debug info for many more
compilation units than in the non-LTO case, resulting in many more DIEs being
loaded, process_die being called many more times (a factor of about 10) and
much more time being spent in it (a large proportion of execution time in the
LTO case is spent in process_die and its children).
The key difference in the debug info in the LTO and non-LTO cases that causes
this is references from the debug info for one CU to the debug info for another
CU, as handled by follow_die_offset. In the non-LTO case these don't occur at
all. In the LTO case, there are many such references - the greatest proportion
are DW_TAG_subprogram, but also various others such as DW_TAG_namespace and
DW_TAG_variable.
The key call is in follow_die_offset:
/* If necessary, add it to the queue and load its DIEs.
Even if maybe_queue_comp_unit doesn't require us to load the CU's
DIEs,
it doesn't mean they are currently loaded. Since we require them
to be loaded, we must check for ourselves. */
if (maybe_queue_comp_unit (cu, per_cu, per_objfile, cu->per_cu->lang)
|| per_objfile->get_cu (per_cu) == nullptr)
load_full_comp_unit (per_cu, per_objfile, per_objfile->get_cu (per_cu),
false, cu->per_cu->lang);
This call to load_full_comp_unit gets executed 9960 times in the LTO case, but
not at all in the non-LTO case. The other call to load_full_comp_unit that gets
executed is the one from load_cu (201 times in the non-LTO case, 150 in the LTO
case). So the DIEs from many more CUs are loaded in the LTO case. Then
process_full_comp_unit calls process_die 2250 times in the LTO case but only
186 times in the non-LTO case (and that recurses down to process all the DIEs
in the CU).
The underlying issue here looks like GDB's strategy of loading all the DIEs
from any CU referenced by the debug info from a CU it's loading debug info
from, rather than somehow e.g. only selectively loading the DIEs it needs for
the particular backtrace being printed.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (15 preceding siblings ...)
2022-06-17 19:06 ` jsm28 at gcc dot gnu.org
@ 2022-06-17 19:12 ` dpmendenhall at gmail dot com
2022-06-18 18:13 ` tromey at sourceware dot org
` (3 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: dpmendenhall at gmail dot com @ 2022-06-17 19:12 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
David Mendenhall <dpmendenhall at gmail dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |dpmendenhall at gmail dot com
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (16 preceding siblings ...)
2022-06-17 19:12 ` dpmendenhall at gmail dot com
@ 2022-06-18 18:13 ` tromey at sourceware dot org
2022-06-20 6:59 ` rguenth at gcc dot gnu.org
` (2 subsequent siblings)
20 siblings, 0 replies; 21+ messages in thread
From: tromey at sourceware dot org @ 2022-06-18 18:13 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #25 from Tom Tromey <tromey at sourceware dot org> ---
Changing the full reader to read DIEs on demand is almost doable.
The major problem I see here is that some code looks at a DIE's
parent -- but due to the nature of DWARF, this requires a more
full scan of the DIE tree (there are no parent links in the
.debug_info itself)
Longer term I would like to have gdb make a symtab directly
from the index, and then lazily instantiate symbol contents
when needed. I think this would be much faster.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (17 preceding siblings ...)
2022-06-18 18:13 ` tromey at sourceware dot org
@ 2022-06-20 6:59 ` rguenth at gcc dot gnu.org
2023-01-18 13:47 ` fabian@ritter-vogt.de
2023-01-19 2:25 ` sam at gentoo dot org
20 siblings, 0 replies; 21+ messages in thread
From: rguenth at gcc dot gnu.org @ 2022-06-20 6:59 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
--- Comment #26 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #25)
> Changing the full reader to read DIEs on demand is almost doable.
> The major problem I see here is that some code looks at a DIE's
> parent -- but due to the nature of DWARF, this requires a more
> full scan of the DIE tree (there are no parent links in the
> .debug_info itself)
That's indeed an issue. Still even here scanning for the parent
chain up to the CU header shouldn't be too expensive if you do not
materialize all other objects (maybe avoid redundant work by having
'placeholder' DIEs read-in with just TAG and sibling info, not populating
any of its actual content to avoid reading other DIEs). mmap vs. read
might be another thing to consider here (I also wonder if the actual
DWARF might be good enough of a data format to work with for most parts
of a DIE to avoid duplicating/exploding this already large data in memory).
> Longer term I would like to have gdb make a symtab directly
> from the index, and then lazily instantiate symbol contents
> when needed. I think this would be much faster.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (18 preceding siblings ...)
2022-06-20 6:59 ` rguenth at gcc dot gnu.org
@ 2023-01-18 13:47 ` fabian@ritter-vogt.de
2023-01-19 2:25 ` sam at gentoo dot org
20 siblings, 0 replies; 21+ messages in thread
From: fabian@ritter-vogt.de @ 2023-01-18 13:47 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
Fabian Vogt <fabian@ritter-vogt.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |fabian@ritter-vogt.de
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread* [Bug gdb/23710] gdb is slow and memory hungry consuming debug generated with LTO by GCC
[not found] <bug-23710-4717@http.sourceware.org/bugzilla/>
` (19 preceding siblings ...)
2023-01-18 13:47 ` fabian@ritter-vogt.de
@ 2023-01-19 2:25 ` sam at gentoo dot org
20 siblings, 0 replies; 21+ messages in thread
From: sam at gentoo dot org @ 2023-01-19 2:25 UTC (permalink / raw)
To: gdb-prs
https://sourceware.org/bugzilla/show_bug.cgi?id=23710
Sam James <sam at gentoo dot org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |sam at gentoo dot org
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 21+ messages in thread