* [Bug debuginfod/29478] New: Slow query of a debuginfo in a big rpm with very many files
@ 2022-08-12 8:58 mliska at suse dot cz
2022-08-12 11:23 ` [Bug debuginfod/29478] " fche at redhat dot com
2022-08-12 12:18 ` mliska at suse dot cz
0 siblings, 2 replies; 3+ messages in thread
From: mliska at suse dot cz @ 2022-08-12 8:58 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29478
Bug ID: 29478
Summary: Slow query of a debuginfo in a big rpm with very many
files
Product: elfutils
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: debuginfod
Assignee: unassigned at sourceware dot org
Reporter: mliska at suse dot cz
CC: elfutils-devel at sourceware dot org
Target Milestone: ---
Consider a very common test case where you debug a binary and the binary
depends on linux-vdso.so.1, well, actually each Linux binary depends on it.
$ file /usr/lib/modules/5.19.0-1-default/vdso/vdso64.so
/usr/lib/modules/5.19.0-1-default/vdso/vdso64.so: ELF 64-bit LSB shared object,
x86-64, version 1 (SYSV), static-pie linked,
BuildID[sha1]=6c5e8eefd4b928a12d9eec149ba8eab49b3febcc, stripped
it belongs to the following debuginfo package:
$ du -hs ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm
1.6G /home/marxin/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm
and we (openSUSE) use bzip2 compression for it (mainly due to compatibility
with older system where one can install a recent kernel).
Anyway it contains quite many files:
$ rpm -qpl ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm | wc -l
15906
and it occupies 5.5 GB when extracted.
$ DEBUGINFOD_VERBOSE=1 gdb `which htop`
GNU gdb (GDB; openSUSE Tumbleweed) 12.1
...
url 0
https://debuginfod.opensuse.org/buildid/6c5e8eefd4b928a12d9eec149ba8eab49b3febcc/debuginfo
query 1 urls in parallel
server response HTTP response code said error
url 0 The requested URL returned error: 504
not found No such file or directory (err=-2)
So as seen, there's a timeout of the request. Apparently, libarchive
uncompresses the content of each entry:
perf confirms that:
23.14% libbz2.so.1.0.6 [.] BZ2_bzDecompress
4.86% libbz2.so.1.0.6 [.] 0x0000000000015f62
2.21% [kernel] [k] virtnet_poll
1.86% [kernel] [k] __raw_callee_save___pv_queued_spin_unlock
and I can reproduce it locally with:
$ cat /tmp/demo.c
#include <archive.h>
#include <archive_entry.h>
#include <stdlib.h>
int main()
{
struct archive *a;
struct archive_entry *entry;
int r;
a = archive_read_new();
archive_read_support_filter_all(a);
archive_read_support_format_all(a);
r = archive_read_open_filename(a,
"/home/marxin/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm", 10240);
// Note 1
if (r != ARCHIVE_OK)
exit(1);
while (archive_read_next_header(a, &entry) == ARCHIVE_OK) {
printf("%s\n",archive_entry_pathname(entry));
archive_read_data_skip(a); // Note 2
}
r = archive_read_free(a); // Note 3
if (r != ARCHIVE_OK)
exit(1);
}
Note rpm -qpl finishes very quickly:
$ time rpm -qpl ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm
>/dev/null
real 0m0.064s
user 0m0.060s
sys 0m0.004s
If I see correctly, the function handle_buildid_r_match skips all entries
different from b_source1, but it's really slow
due to mentioned libarchive slownes.
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 3+ messages in thread
* [Bug debuginfod/29478] Slow query of a debuginfo in a big rpm with very many files
2022-08-12 8:58 [Bug debuginfod/29478] New: Slow query of a debuginfo in a big rpm with very many files mliska at suse dot cz
@ 2022-08-12 11:23 ` fche at redhat dot com
2022-08-12 12:18 ` mliska at suse dot cz
1 sibling, 0 replies; 3+ messages in thread
From: fche at redhat dot com @ 2022-08-12 11:23 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29478
Frank Ch. Eigler <fche at redhat dot com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |NOTABUG
--- Comment #1 from Frank Ch. Eigler <fche at redhat dot com> ---
Unfortunately, we're not aware of an easy way around this. There is no way to
seek in the compressed files to an arbitrary index, so one is stuck reading
from the beginning to the file of interest.
I think that, unless the kernel packaging changes, one needs to rely on
(trigger!) debuginfod to fetch & cache highly likely files such as the vdso*.
Some work is underway to allow this automation, but until then, make sure your
debuginfod server has a large enough fdcache, and you don't groom too
frequently (which clears out the fdcache).
See also https://bugzilla.redhat.com/show_bug.cgi?id=1970578
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 3+ messages in thread
* [Bug debuginfod/29478] Slow query of a debuginfo in a big rpm with very many files
2022-08-12 8:58 [Bug debuginfod/29478] New: Slow query of a debuginfo in a big rpm with very many files mliska at suse dot cz
2022-08-12 11:23 ` [Bug debuginfod/29478] " fche at redhat dot com
@ 2022-08-12 12:18 ` mliska at suse dot cz
1 sibling, 0 replies; 3+ messages in thread
From: mliska at suse dot cz @ 2022-08-12 12:18 UTC (permalink / raw)
To: elfutils-devel
https://sourceware.org/bugzilla/show_bug.cgi?id=29478
--- Comment #2 from Martin Liska <mliska at suse dot cz> ---
Thanks for the reply, after reading of the RH bug, I can see you faced the very
same problem as me ;)
--
You are receiving this mail because:
You are on the CC list for the bug.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-08-12 12:18 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-12 8:58 [Bug debuginfod/29478] New: Slow query of a debuginfo in a big rpm with very many files mliska at suse dot cz
2022-08-12 11:23 ` [Bug debuginfod/29478] " fche at redhat dot com
2022-08-12 12:18 ` mliska at suse dot cz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).