From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 92092385C33F; Fri, 12 Aug 2022 08:58:10 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 92092385C33F From: "mliska at suse dot cz" To: elfutils-devel@sourceware.org Subject: [Bug debuginfod/29478] New: Slow query of a debuginfo in a big rpm with very many files Date: Fri, 12 Aug 2022 08:58:10 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: elfutils X-Bugzilla-Component: debuginfod X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: mliska at suse dot cz X-Bugzilla-Status: NEW X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status bug_severity priority component assigned_to reporter cc target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Aug 2022 08:58:10 -0000 https://sourceware.org/bugzilla/show_bug.cgi?id=3D29478 Bug ID: 29478 Summary: Slow query of a debuginfo in a big rpm with very many files Product: elfutils Version: unspecified Status: NEW Severity: normal Priority: P2 Component: debuginfod Assignee: unassigned at sourceware dot org Reporter: mliska at suse dot cz CC: elfutils-devel at sourceware dot org Target Milestone: --- Consider a very common test case where you debug a binary and the binary depends on linux-vdso.so.1, well, actually each Linux binary depends on it. $ file /usr/lib/modules/5.19.0-1-default/vdso/vdso64.so /usr/lib/modules/5.19.0-1-default/vdso/vdso64.so: ELF 64-bit LSB shared obj= ect, x86-64, version 1 (SYSV), static-pie linked, BuildID[sha1]=3D6c5e8eefd4b928a12d9eec149ba8eab49b3febcc, stripped it belongs to the following debuginfo package: $ du -hs ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm=20 1.6G /home/marxin/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm and we (openSUSE) use bzip2 compression for it (mainly due to compatibility with older system where one can install a recent kernel). Anyway it contains quite many files: $ rpm -qpl ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm | wc -l 15906 and it occupies 5.5 GB when extracted. $ DEBUGINFOD_VERBOSE=3D1 gdb `which htop` GNU gdb (GDB; openSUSE Tumbleweed) 12.1 ... url 0 https://debuginfod.opensuse.org/buildid/6c5e8eefd4b928a12d9eec149ba8eab49b3= febcc/debuginfo query 1 urls in parallel server response HTTP response code said error url 0 The requested URL returned error: 504 not found No such file or directory (err=3D-2) So as seen, there's a timeout of the request. Apparently, libarchive uncompresses the content of each entry: perf confirms that: 23.14% libbz2.so.1.0.6 [.] BZ2_bzDecompress 4.86% libbz2.so.1.0.6 [.] 0x0000000000015f62 2.21% [kernel] [k] virtnet_poll 1.86% [kernel] [k] __raw_callee_save___pv_queued_spin_unlock and I can reproduce it locally with: $ cat /tmp/demo.c #include #include #include int main() { struct archive *a; struct archive_entry *entry; int r; a =3D archive_read_new(); archive_read_support_filter_all(a); archive_read_support_format_all(a); r =3D archive_read_open_filename(a, "/home/marxin/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm", 102= 40); // Note 1 if (r !=3D ARCHIVE_OK) exit(1); while (archive_read_next_header(a, &entry) =3D=3D ARCHIVE_OK) { printf("%s\n",archive_entry_pathname(entry)); archive_read_data_skip(a); // Note 2 } r =3D archive_read_free(a); // Note 3 if (r !=3D ARCHIVE_OK) exit(1); } Note rpm -qpl finishes very quickly: $ time rpm -qpl ~/BIG/test/kernel-default-debuginfo-5.19.0-1.1.x86_64.rpm >/dev/null real 0m0.064s user 0m0.060s sys 0m0.004s If I see correctly, the function handle_buildid_r_match skips all entries different from b_source1, but it's really slow due to mentioned libarchive slownes. --=20 You are receiving this mail because: You are on the CC list for the bug.=