From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id A041D3858D1E for ; Tue, 20 Jun 2023 13:15:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A041D3858D1E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from r6.localdomain (82-217-174-174.cable.dynamic.v4.ziggo.nl [82.217.174.174]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 48343313AC85; Tue, 20 Jun 2023 15:15:14 +0200 (CEST) Received: by r6.localdomain (Postfix, from userid 1000) id B03BF3402E6; Tue, 20 Jun 2023 15:15:13 +0200 (CEST) Message-ID: <14d019389243b2b3e9d21392ef9f91a42ccfbe14.camel@klomp.org> Subject: Re: Performance issue with systemd-coredump and container process linking 2000 shared libraries. From: Mark Wielaard To: Romain GEISSLER Cc: "elfutils-devel@sourceware.org" Date: Tue, 20 Jun 2023 15:15:13 +0200 In-Reply-To: References: <20230614133920.10-1-romain.geissler@amadeus.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.48.3 (3.48.3-1.fc38) MIME-Version: 1.0 X-Spam-Status: No, score=-3028.1 required=5.0 tests=BAYES_00,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,RCVD_IN_BARRACUDACENTRAL,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi Romain, On Mon, 2023-06-19 at 19:56 +0000, Romain GEISSLER via Elfutils-devel wrote: >=20 > Thanks ! And sorry that Laurent had pinged you directly on Slack, I > wanted to reach you via this mailing list instead of through the Red > Hat customer network ;) Slack isn't a very effective way to reach me. Most elfutils hackers do hang out on the Libera.Chat irc channel #elfutils. > I don=E2=80=99t know if you read the Red Hat case too. There you can find > things a bit more clarified, and splitted into what I think are potential= ly > 3 distinct "problems" which 3 distinct possible fix. Since there is nothi= ng > private, I can write on this here as well on this public mailing list. I haven't looked if I have access to the customer case since you provided such a great reproducer. > So in the end I see 3 points (in addition to not understanding why > finding the elf header returns NULL while it should not and which I > guess you are currently looking at): > =C2=A0- the idea that systemd developers should invert their logic: first > try to parse elf/program headers from the (maybe partial) core dump > PT_LOAD program headers yes, that could in theory also be done through a custom callbacks- >find_elf. > =C2=A0- This special "if" condition that I have added in the original sys= temd > code: >=20 > + /* This PT_LOAD section doesn't contain the start addres= s, so it can't be the module we are looking for. */ > + if (start < program_header->p_vaddr || start >=3D progra= m_header->p_vaddr + program_header->p_memsz) > + continue; >=20 > to be added near this line: https://github.com/systemd/systemd/blob/72e7b= fe02d7814fff15602726c7218b389324159/src/shared/elf-util.c#L540 >=20 > on which I would like to ask you if indeed it seems like a "right" fix wi= th > your knowledge of how core dump and elf files are shaped. Yes, that does make sense. > =C2=A0- The idea that maybe this commit https://sourceware.org/git/?p=3De= lfutils.git;a=3Dcommitdiff;h=3D8db849976f07046d27b4217e9ebd08d5623acc4f > which assumed that normally the order of magnitude of program headers > is 10 for a "normal" elf file, so a linked list would be enough might be > wrong in the special case of core dump which may have much more > program headers. And if indeed it makes sense to elf_getdata_rawchunk > for each and every program header of a core, in that case should this > linked list be changed into some set/hashmap indexed by start > address/size ? Interesting. Yeah, a linked list is not the ideal datastructure here. But I don't think it is causing the really long delay. That clearly comes from the (negative) inode/dentry file search cache. But we could look at this after we solve the other issues and we maybe want to speed things up a bit more. >=20 Cheers, Mark