From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 5264E3858C52 for ; Tue, 13 Feb 2024 13:28:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5264E3858C52 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5264E3858C52 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=45.83.234.184 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707830888; cv=none; b=gV5gZLZdqVl1dwSaptPA3dwZo8JeGZ9O0r/VU7WEVMpx3r5VPC3UAQo982LAzII/av1oI0YojNZ4czs9QQK1lNO52lT8IBhJSz5vWXHsMDJkvB5MSusFvBHQRqSJ49yUznIhZHEfKPfREWU3eprBGa/aOBiN4xRnTQqk3BAFW9I= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1707830888; c=relaxed/simple; bh=TW3uwSWE6phaT8GRlFl9Epd9hqIrTwzcXEvmIfAujr8=; h=Message-ID:Subject:From:To:Date:MIME-Version; b=HAYb7Bb2Z+TcAg0Bs66E2QMnn3JqM0V7PB1Flom8mPjaYp7RQZD72FxHz7Vgx3wKsAOlmshQMl4hnPnQTHvTWt9x4Phlc80jCU6u6YYdnQRvdzSu7Cu2HAW8yANBgcro5auLE95Rz7IV0dyx8QeLFBS9pjlFBCX6cP7gfGJJMx0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from r6.localdomain (82-217-174-174.cable.dynamic.v4.ziggo.nl [82.217.174.174]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 147D130005AA; Tue, 13 Feb 2024 14:28:05 +0100 (CET) Received: by r6.localdomain (Postfix, from userid 1000) id 55ED63402EC; Tue, 13 Feb 2024 14:28:04 +0100 (CET) Message-ID: Subject: Re: [PATCH v2] dwarf_getaranges: Build aranges list from CUs instead of .debug_aranges From: Mark Wielaard To: Aaron Merey , elfutils-devel@sourceware.org Date: Tue, 13 Feb 2024 14:28:04 +0100 In-Reply-To: <20231211231853.116254-1-amerey@redhat.com> References: <20231207013504.40300-3-amerey@redhat.com> <20231211231853.116254-1-amerey@redhat.com> Autocrypt: addr=mark@klomp.org; prefer-encrypt=mutual; keydata=mQINBFxDPtIBEAC8xePaWvq5cDMBYtrpdmR20YX5xrDXUeHgueSVE9Mw8yCan2Cq1Ac1jHYnXxp4Jj3q4tIS9Jq2oAbqxyvBMdJYqEz4z709eDnYBacZQcGqojLh3HI2P7lmmKxkL8rS3Q3Ug05mYT+MwgmRvIO1+kwQTcq1AeB9z9/zikgY1Jv1R86gH8G84OPbJUowdE/mT3pQ+2UQnarCVPJ3WJtZNDXPRzskk0t5h+Mg5RtX+COoHfsvWHiEUTmHMfynw49GG/YF6jLSVzMlKMz3jdOePIArpm2BNUu8DvEn9at6daKR4Ah+ujDd08l9j8wFhJnenn/9+ENjm9kOGQWOmH/fEIOlMAATVdZEfHVfAWbgICPSrPyi+v3ACE4uEoaw85LgbAAbhzdswlLezLxS7LLTpfDZUFZfkho1MSGXaCQ475/iVAeuxi61B2VcmH4lOSH7HYNkMY8ggGk2/WG35eq8PZme8PvXUmLu+f2jzy9XFekIr+/Ks2TchCTYCw6bhSLPa19HapCxvWXgNcYzZ8jULqBXsAfj4NnzBTn6u5nsJ1reA8GoO9vLaQf3LDgr+UY/z+6N474lAVfr0eIdWzkWPvX8fcBCfiB944rPr+Q50RUfc9ngIKP4JsflhXTn601aL4r6qkCcBVqO/eRhb4vCAUo7csemTKzI/05ooGfQtZ3O5QARAQABtB5NYXJrIFdpZWxhYXJkIDxtYXJrQGtsb21wLm9yZz6JAlEEEwEKADsCGwEFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXE37mQIZAQAKCRAapEvmSd52CuO9D/9Fi6LOrU+iYHjfIk+wT8jyhF1YNATnooa5W7y/4QlXOIyKmkXM/0faH1hZNGf4qVK4dBUewuhALMEzudkXEhzudg9KpB9SaHZNR5DZ+YHo204zJ84P+aJa7F8FOScbNAiG4 pFGC7sQxvtAz0skM0yLsdhNg2tM8lM3n9e/rO4EK7aR55ojzE9pCWhRSx/AKYT7545KzXuCRTky8fRcI8YeNNLPIseoV3QPkf7qNi6YXl0yUHV5gQMCWqgbfdHAljd2+N1RZvdzfEOLVPLX4/dgxb36i9OKkuCAHLoL2UXfzcAElltHQundNi/xYzSizzEYBeIbVrbuqiJP1zmiPUKxHibkU3ThZZUbonKRNVPQe1hO47Cxyj1RxXl6Nt9uda3W9ow6Kr96Bjs3WVBSqsuohqaAlAxC6RccslrEw/7N7l8S423LJI6ZV+FvyJzmSAqkLNz/tuFSMj76uH4s1dLbRv8K4fcw1vZgqy/4jIhBFycn29hMNvImKbMnLDwC7K92lBGQ6hp75/0Hf1qHOpDaiyV9Qqzr3sTOMXJiYm3ac5bDqJb9Mi5YPNB2OD3w3bDMGT5+eWjmw9RiYT5gNjY6nZhDiQS/PtIc6l3i2GaBjSdurwj47TlCWALj3ZiiEKiybanK5/YXdVXGchLnoNkxeI1YcANZhX60FWEyaHZsa7QbTWFyayBXaWVsYWFyZCA8bWp3QGdudS5vcmc+iQJOBBMBCgA4FiEE7Dz+iPbKB4h3T1wdGqRL5knedgoFAlxN+lMCGwEFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQGqRL5knedgo1bhAArI7kReYq4YtaxS8Pxb5MdPxiQVtvfkbycWCZ4owzPeEIkJqcbadNUiGSqCRR2xeT4kuzFZWILiZfBTwHwFM/bXRDK/FOn7F8aqUAV1tq2W70Z7BUpTwpAv7Xm5YvsfbTBZmllJltEiIrKIzULCtRKKVXgtOKg0sd/W2aXwyl+OX+PVzu4mXXNEkO10J7VpnCvjyaJNeKgeJYQLizSWdEf7i6RX31yC29+GsSqikaOHdfxJMM+bo/x/aCuYlgDB+OQ6LZzpXZO0C8B5SMgMfZaK1rxDtUtViajSyOFJ4Ig6 bcgc5qDCLnk407oEN1yBWps867uN/Bi4Dk+xh691feGsyq95DvPis2Ut+0X0/Wi/uLg3uu/X5EcNHynwht7KaGCLeuOZKxvzfeudNeyKFX34HtFyE/2k9LR0mFX8XnXQGBD9psOxcd2K8Rku9BjjKDZ/vf53sMh5vxUNo+zkd+5dLZWPnLrhkfQrepDBP+Tc/6W0VSZCP5/nKX6GjPwmELtZj4jGf33tgfNMJrmxGUjpDxtiJc7OroNC4he3F5AF4RNRa5VvHs6ah57swVvKyJmLH5mxxKIn39PspOhzVxSbkWNPLS+km2InPum+fmYKQL6IrHcqt/ecrR7o9GRgI0cJjLJ+wv93ti+gxsUWPbAUBaJPk24omIpQafFT/YAEW0Hk1hcmsgV2llbGFhcmQgPG1qd0ByZWRoYXQuY29tPokCTgQTAQoAOBYhBOw8/oj2ygeId09cHRqkS+ZJ3nYKBQJcTfqnAhsBBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEBqkS+ZJ3nYK6JIP/jGq2adAkk1IPsVx7X5argCXaI5o3aTG+N6BPMvizGY4CyqTBmYAg3qRPWN4U+wxNFgzQ3nPfptJK6EDBAY4Lw+kKR4o8o5zPyjAjbc9/be5bvTwMFmMbzWxGWoJ4ch9W+wD4W+W8ssnJDbVi5Btp7kKyUgg3KP7oUQpxZ9MTgmKEmPuce9zOQ78q+LIPbkhI9zCS/1VCHkeVyn+TrnACoHx7sKgJoOwjVlcbW3S0sdCuaGg3+VLI3v3IpQ56UXIX6RVeLX9zVDZnAq72Age4HHcxjX97j16iL5ZhZRc24J5tpSkJgHt+RASOKHJGPIivCqKbQKhYc1G4MbFbGzclaLTXya3Q4ekFzo4ohd2ga7mgjG0SJ+aIU5wCYxEUDsqJLm+uH/nbQzXTxIoQhH5yub4OhW88z6LpwPGhLvzS2SuPJIXAlbGyrH70/uRZWkwKF/8mQjMCsLEYkZ 9DRB815rUTc9CJkkeOlPXQUbxr2fDAgi0j3yAUxlrC7jESO/zUJrICbxChYAx9KMWG/2PsKbrGAAMKiC7+q6mY09Q63F/g1DEF2sb+bguMdWc7SEj64jFUf9wJ+vjU1F321Bkh/QWMABv6n+7EFkwnNkylCR5H1boOHO03TNT0jyLbBECR7/Mtpwt46c4+n9EPCmQyvdU3MVPORvZge1hzvuvfo22uQENBFxDuhkBCAC19Q021v7kTuwYKwEmbqQC5wvmbEMT4ldvQ8gWCUIFL9kTxM67IF0annsys+rrAyqqFUTq2onVmgjciu9upl6uDdV3wivCBEDN9ZLZAVHTSviiXDhnHUSg6EhCdZKhal9DKAi+vGSLSe14e2Kfoe4c6R0yDVI+Dn0OfUhlMXu2NoDSFLAdHsDHSCrE6xKO+BNgL2MPuMeXLhNitNIVrykoZMkFrUMcMsHrvrk05ah87RQO1e2ljenn8qxPRLdOVWc0TJiosjiy04vwDAYNUCPDL5W2Mp2bv2AeTPCzF1qkDnGKZEqV2peWKCPB608lS1icw5oKtOl50PSgzTdaLVRXABEBAAGJAjYEGAEKACAWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXEO6GQIbDAAKCRAapEvmSd52Cpy8D/9tq4BQ3VwrDNCxycALqWvZSPv/AgsT6hRvQsLc6Yp0FEtz+frFPLWt7bylMrzKItpsr0G2FofWw0yNyHNYPmGlCi+SrWLJnUTEm5TZgwT+9kLt/mJ4B0J1gHkknXSo91S84DPaik9CH0GmXIQyPANkDDlmp9W/Hk8oKxxvCx+SSsZ6ANXakcNVg/w4MhDW2HowW4sBvtltOFSgPRs9zISiNw//GYjeYrdOOnieMhszwpjQuK5XYnDhwiSap2D8nQlD/VpAa2CvE/fOFV2CJyKZfE0J8v5DZOU+SUwnty1f52ZA1s/OCysaK1LLdCXz3bQiybQZhobcAneBVZFl Nzf6xpR+pGtw3OVSyLQo4LSQf4lFszNy8FfE+BJ1/yUWFBjljLwIHd4IW7Y17PugAc19fQ23krOIc3O4qsuYzqdhzYzqGbPvf7fY3Tz0BNcW5885KEQJH7VJJLqpf3EELhmkLBONYiF10iggFSmn8WSQWbXm0kGRETvAzf+FYcJsKDu9QASDRNck8J20ZJGVLbZNdP+VuLOXCDAkSGIxi91TLi6bY0Mb2yNRgAq6cnIJUTAbcnw05BLxRW+e8AS3HodjZHWzAMDPpZn5TFfJOXdDhdeePVGgkypxwnbeyTT3OjUEh37vr+XIgrTMpz+ZNpHxLr4bJatQEVK3H6Q3ZbQkMbkBDQRcQ7q3AQgAqSM4Wx4QvvCIf8is+57mLJhceB2kLt3VR67UFZC0ywcr5V0pvTuu2U1oUB+BVYC/A9UdnvWTyDef3xTCx0hAiiFhlMe6CkODOalmxI+KwPxD276+70tcxd8vR2FJviDQKw96f2hlLAnxR47GUp3cPfIgVfkvIXnXLMUJQvBhXeXqgYhOcAplI677n/zTeFjBtd/JqtyDoJ0De1odEyC+ZZD/Jo5q80Sydhvb99BHQMgLTJTJPW1iRV2AK4xfBjxOMwqml9Lx4HRIpV/IHs3MTyhEpEA+I/eKpO6UxApHWHZ76Zm8BL8RwnfFaXjMueRhIGMFtJnLuNFc5mOLXa3uhwARAQABiQNsBBgBCgAgFiEE7Dz+iPbKB4h3T1wdGqRL5knedgoFAlxDurcCGwIBQAkQGqRL5knedgrAdCAEGQEKAB0WIQQSdoqWeVmQEHoNL9/8V+PMrNmaeAUCXEO6twAKCRD8V+PMrNmaeEvuB/92qMj2mQN3CXRQUTlmzVNUJLJAwzjRDoSt3kqDrACJ2N8JLSxWFeHmEmrrmFPUmXfBUkT+F2W+OrsJlUtwepuTYROgLNZebFQdjB38oqsj8RMKb5ikWntRUka2xhSDRBa0IlpxHBWLHS8nEx1x4 HB4uYRK3IpWShAVmWk7jiATGJLFYJGVo4TBfM27zCty8/GQN/3A2DAJ2OJbiJ12ByTgzztGdhJ69H/QUltkK7eJUGMjPwhpmp07lrolyUurbzaLMQow4SLo/ZIxa0nPC+AoMSk06teichCZwIyiU/70S0c/uL3RFhnTbgWcdQkAVpWdkwFqIES4xG5QLUu85/WT7lMQALJKKuOOpbOeKvyLV16Oo70OTms/LbmXU9+bjCjz7QISuzhI4rua0onjQzBaRXFYkfCjBudWaEpy/wP5wk6QlqxLkeZNCk0TswksLxQjyO2XgBcOnrSsQIEJ7VICG9PDvtVzbrSBYMjoDo58AyniEMVANyUnFYl1YBlFt506PDh86ZEqlpbbReAsYfEuBQdBfJhWph9WZgJDVtEHUAxaiqisvNEbz4xRIAsxX/OxnQMdD09Xs50yvl38ERIadacejtQnAIYeEaUBsgQk3rt0+g9lm6trD7P4FXYhUD9vml6/n8TGB3UJi3lKpX41GSUC1y+oPna8p+EEmrm3BbB4fgnIkfYiEDNogvm2pe7nzUP7sNnE8RcyYcjUoEQ0Uo+HB6fk6NeBGKqaIKVexCcExnWKHvl0DZzGydvKx41nyzFI1sueg34LcWwpGHXzJyhmpjhNe1GOKtVGHCGKhKhppK4ntUZISciGh38wvKuFDohHO3JVZ9AhyRWKTuynzLarBpmvu11TDbv0lfnZcghlWWHNlx8x8DdaEuFWXZTDuVXqGclmeV2hS0LomX33LCB4n0XkZtC9LsmTIsr+ZdVCAXUeX/pJONNxNF8G47lZLLgLWF9beuHWp3u1Io31fzh44TZxm1Z31wCZjOrsL9bvy3xHyDFaDL+/7i6TXsSxtqTXuQENBFxDu6IBCACgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65 nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNzABEBAAGJAjYEGAEKACAWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXEO7ogIbIAAKCRAapEvmSd52Ch8ZD/9wKuIlaRMSB1AMCwhGPaqXZahrJ649Y0jI4JqpFKv2/U5hKHOG7bihRAeEj7pZzhlgBrkZg1SBdZ3vHs1ufElnfe5RQApdDm93daU5SP29iEivJQxKjF91EfEffl0trxxztBipI5/2D+kaS8cnNVfzo5ZEWy/cd6AShvRVHM7Y2QHc+mlaZhYhBvTtwC6avXNnG55WYgobGENeAwkyD072JF3XrxFb+XkcKxla9yRdWdHxJd5PYJqsKM+nVeJM226OwOyU235gfIhIP6pfGqF9UVH0uFoCYkVkUSjVd96Q+Cj0kdhTOrtLW1OY11d9TBxje42GOtc7X9Zzx1nhwU8rCCErF9/uJIJKlq7I08rMX3rFDTtizwN7g7ZBkDDiZO+BIKQPt/awA9NM+tda02hyfQokBBi+v8b/iKifKIfUaqPDo1PA5uxljdluyX8AXIotKjJXF6Elsiz7bVpcIc0ZXOOFr9ylmtZm51YNmOzDNznEBmol2oBZfsk2G55/QgShHmKUnvzKANBGfnfS/a/K7Hv4sfZAb58Prl6OmQSrkmhzFry/4BNLKq+nd4s8VXkJPpx3Ogf3DoIynqpNF0bwf52U5IgJSNcJN/HrAwhaG1W+Y3LDe7S19M0cUzftEUeq3Jd89hoijC72tdba+BRfW0ncfvEcsk9 QifSU1tvZxQ== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.3 (3.50.3-1.fc39) MIME-Version: 1.0 X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,RCVD_IN_BARRACUDACENTRAL,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi Aaron, On Mon, 2023-12-11 at 18:18 -0500, Aaron Merey wrote: > No longer use .debug_aranges to build the aranges list since it could be > absent or incomplete. >=20 > Instead build the aranges list by iterating over each CU and recording > each address range. >=20 > https://sourceware.org/bugzilla/show_bug.cgi?id=3D22288 > https://sourceware.org/bugzilla/show_bug.cgi?id=3D30948 >=20 > Signed-off-by: Aaron Merey > --- >=20 > v2 adds a test for generating aranges from a binary with no > .debug_aranges. >=20 > This patch's method of building the aranges list is slower than simply > reading .debug_aranges. On my machine, running eu-stack on a 2.9G > firefox core file takes about 8.7 seconds with this patch applied, > compared to about 3.3 seconds without this patch. That is significant. 2.5 times slower. Did you check with perf or some other profiler where exactly the extra time goes. Does the new method find more aranges (and so produces "better" backtraces)? > Ideally we could assume that .debug_aranges is complete if it is present > and build the aranges list via CU iteration only when .debug_aranges > is absent. This would let us save time on gcc-compiled binaries, which > include complete .debug_aranges by default. Right. This why the question is if the firefox case sees more/less aranges. If I remember correctly it is build with gcc and rustc, and rustc might not produce .debug_aranges. > However the DWARF spec appears to permit partially complete > .debug_aranges [1]. We could improve performance by starting with a > potentially incomplete list built from .debug_aranges. If a lookup > fails then search the CUs for missing aranges and add to the list > when found. >=20 > This approach would complicate the dwarf_get_aranges interface. The > list it initially provides could no longer be assumed to be complete. > The number of elements in the list could change during calls to > dwarf_getarange{info, _addr}. This would invalidate the naranges value > set by dwarf_getaranges. The current API doesn't include a way to > communicate to the caller when narages changes and by how much. >=20 > Due to these complications I think it's better to simply ignore > .debug_aranges altogether and build the aranges table via CU iteration, > as is done in this patch. Might it be an idea to leave dwarf_getaranges as it is and introduce a new (internal) function to get "dynamic" ranges? It looks like what programs (like eu-stack and eu-addr2line) really use is dwarf_addrdie and dwfl_module_addrdie. These are currently build on dwarf_getaranges, but could maybe use a new interface? > [1] https://sourceware.org/bugzilla/show_bug.cgi?id=3D22288#c5 So this comment says that "parsing CUs lightly (just enough to get their CU ranges) should be fairly cheap", but it seems that isn't really true. Or at least parsing .debug_aranges is a lot (2.5 times) faster (measured in seconds). It would be good to better understand why this is. Cheers, Mark