From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id BECD53858C35 for ; Fri, 1 Mar 2024 14:59:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BECD53858C35 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org BECD53858C35 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=45.83.234.184 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709305169; cv=none; b=wTW8oub+HKQbdQn/JYV28I92Ob4ih0fQr2xFIooazosEU3QLEK5aMOvr7d5M1pYqf18G3mP36iTnlwZfhOZvYFK1rGj5e4cSOtEbhJNQH1ic10Gm1Kt332HexA42InguOssM0Lt6HXZGd0iX9a/FGqmmJ800HgnMFXl8GIKec/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709305169; c=relaxed/simple; bh=ji0FfJQQqLSE8xYdbYUnWrjYgb1+0rkozxICc1nKXxk=; h=Message-ID:Subject:From:To:Date:MIME-Version; b=XA5sx3fkUFGfvA+kxZYa986/c9ucLE7zyAopmEoAn8aFTf3q3T+Vt9ntwpf48h51QK831IctHr0YFiw0h237cBDA8h3hlE+3hwbiWi8WsWqyxZNd6SNlp4qE7xE3lEkWfEbeGVOzwhdGChRVM+44RkVmfCWCHAdlixVnRiiO/6s= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from r6.localdomain (82-217-174-174.cable.dynamic.v4.ziggo.nl [82.217.174.174]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id F3F77307C2BC; Fri, 1 Mar 2024 15:59:23 +0100 (CET) Received: by r6.localdomain (Postfix, from userid 1000) id DC7A43403B0; Fri, 1 Mar 2024 15:59:22 +0100 (CET) Message-ID: Subject: Re: [PATCH v3 4/4] libdw: Handle overflowed DW_SECT_INFO offsets in DWARF package file indexes From: Mark Wielaard To: Omar Sandoval , elfutils-devel@sourceware.org Date: Fri, 01 Mar 2024 15:59:22 +0100 In-Reply-To: References: Autocrypt: addr=mark@klomp.org; prefer-encrypt=mutual; keydata=mQINBFxDPtIBEAC8xePaWvq5cDMBYtrpdmR20YX5xrDXUeHgueSVE9Mw8yCan2Cq1Ac1jHYnXxp4Jj3q4tIS9Jq2oAbqxyvBMdJYqEz4z709eDnYBacZQcGqojLh3HI2P7lmmKxkL8rS3Q3Ug05mYT+MwgmRvIO1+kwQTcq1AeB9z9/zikgY1Jv1R86gH8G84OPbJUowdE/mT3pQ+2UQnarCVPJ3WJtZNDXPRzskk0t5h+Mg5RtX+COoHfsvWHiEUTmHMfynw49GG/YF6jLSVzMlKMz3jdOePIArpm2BNUu8DvEn9at6daKR4Ah+ujDd08l9j8wFhJnenn/9+ENjm9kOGQWOmH/fEIOlMAATVdZEfHVfAWbgICPSrPyi+v3ACE4uEoaw85LgbAAbhzdswlLezLxS7LLTpfDZUFZfkho1MSGXaCQ475/iVAeuxi61B2VcmH4lOSH7HYNkMY8ggGk2/WG35eq8PZme8PvXUmLu+f2jzy9XFekIr+/Ks2TchCTYCw6bhSLPa19HapCxvWXgNcYzZ8jULqBXsAfj4NnzBTn6u5nsJ1reA8GoO9vLaQf3LDgr+UY/z+6N474lAVfr0eIdWzkWPvX8fcBCfiB944rPr+Q50RUfc9ngIKP4JsflhXTn601aL4r6qkCcBVqO/eRhb4vCAUo7csemTKzI/05ooGfQtZ3O5QARAQABtB5NYXJrIFdpZWxhYXJkIDxtYXJrQGtsb21wLm9yZz6JAlEEEwEKADsCGwEFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXE37mQIZAQAKCRAapEvmSd52CuO9D/9Fi6LOrU+iYHjfIk+wT8jyhF1YNATnooa5W7y/4QlXOIyKmkXM/0faH1hZNGf4qVK4dBUewuhALMEzudkXEhzudg9KpB9SaHZNR5DZ+YHo204zJ84P+aJa7F8FOScbNAiG4 pFGC7sQxvtAz0skM0yLsdhNg2tM8lM3n9e/rO4EK7aR55ojzE9pCWhRSx/AKYT7545KzXuCRTky8fRcI8YeNNLPIseoV3QPkf7qNi6YXl0yUHV5gQMCWqgbfdHAljd2+N1RZvdzfEOLVPLX4/dgxb36i9OKkuCAHLoL2UXfzcAElltHQundNi/xYzSizzEYBeIbVrbuqiJP1zmiPUKxHibkU3ThZZUbonKRNVPQe1hO47Cxyj1RxXl6Nt9uda3W9ow6Kr96Bjs3WVBSqsuohqaAlAxC6RccslrEw/7N7l8S423LJI6ZV+FvyJzmSAqkLNz/tuFSMj76uH4s1dLbRv8K4fcw1vZgqy/4jIhBFycn29hMNvImKbMnLDwC7K92lBGQ6hp75/0Hf1qHOpDaiyV9Qqzr3sTOMXJiYm3ac5bDqJb9Mi5YPNB2OD3w3bDMGT5+eWjmw9RiYT5gNjY6nZhDiQS/PtIc6l3i2GaBjSdurwj47TlCWALj3ZiiEKiybanK5/YXdVXGchLnoNkxeI1YcANZhX60FWEyaHZsa7QbTWFyayBXaWVsYWFyZCA8bWp3QGdudS5vcmc+iQJOBBMBCgA4FiEE7Dz+iPbKB4h3T1wdGqRL5knedgoFAlxN+lMCGwEFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQGqRL5knedgo1bhAArI7kReYq4YtaxS8Pxb5MdPxiQVtvfkbycWCZ4owzPeEIkJqcbadNUiGSqCRR2xeT4kuzFZWILiZfBTwHwFM/bXRDK/FOn7F8aqUAV1tq2W70Z7BUpTwpAv7Xm5YvsfbTBZmllJltEiIrKIzULCtRKKVXgtOKg0sd/W2aXwyl+OX+PVzu4mXXNEkO10J7VpnCvjyaJNeKgeJYQLizSWdEf7i6RX31yC29+GsSqikaOHdfxJMM+bo/x/aCuYlgDB+OQ6LZzpXZO0C8B5SMgMfZaK1rxDtUtViajSyOFJ4Ig6 bcgc5qDCLnk407oEN1yBWps867uN/Bi4Dk+xh691feGsyq95DvPis2Ut+0X0/Wi/uLg3uu/X5EcNHynwht7KaGCLeuOZKxvzfeudNeyKFX34HtFyE/2k9LR0mFX8XnXQGBD9psOxcd2K8Rku9BjjKDZ/vf53sMh5vxUNo+zkd+5dLZWPnLrhkfQrepDBP+Tc/6W0VSZCP5/nKX6GjPwmELtZj4jGf33tgfNMJrmxGUjpDxtiJc7OroNC4he3F5AF4RNRa5VvHs6ah57swVvKyJmLH5mxxKIn39PspOhzVxSbkWNPLS+km2InPum+fmYKQL6IrHcqt/ecrR7o9GRgI0cJjLJ+wv93ti+gxsUWPbAUBaJPk24omIpQafFT/YAEW0Hk1hcmsgV2llbGFhcmQgPG1qd0ByZWRoYXQuY29tPokCTgQTAQoAOBYhBOw8/oj2ygeId09cHRqkS+ZJ3nYKBQJcTfqnAhsBBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEBqkS+ZJ3nYK6JIP/jGq2adAkk1IPsVx7X5argCXaI5o3aTG+N6BPMvizGY4CyqTBmYAg3qRPWN4U+wxNFgzQ3nPfptJK6EDBAY4Lw+kKR4o8o5zPyjAjbc9/be5bvTwMFmMbzWxGWoJ4ch9W+wD4W+W8ssnJDbVi5Btp7kKyUgg3KP7oUQpxZ9MTgmKEmPuce9zOQ78q+LIPbkhI9zCS/1VCHkeVyn+TrnACoHx7sKgJoOwjVlcbW3S0sdCuaGg3+VLI3v3IpQ56UXIX6RVeLX9zVDZnAq72Age4HHcxjX97j16iL5ZhZRc24J5tpSkJgHt+RASOKHJGPIivCqKbQKhYc1G4MbFbGzclaLTXya3Q4ekFzo4ohd2ga7mgjG0SJ+aIU5wCYxEUDsqJLm+uH/nbQzXTxIoQhH5yub4OhW88z6LpwPGhLvzS2SuPJIXAlbGyrH70/uRZWkwKF/8mQjMCsLEYkZ 9DRB815rUTc9CJkkeOlPXQUbxr2fDAgi0j3yAUxlrC7jESO/zUJrICbxChYAx9KMWG/2PsKbrGAAMKiC7+q6mY09Q63F/g1DEF2sb+bguMdWc7SEj64jFUf9wJ+vjU1F321Bkh/QWMABv6n+7EFkwnNkylCR5H1boOHO03TNT0jyLbBECR7/Mtpwt46c4+n9EPCmQyvdU3MVPORvZge1hzvuvfo22uQENBFxDuhkBCAC19Q021v7kTuwYKwEmbqQC5wvmbEMT4ldvQ8gWCUIFL9kTxM67IF0annsys+rrAyqqFUTq2onVmgjciu9upl6uDdV3wivCBEDN9ZLZAVHTSviiXDhnHUSg6EhCdZKhal9DKAi+vGSLSe14e2Kfoe4c6R0yDVI+Dn0OfUhlMXu2NoDSFLAdHsDHSCrE6xKO+BNgL2MPuMeXLhNitNIVrykoZMkFrUMcMsHrvrk05ah87RQO1e2ljenn8qxPRLdOVWc0TJiosjiy04vwDAYNUCPDL5W2Mp2bv2AeTPCzF1qkDnGKZEqV2peWKCPB608lS1icw5oKtOl50PSgzTdaLVRXABEBAAGJAjYEGAEKACAWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXEO6GQIbDAAKCRAapEvmSd52Cpy8D/9tq4BQ3VwrDNCxycALqWvZSPv/AgsT6hRvQsLc6Yp0FEtz+frFPLWt7bylMrzKItpsr0G2FofWw0yNyHNYPmGlCi+SrWLJnUTEm5TZgwT+9kLt/mJ4B0J1gHkknXSo91S84DPaik9CH0GmXIQyPANkDDlmp9W/Hk8oKxxvCx+SSsZ6ANXakcNVg/w4MhDW2HowW4sBvtltOFSgPRs9zISiNw//GYjeYrdOOnieMhszwpjQuK5XYnDhwiSap2D8nQlD/VpAa2CvE/fOFV2CJyKZfE0J8v5DZOU+SUwnty1f52ZA1s/OCysaK1LLdCXz3bQiybQZhobcAneBVZFl Nzf6xpR+pGtw3OVSyLQo4LSQf4lFszNy8FfE+BJ1/yUWFBjljLwIHd4IW7Y17PugAc19fQ23krOIc3O4qsuYzqdhzYzqGbPvf7fY3Tz0BNcW5885KEQJH7VJJLqpf3EELhmkLBONYiF10iggFSmn8WSQWbXm0kGRETvAzf+FYcJsKDu9QASDRNck8J20ZJGVLbZNdP+VuLOXCDAkSGIxi91TLi6bY0Mb2yNRgAq6cnIJUTAbcnw05BLxRW+e8AS3HodjZHWzAMDPpZn5TFfJOXdDhdeePVGgkypxwnbeyTT3OjUEh37vr+XIgrTMpz+ZNpHxLr4bJatQEVK3H6Q3ZbQkMbkBDQRcQ7q3AQgAqSM4Wx4QvvCIf8is+57mLJhceB2kLt3VR67UFZC0ywcr5V0pvTuu2U1oUB+BVYC/A9UdnvWTyDef3xTCx0hAiiFhlMe6CkODOalmxI+KwPxD276+70tcxd8vR2FJviDQKw96f2hlLAnxR47GUp3cPfIgVfkvIXnXLMUJQvBhXeXqgYhOcAplI677n/zTeFjBtd/JqtyDoJ0De1odEyC+ZZD/Jo5q80Sydhvb99BHQMgLTJTJPW1iRV2AK4xfBjxOMwqml9Lx4HRIpV/IHs3MTyhEpEA+I/eKpO6UxApHWHZ76Zm8BL8RwnfFaXjMueRhIGMFtJnLuNFc5mOLXa3uhwARAQABiQNsBBgBCgAgFiEE7Dz+iPbKB4h3T1wdGqRL5knedgoFAlxDurcCGwIBQAkQGqRL5knedgrAdCAEGQEKAB0WIQQSdoqWeVmQEHoNL9/8V+PMrNmaeAUCXEO6twAKCRD8V+PMrNmaeEvuB/92qMj2mQN3CXRQUTlmzVNUJLJAwzjRDoSt3kqDrACJ2N8JLSxWFeHmEmrrmFPUmXfBUkT+F2W+OrsJlUtwepuTYROgLNZebFQdjB38oqsj8RMKb5ikWntRUka2xhSDRBa0IlpxHBWLHS8nEx1x4 HB4uYRK3IpWShAVmWk7jiATGJLFYJGVo4TBfM27zCty8/GQN/3A2DAJ2OJbiJ12ByTgzztGdhJ69H/QUltkK7eJUGMjPwhpmp07lrolyUurbzaLMQow4SLo/ZIxa0nPC+AoMSk06teichCZwIyiU/70S0c/uL3RFhnTbgWcdQkAVpWdkwFqIES4xG5QLUu85/WT7lMQALJKKuOOpbOeKvyLV16Oo70OTms/LbmXU9+bjCjz7QISuzhI4rua0onjQzBaRXFYkfCjBudWaEpy/wP5wk6QlqxLkeZNCk0TswksLxQjyO2XgBcOnrSsQIEJ7VICG9PDvtVzbrSBYMjoDo58AyniEMVANyUnFYl1YBlFt506PDh86ZEqlpbbReAsYfEuBQdBfJhWph9WZgJDVtEHUAxaiqisvNEbz4xRIAsxX/OxnQMdD09Xs50yvl38ERIadacejtQnAIYeEaUBsgQk3rt0+g9lm6trD7P4FXYhUD9vml6/n8TGB3UJi3lKpX41GSUC1y+oPna8p+EEmrm3BbB4fgnIkfYiEDNogvm2pe7nzUP7sNnE8RcyYcjUoEQ0Uo+HB6fk6NeBGKqaIKVexCcExnWKHvl0DZzGydvKx41nyzFI1sueg34LcWwpGHXzJyhmpjhNe1GOKtVGHCGKhKhppK4ntUZISciGh38wvKuFDohHO3JVZ9AhyRWKTuynzLarBpmvu11TDbv0lfnZcghlWWHNlx8x8DdaEuFWXZTDuVXqGclmeV2hS0LomX33LCB4n0XkZtC9LsmTIsr+ZdVCAXUeX/pJONNxNF8G47lZLLgLWF9beuHWp3u1Io31fzh44TZxm1Z31wCZjOrsL9bvy3xHyDFaDL+/7i6TXsSxtqTXuQENBFxDu6IBCACgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65 nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNzABEBAAGJAjYEGAEKACAWIQTsPP6I9soHiHdPXB0apEvmSd52CgUCXEO7ogIbIAAKCRAapEvmSd52Ch8ZD/9wKuIlaRMSB1AMCwhGPaqXZahrJ649Y0jI4JqpFKv2/U5hKHOG7bihRAeEj7pZzhlgBrkZg1SBdZ3vHs1ufElnfe5RQApdDm93daU5SP29iEivJQxKjF91EfEffl0trxxztBipI5/2D+kaS8cnNVfzo5ZEWy/cd6AShvRVHM7Y2QHc+mlaZhYhBvTtwC6avXNnG55WYgobGENeAwkyD072JF3XrxFb+XkcKxla9yRdWdHxJd5PYJqsKM+nVeJM226OwOyU235gfIhIP6pfGqF9UVH0uFoCYkVkUSjVd96Q+Cj0kdhTOrtLW1OY11d9TBxje42GOtc7X9Zzx1nhwU8rCCErF9/uJIJKlq7I08rMX3rFDTtizwN7g7ZBkDDiZO+BIKQPt/awA9NM+tda02hyfQokBBi+v8b/iKifKIfUaqPDo1PA5uxljdluyX8AXIotKjJXF6Elsiz7bVpcIc0ZXOOFr9ylmtZm51YNmOzDNznEBmol2oBZfsk2G55/QgShHmKUnvzKANBGfnfS/a/K7Hv4sfZAb58Prl6OmQSrkmhzFry/4BNLKq+nd4s8VXkJPpx3Ogf3DoIynqpNF0bwf52U5IgJSNcJN/HrAwhaG1W+Y3LDe7S19M0cUzftEUeq3Jd89hoijC72tdba+BRfW0ncfvEcsk9 QifSU1tvZxQ== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.3 (3.50.3-1.fc39) MIME-Version: 1.0 X-Spam-Status: No, score=-8.0 required=5.0 tests=BAYES_00,GIT_PATCH_0,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,RCVD_IN_BARRACUDACENTRAL,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi Omar, On Mon, 2024-02-26 at 11:32 -0800, Omar Sandoval wrote: > Meta uses DWARF package files for our large, statically-linked C++ > applications. Some of our largest applications have more than 4GB in > .debug_info.dwo, but the section offsets in .debug_cu_index and > .debug_tu_index are 32 bits; see the discussion here [1]. We > implemented a workaround/extension for this in LLVM. Implement the > equivalent in libdw. >=20 > To test this, we need files with more than 4GB in .debug_info.dwo. I > created these artificially by editing GCC's assembly output. They > compress down to 6KB. I test them from run-large-elf-file.sh to take > advantage of the existing checks for large file support. >=20 > 1: https://discourse.llvm.org/t/dwarf-dwp-4gb-limit/63902. >=20 > * libdw/dwarf_end.c (dwarf_package_index_free): New function. > * tests/testfile-dwp-4-cu-index-overflow.bz2: New test file. > * tests/testfile-dwp-4-cu-index-overflow.dwp.bz2: New test file. > * tests/testfile-dwp-5-cu-index-overflow.bz2: New test file. > * tests/testfile-dwp-5-cu-index-overflow.dwp.bz2: New test file. > * tests/testfile-dwp-cu-index-overflow.source: New file. > * tests/run-large-elf-file.sh: Check > testfile-dwp-5-cu-index-overflow and > testfile-dwp-4-cu-index-overflow. The hack is kind of horrible, but given that this doesn't really impacts "normal" dwp files and it does work with clang/lldb, lets just support it too. > Signed-off-by: Omar Sandoval > --- > libdw/dwarf_cu_dwp_section_info.c | 147 ++++++++++++++- > libdw/dwarf_end.c | 15 +- > libdw/libdwP.h | 3 + > tests/Makefile.am | 7 +- > tests/run-large-elf-file.sh | 174 ++++++++++++++++++ > tests/testfile-dwp-4-cu-index-overflow.bz2 | Bin 0 -> 4490 bytes > .../testfile-dwp-4-cu-index-overflow.dwp.bz2 | Bin 0 -> 5584 bytes > tests/testfile-dwp-5-cu-index-overflow.bz2 | Bin 0 -> 4544 bytes > .../testfile-dwp-5-cu-index-overflow.dwp.bz2 | Bin 0 -> 5790 bytes > tests/testfile-dwp-cu-index-overflow.source | 86 +++++++++ > 10 files changed, 426 insertions(+), 6 deletions(-) > create mode 100755 tests/testfile-dwp-4-cu-index-overflow.bz2 > create mode 100644 tests/testfile-dwp-4-cu-index-overflow.dwp.bz2 > create mode 100755 tests/testfile-dwp-5-cu-index-overflow.bz2 > create mode 100644 tests/testfile-dwp-5-cu-index-overflow.dwp.bz2 > create mode 100644 tests/testfile-dwp-cu-index-overflow.source >=20 > diff --git a/libdw/dwarf_cu_dwp_section_info.c b/libdw/dwarf_cu_dwp_secti= on_info.c > index 298f36f9..3d11c87a 100644 > --- a/libdw/dwarf_cu_dwp_section_info.c > +++ b/libdw/dwarf_cu_dwp_section_info.c > @@ -30,6 +30,8 @@ > # include > #endif > =20 > +#include > + > #include "libdwP.h" > =20 > static Dwarf_Package_Index * > @@ -110,7 +112,9 @@ __libdw_read_package_index (Dwarf *dbg, bool tu) > =20 > index->dbg =3D dbg; > /* Set absent sections to UINT32_MAX. */ > - memset (index->sections, 0xff, sizeof (index->sections)); > + for (size_t i =3D 0; > + i < sizeof (index->sections) / sizeof (index->sections[0]); i++) > + index->sections[i] =3D UINT32_MAX; > for (size_t i =3D 0; i < section_count; i++) > { > uint32_t section =3D read_4ubyte_unaligned (dbg, sections + i * 4)= ; > @@ -161,6 +165,7 @@ __libdw_read_package_index (Dwarf *dbg, bool tu) > index->indices =3D indices; > index->section_offsets =3D section_offsets; > index->section_sizes =3D section_sizes; > + index->debug_info_offsets =3D NULL; > =20 > return index; > } > @@ -177,6 +182,137 @@ __libdw_package_index (Dwarf *dbg, bool tu) > if (index =3D=3D NULL) > return NULL; > =20 > + /* Offsets in the section offset table are 32-bit unsigned integers. = In > + practice, the .debug_info.dwo section for very large executables ca= n be > + larger than 4GB. GNU dwp as of binutils 2.41 and llvm-dwp before L= LVM 15 > + both accidentally truncate offsets larger than 4GB. > + > + LLVM 15 detects the overflow and errors out instead; see LLVM commi= t > + f8df8114715b ("[DWP][DWARF] Detect and error on debug info offset > + overflow"). However, lldb in LLVM 16 supports using dwp files with > + truncated offsets by recovering them directly from the unit headers= in the > + .debug_info.dwo section; see LLVM commit c0db06227721 ("[DWARFLibra= ry] Add > + support to re-construct cu-index"). Since LLVM 17, the overflow er= ror can > + be turned into a warning instead; see LLVM commit 53a483cee801 ("[D= WP] add > + overflow check for llvm-dwp tools if offset overflow"). > + > + LLVM's support for > 4GB offsets is effectively an extension to the= DWARF > + package file format, which we implement here. The strategy is to w= alk the > + unit headers in .debug_info.dwo in lockstep with the DW_SECT_INFO c= olumns > + in the section offset tables. As long as they are in the same orde= r > + (which they are in practice for both GNU dwp and llvm-dwp), we can > + correlate the truncated offset and produce a corrected array of off= sets. > + > + Note that this will be fixed properly in DWARF 6: > + https://dwarfstd.org/issues/220708.2.html. */ > + if (index->sections[DW_SECT_INFO - 1] !=3D UINT32_MAX > + && dbg->sectiondata[IDX_debug_info]->d_size > UINT32_MAX) > + { > + Dwarf_Package_Index *cu_index, *tu_index =3D NULL; > + if (tu) > + { > + tu_index =3D index; > + assert (dbg->cu_index =3D=3D NULL); > + cu_index =3D __libdw_read_package_index (dbg, false); > + if (cu_index =3D=3D NULL) > + { > + free(index); > + return NULL; > + } > + } > + else > + { > + cu_index =3D index; > + if (dbg->sectiondata[IDX_debug_tu_index] !=3D NULL > + && dbg->sectiondata[IDX_debug_types] =3D=3D NULL) > + { > + assert (dbg->tu_index =3D=3D NULL); > + tu_index =3D __libdw_read_package_index (dbg, true); > + if (tu_index =3D=3D NULL) > + { > + free(index); > + return NULL; > + } > + } > + } > + > + cu_index->debug_info_offsets =3D malloc (cu_index->unit_count > + * sizeof (Dwarf_Off)); > + if (cu_index->debug_info_offsets =3D=3D NULL) > + { > + free (tu_index); > + free (cu_index); > + __libdw_seterrno (DWARF_E_NOMEM); > + return NULL; > + } > + if (tu_index !=3D NULL) > + { > + tu_index->debug_info_offsets =3D malloc (tu_index->unit_count > + * sizeof (Dwarf_Off)); > + if (tu_index->debug_info_offsets =3D=3D NULL) > + { > + free (tu_index); > + free (cu_index->debug_info_offsets); > + free (cu_index); > + __libdw_seterrno (DWARF_E_NOMEM); > + return NULL; > + } > + } > + > + Dwarf_Off off =3D 0; > + uint32_t cui =3D 0, tui =3D 0; > + uint32_t cu_count =3D cu_index->unit_count; > + const unsigned char *cu_offset > + =3D cu_index->section_offsets + cu_index->sections[DW_SECT_INFO - 1] * = 4; > + uint32_t tu_count =3D 0; > + const unsigned char *tu_offset; > + if (tu_index !=3D NULL) > + { > + tu_count =3D tu_index->unit_count; > + tu_offset =3D tu_index->section_offsets > + + tu_index->sections[DW_SECT_INFO - 1] * 4; > + } > + while (cui < cu_count || tui < tu_count) > + { > + Dwarf_Off next_off; > + uint8_t unit_type; > + if (__libdw_next_unit (dbg, false, off, &next_off, NULL, NULL, > + &unit_type, NULL, NULL, NULL, NULL, NULL) > + !=3D 0) > + { > + not_sorted: > + free (cu_index->debug_info_offsets); > + cu_index->debug_info_offsets =3D NULL; > + if (tu_index !=3D NULL) > + { > + free (tu_index->debug_info_offsets); > + tu_index->debug_info_offsets =3D NULL; > + } > + break; > + } > + if (unit_type !=3D DW_UT_split_type && cui < cu_count) > + { > + if ((off & UINT32_MAX) !=3D read_4ubyte_unaligned (dbg, cu_offset= )) > + goto not_sorted; > + cu_index->debug_info_offsets[cui++] =3D off; > + cu_offset +=3D cu_index->section_count * 4; > + } > + else if (unit_type =3D=3D DW_UT_split_type && tui < tu_count) > + { > + if ((off & UINT32_MAX) !=3D read_4ubyte_unaligned (dbg, tu_offset= )) > + goto not_sorted; > + tu_index->debug_info_offsets[tui++] =3D off; > + tu_offset +=3D tu_index->section_count * 4; > + } > + off =3D next_off; > + } > + > + if (tu) > + dbg->cu_index =3D cu_index; > + else if (tu_index !=3D NULL) > + dbg->tu_index =3D tu_index; > + } > + > if (tu) > dbg->tu_index =3D index; > else This looks correct, but gcc noticed a path to use tu_offset (and tu_index) if they weren't initialized or NULL: In file included from /home/mark/src/elfutils/libdw/libdwP.h:684, from /home/mark/src/elfutils/libdw/dwarf_cu_dwp_section_in= fo.c:35: In function =E2=80=98read_4ubyte_unaligned_1=E2=80=99, inlined from =E2=80=98__libdw_package_index=E2=80=99 at /home/mark/src/= elfutils/libdw/dwarf_cu_dwp_section_info.c:302:34: /home/mark/src/elfutils/libdw/memory-access.h:291:12: error: =E2=80=98tu_of= fset=E2=80=99 may be used uninitialized [-Werror=3Dmaybe-uninitialized] 291 | return up->u4; | ~~^~~~ /home/mark/src/elfutils/libdw/dwarf_cu_dwp_section_info.c: In function =E2= =80=98__libdw_package_index=E2=80=99: /home/mark/src/elfutils/libdw/dwarf_cu_dwp_section_info.c:268:28: note: =E2= =80=98tu_offset=E2=80=99 was declared here 268 | const unsigned char *tu_offset; | ^~~~~~~~~ cc1: all warnings being treated as errors I couldn't immediately disprove gcc here, so I think it is a good idea to add an explicit check for tu_index !=3D NULL. diff --git a/libdw/dwarf_cu_dwp_section_info.c b/libdw/dwarf_cu_dwp_section= _info.c index 3d11c87a..9fdc15bf 100644 --- a/libdw/dwarf_cu_dwp_section_info.c +++ b/libdw/dwarf_cu_dwp_section_info.c @@ -297,7 +297,8 @@ __libdw_package_index (Dwarf *dbg, bool tu) cu_index->debug_info_offsets[cui++] =3D off; cu_offset +=3D cu_index->section_count * 4; } - else if (unit_type =3D=3D DW_UT_split_type && tui < tu_count) + else if (unit_type =3D=3D DW_UT_split_type && tu_index !=3D NULL + && tui < tu_count) { if ((off & UINT32_MAX) !=3D read_4ubyte_unaligned (dbg, tu_of= fset)) goto not_sorted; Which makes gcc happy again. > @@ -244,8 +380,13 @@ __libdw_dwp_section_info (Dwarf_Package_Index *index= , uint32_t unit_row, > size_t i =3D (size_t)(unit_row - 1) * index->section_count > + index->sections[section - 1]; > if (offsetp !=3D NULL) > - *offsetp =3D read_4ubyte_unaligned (index->dbg, > - index->section_offsets + i * 4); > + { > + if (section =3D=3D DW_SECT_INFO && index->debug_info_offsets !=3D = NULL) > + *offsetp =3D index->debug_info_offsets[unit_row - 1]; > + else > + *offsetp =3D read_4ubyte_unaligned (index->dbg, > + index->section_offsets + i * 4); > + } > if (sizep !=3D NULL) > *sizep =3D read_4ubyte_unaligned (index->dbg, > index->section_sizes + i * 4); OK. > diff --git a/libdw/dwarf_end.c b/libdw/dwarf_end.c > index 78224ddb..ed8d27be 100644 > --- a/libdw/dwarf_end.c > +++ b/libdw/dwarf_end.c > @@ -40,6 +40,17 @@ > #include "cfi.h" > =20 > =20 > +static void > +dwarf_package_index_free (Dwarf_Package_Index *index) > +{ > + if (index !=3D NULL) > + { > + free (index->debug_info_offsets); > + free (index); > + } > +} > + > + > static void > noop_free (void *arg __attribute__ ((unused))) > { > @@ -79,8 +90,8 @@ dwarf_end (Dwarf *dwarf) > { > if (dwarf !=3D NULL) > { > - free (dwarf->tu_index); > - free (dwarf->cu_index); > + dwarf_package_index_free (dwarf->tu_index); > + dwarf_package_index_free (dwarf->cu_index); > =20 > if (dwarf->cfi !=3D NULL) > /* Clean up the CFI cache. */ OK. > diff --git a/libdw/libdwP.h b/libdw/libdwP.h > index 1a0a4df3..6018399c 100644 > --- a/libdw/libdwP.h > +++ b/libdw/libdwP.h > @@ -371,6 +371,9 @@ typedef struct Dwarf_Package_Index_s > const unsigned char *indices; > const unsigned char *section_offsets; > const unsigned char *section_sizes; > + /* If DW_SECT_INFO section offsets were truncated to 32 bits, recovere= d > + 64-bit offsets. */ > + Dwarf_Off *debug_info_offsets; > } Dwarf_Package_Index; > =20 > /* CU representation. */ OK. > diff --git a/tests/Makefile.am b/tests/Makefile.am > index 3f80c451..98131a6b 100644 > --- a/tests/Makefile.am > +++ b/tests/Makefile.am > @@ -641,7 +641,12 @@ EXTRA_DIST =3D run-arextract.sh run-arsymtest.sh run= -ar.sh \ > testfile-dwp-4.bz2 testfile-dwp-4.dwp.bz2 \ > testfile-dwp-4-strict.bz2 testfile-dwp-4-strict.dwp.bz2 \ > testfile-dwp-5.bz2 testfile-dwp-5.dwp.bz2 testfile-dwp.source \ > - run-cu-dwp-section-info.sh run-declfiles.sh > + run-cu-dwp-section-info.sh run-declfiles.sh \ > + testfile-dwp-5-cu-index-overflow \ > + testfile-dwp-5-cu-index-overflow.dwp \ > + testfile-dwp-4-cu-index-overflow \ > + testfile-dwp-4-cu-index-overflow.dwp \ > + testfile-dwp-cu-index-overflow.source > =20 > =20 > if USE_VALGRIND You mean the .bz2 files here. Fixed. diff --git a/tests/Makefile.am b/tests/Makefile.am index 98131a6b..9141074f 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -642,10 +642,10 @@ EXTRA_DIST =3D run-arextract.sh run-arsymtest.sh run-= ar.sh \ testfile-dwp-4-strict.bz2 testfile-dwp-4-strict.dwp.bz2 \ testfile-dwp-5.bz2 testfile-dwp-5.dwp.bz2 testfile-dwp.source = \ run-cu-dwp-section-info.sh run-declfiles.sh \ - testfile-dwp-5-cu-index-overflow \ - testfile-dwp-5-cu-index-overflow.dwp \ - testfile-dwp-4-cu-index-overflow \ - testfile-dwp-4-cu-index-overflow.dwp \ + testfile-dwp-5-cu-index-overflow.bz2 \ + testfile-dwp-5-cu-index-overflow.dwp.bz2 \ + testfile-dwp-4-cu-index-overflow.bz2 \ + testfile-dwp-4-cu-index-overflow.dwp.bz2 \ testfile-dwp-cu-index-overflow.source =20 =20 Thanks for the new tests. Pushed with the fixlets above. Cheers, Mark