From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) by sourceware.org (Postfix) with ESMTPS id C14C83858032 for ; Wed, 6 Mar 2024 20:47:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C14C83858032 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C14C83858032 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::32a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709758067; cv=none; b=F/I5G5FhjRvSj6jNdvYOfr62KVTijZlHYe6Yr9q/UtnSVyt9w/y/KYlgwENxpFGof5McF9o/825RGTZmF0xAws1pVclHsBswNHLlmydcPCsDc3zdjidJe+DMW0U3wXfXPw+7Ym0WglzcbYI7zFGz171EbZ/dECL1u2s2nD7iFXk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709758067; c=relaxed/simple; bh=hmZEpVWvBb34mK/ftwMD7o4n5QnYMkWMNQTuJZRrZQc=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=utQ7CBSP6LEmAYeK9IVoZMBW4NBx2zeJihlUGLSZ3V9eWY2XQaDJqXhSQDWCm7XYs7CaALEZXKIFfAL10fWfPgKuDZ/33QdWSmjwj9YgFwq7U98b/fmF4Sg1zcAYHQgPSxdLbhVVJ8JAOAO+zRq/y0ETnwStlR9ubyuqx73euFI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ot1-x32a.google.com with SMTP id 46e09a7af769-6e4e9f8539bso80162a34.2 for ; Wed, 06 Mar 2024 12:47:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709758056; x=1710362856; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mKJzpE9FGXJv0V4eJ+Br8UbcRNCT1ESwdfEAqGl2cAs=; b=jZ8Ta4RXbkDE9T4baXqQZnTN53AjhnseCmiYN6OBQDyeMLmoYLfSd3FtWXuNtJlRRV eoaADH+4Wo/smp96ZXlTJQ/rDU65tN05KDIVbkzPYWUapBwCMPDWFyD9FIMPPOTChHVn vbZrUOewKWYdqT4HsSq3NMcUAerpAin8SeVCResczwJ4ySh4L3E3ui3I5oORcnp8Lrsa Eh7Q1zuh57PYLspUmMWjWU4gfm19zr5JnDwrYE5mqXF50VypUwbeM/K5uqq3dcOd893P 6k0K0OEiqPZEVHNGP3BdZwuxKcS2p1bzviCHDaV7L0FMrxhC7B9ldyEnM6RU2/gnOCxk 9ISw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709758056; x=1710362856; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mKJzpE9FGXJv0V4eJ+Br8UbcRNCT1ESwdfEAqGl2cAs=; b=BJtWYxBPMj31MNkhKlvbMijpkPippMq6c1yBxoFKaaa98UGEEYRvbXzSr+nWHalWIg t4+PA7/7o+Te3jSu25JamkB/OPopMJJOyrcofrQKsNxkB5+GntVYdDN0cIvOWMi7ayUC tql1DnGri3pCgmCbWqMRoC44vuXMAnbBOweRLSBjmIeF8AXVWG7e8LzGYH+xVAbWT3GR f6SX9r3lnXzO8TIOC8swIeD1YNe8qqdDIBCn3zfeKwYpCkQjAekAaUUmu+cnhm6BSPDj 5Iuv7l9rI1K/8/eTka4nzS+aY9nEOOVQlZfsMQcxlCsrs/r+2zF0txjnfbxzDepL9VlC m0Wg== X-Gm-Message-State: AOJu0YwGEHR37eXb9XetyEKrDG5x3RCpBQQmyCniDIC9JP2DBFe3Kyq/ 7zbIsFSRTMABWhxGUgAiaIX09QGJ5naCmDpCZeiDygTK/mC8BasRr+qisTsb X-Google-Smtp-Source: AGHT+IGcBClqSe927qf909jNT/fDKqjKv7HyamdJ+21XcYFgPTa0GtLKngkqqTcG0JgaFOe301Juug== X-Received: by 2002:a05:6830:108b:b0:6e4:f37e:cb5d with SMTP id y11-20020a056830108b00b006e4f37ecb5dmr6442783oto.5.1709758055952; Wed, 06 Mar 2024 12:47:35 -0800 (PST) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id 15-20020a63184f000000b005c662e103a1sm11346379pgy.41.2024.03.06.12.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Mar 2024 12:47:35 -0800 (PST) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id CF4507406C7; Wed, 6 Mar 2024 12:47:31 -0800 (PST) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, amodra@gmail.com Subject: [PATCH v3 5/6] elf: Use mmap to map in symbol and relocation tables Date: Wed, 6 Mar 2024 12:47:30 -0800 Message-ID: <20240306204731.1098405-6-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240306204731.1098405-1-hjl.tools@gmail.com> References: <20240306204731.1098405-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Add _bfd_mmap_read_untracked to mmap in symbol tables and relocations whose sizes >= 4 * page size. Don't cache external relocations when mmap is used. When mmap is used to map in all ELF sections, data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: stdio mmap improvement user 84.28 85.04 -0.9% system 12.46 10.16 14% total 96 95.35 0.7% page faults 4837944 4047667 16% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.22 5.27 -1% system 0.94 0.84 11% total 6.20 6.13 0.7% page faults 361272 323377 10% * elf.c (bfd_elf_get_elf_syms): Replace bfd_read with _bfd_mmap_read_untracked. * elflink.c (elf_link_read_relocs_from_section): Add 2 arguments to return mmap memory address and size. (_bfd_elf_link_info_read_relocs); Replace bfd_read with _bfd_mmap_read_untracked. (bfd_elf_final_link): Don't cache external relocations when mmap is used. * libbfd.c (_bfd_mmap_read_untracked ): New. * libbfd-in.h (_bfd_mmap_read_untracked): Likewise. * libbfd.h: Regenerated. --- bfd/elf.c | 47 +++++++++++++++++++++++++++++++++++------------ bfd/elflink.c | 44 ++++++++++++++++++++++++++++++++------------ bfd/libbfd-in.h | 3 +++ bfd/libbfd.c | 33 +++++++++++++++++++++++++++++++++ bfd/libbfd.h | 3 +++ 5 files changed, 106 insertions(+), 24 deletions(-) diff --git a/bfd/elf.c b/bfd/elf.c index e2e31a93950..7427dca2ba0 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -464,19 +464,30 @@ bfd_elf_get_elf_syms (bfd *ibfd, goto out; } pos = symtab_hdr->sh_offset + symoffset * extsym_size; + size_t alloc_ext_size = amt; if (extsym_buf == NULL) { - alloc_ext = bfd_malloc (amt); - extsym_buf = alloc_ext; +#ifdef USE_MMAP + if ((ibfd->flags & BFD_PLUGIN) != 0 + || amt < _bfd_minimum_mmap_size) + { +#endif + alloc_ext = bfd_malloc (amt); + extsym_buf = alloc_ext; +#ifdef USE_MMAP + } +#endif } - if (extsym_buf == NULL - || bfd_seek (ibfd, pos, SEEK_SET) != 0 - || bfd_read (extsym_buf, amt, ibfd) != amt) + + if (bfd_seek (ibfd, pos, SEEK_SET) != 0 + || !_bfd_mmap_read_untracked (&extsym_buf, &alloc_ext_size, + &alloc_ext, ibfd)) { intsym_buf = NULL; goto out; } + size_t alloc_extshndx_size = 0; if (shndx_hdr == NULL || shndx_hdr->sh_size == 0) extshndx_buf = NULL; else @@ -487,15 +498,27 @@ bfd_elf_get_elf_syms (bfd *ibfd, intsym_buf = NULL; goto out; } + alloc_extshndx_size = amt; pos = shndx_hdr->sh_offset + symoffset * sizeof (Elf_External_Sym_Shndx); if (extshndx_buf == NULL) { - alloc_extshndx = (Elf_External_Sym_Shndx *) bfd_malloc (amt); - extshndx_buf = alloc_extshndx; +#ifdef USE_MMAP + if ((ibfd->flags & BFD_PLUGIN) != 0 + || amt < _bfd_minimum_mmap_size) + { +#endif + alloc_extshndx + = (Elf_External_Sym_Shndx *) bfd_malloc (amt); + extshndx_buf = alloc_extshndx; +#ifdef USE_MMAP + } +#endif } - if (extshndx_buf == NULL - || bfd_seek (ibfd, pos, SEEK_SET) != 0 - || bfd_read (extshndx_buf, amt, ibfd) != amt) + if (bfd_seek (ibfd, pos, SEEK_SET) != 0 + || !_bfd_mmap_read_untracked ((void **) &extshndx_buf, + &alloc_extshndx_size, + (void **) &alloc_extshndx, + ibfd)) { intsym_buf = NULL; goto out; @@ -534,8 +557,8 @@ bfd_elf_get_elf_syms (bfd *ibfd, } out: - free (alloc_ext); - free (alloc_extshndx); + _bfd_munmap_readonly_untracked (alloc_ext, alloc_ext_size); + _bfd_munmap_readonly_untracked (alloc_extshndx, alloc_extshndx_size); return intsym_buf; } diff --git a/bfd/elflink.c b/bfd/elflink.c index 47fb890f94f..4602fb3d10f 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -2644,8 +2644,11 @@ _bfd_elf_link_assign_sym_version (struct elf_link_hash_entry *h, void *data) may be either a REL or a RELA section. The relocations are translated into RELA relocations and stored in INTERNAL_RELOCS, which should have already been allocated to contain enough space. - The EXTERNAL_RELOCS are a buffer where the external form of the - relocations should be stored. + The *EXTERNAL_RELOCS_P are a buffer where the external form of the + relocations should be stored. If *EXTERNAL_RELOCS_P is NULL, + *EXTERNAL_RELOCS_P and *EXTERNAL_RELOCS_SIZE_P returns the mmap + memory address and size. Otherwise, *EXTERNAL_RELOCS_SIZE_P is + unchanged and EXTERNAL_RELOCS_SIZE_P returns 0. Returns FALSE if something goes wrong. */ @@ -2653,7 +2656,8 @@ static bool elf_link_read_relocs_from_section (bfd *abfd, asection *sec, Elf_Internal_Shdr *shdr, - void *external_relocs, + void **external_relocs_addr, + size_t *external_relocs_size_addr, Elf_Internal_Rela *internal_relocs) { const struct elf_backend_data *bed; @@ -2663,13 +2667,17 @@ elf_link_read_relocs_from_section (bfd *abfd, Elf_Internal_Rela *irela; Elf_Internal_Shdr *symtab_hdr; size_t nsyms; + void *external_relocs = *external_relocs_addr; /* Position ourselves at the start of the section. */ if (bfd_seek (abfd, shdr->sh_offset, SEEK_SET) != 0) return false; /* Read the relocations. */ - if (bfd_read (external_relocs, shdr->sh_size, abfd) != shdr->sh_size) + *external_relocs_size_addr = shdr->sh_size; + if (!_bfd_mmap_read_untracked (&external_relocs, + external_relocs_size_addr, + external_relocs_addr, abfd)) return false; symtab_hdr = &elf_tdata (abfd)->symtab_hdr; @@ -2754,6 +2762,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, bool keep_memory) { void *alloc1 = NULL; + size_t alloc1_size; Elf_Internal_Rela *alloc2 = NULL; const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data *esdo = elf_section_data (o); @@ -2791,17 +2800,26 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, if (esdo->rela.hdr) size += esdo->rela.hdr->sh_size; - alloc1 = bfd_malloc (size); - if (alloc1 == NULL) - goto error_return; - external_relocs = alloc1; +#ifdef USE_MMAP + if (size < _bfd_minimum_mmap_size) + { +#endif + alloc1 = bfd_malloc (size); + if (alloc1 == NULL) + goto error_return; + external_relocs = alloc1; +#ifdef USE_MMAP + } +#endif } + else + alloc1 = external_relocs; internal_rela_relocs = internal_relocs; if (esdo->rel.hdr) { if (!elf_link_read_relocs_from_section (abfd, o, esdo->rel.hdr, - external_relocs, + &alloc1, &alloc1_size, internal_relocs)) goto error_return; external_relocs = (((bfd_byte *) external_relocs) @@ -2812,7 +2830,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, if (esdo->rela.hdr && (!elf_link_read_relocs_from_section (abfd, o, esdo->rela.hdr, - external_relocs, + &alloc1, &alloc1_size, internal_rela_relocs))) goto error_return; @@ -2820,7 +2838,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, if (keep_memory) esdo->relocs = internal_relocs; - free (alloc1); + _bfd_munmap_readonly_untracked (alloc1, alloc1_size); /* Don't free alloc2, since if it was allocated we are passing it back (under the name of internal_relocs). */ @@ -2828,7 +2846,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd, return internal_relocs; error_return: - free (alloc1); + _bfd_munmap_readonly_untracked (alloc1, alloc1_size); if (alloc2 != NULL) { if (keep_memory) @@ -12741,12 +12759,14 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) goto error_return; } +#ifndef USE_MMAP if (max_external_reloc_size != 0) { flinfo.external_relocs = bfd_malloc (max_external_reloc_size); if (flinfo.external_relocs == NULL) goto error_return; } +#endif if (max_internal_reloc_count != 0) { diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index 7887fad9c92..effe1b86b53 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -905,6 +905,9 @@ extern void _bfd_munmap_readonly_untracked #define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr) #endif +extern bool _bfd_mmap_read_untracked + (void **, size_t *, void **, bfd *) ATTRIBUTE_HIDDEN; + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { diff --git a/bfd/libbfd.c b/bfd/libbfd.c index c847c4f0180..0a31f113999 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -1180,6 +1180,39 @@ _bfd_mmap_readonly_tracked (bfd *abfd, size_t rsize) } #endif +/* Attempt to read *SIZE_ADDR bytes from ABFD's iostream to *PTR_P. + Return true if the full the amount has been read. If *PTR_P is + NULL, mmap should be used, return the memory address at the + current offset in *PTR_P as well as return mmap address and size + in *PTR_ADDR and *SIZE_ADDR. Otherwise, return NULL in *PTR_ADDR + and 0 in *SIZE_ADDR. */ + +bool +_bfd_mmap_read_untracked (void **ptr_p, size_t *size_addr, + void **ptr_addr, bfd *abfd) +{ + void *ptr = *ptr_p; + size_t size = *size_addr; + +#ifdef USE_MMAP + if (ptr == NULL) + { + ptr = _bfd_mmap_readonly_untracked (abfd, size, ptr_addr, + size_addr); + if (ptr == NULL || ptr == (void *) -1) + abort (); + *ptr_p = ptr; + return true; + } + else +#endif + { + *ptr_addr = NULL; + *size_addr = 0; + return bfd_read (ptr, size, abfd) == size; + } +} + /* Default implementation */ bool diff --git a/bfd/libbfd.h b/bfd/libbfd.h index 1515a03b093..c80f5a86ed1 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -911,6 +911,9 @@ extern void _bfd_munmap_readonly_untracked #define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr) #endif +extern bool _bfd_mmap_read_untracked + (void **, size_t *, void **, bfd *) ATTRIBUTE_HIDDEN; + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { -- 2.44.0