From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by sourceware.org (Postfix) with ESMTPS id 48F383857C62 for ; Thu, 14 Mar 2024 13:25:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 48F383857C62 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 48F383857C62 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::42d ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710422762; cv=none; b=LVcFz1If77REaJeOSMyOStK2JFBStNxTrsNYz9r+FKSf3TazKbhMOx/5rmsm/RsSGNct1b0PEDV8Rh45AlSao6sYGlK+sUiEjaDGbc1qRelTJqtc6zUJ6y2033xym6sBPm86kPP1COgBTyAwLhBEiwEpmpJ7e5/XyecB8WuloUs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710422762; c=relaxed/simple; bh=+jnyMc2f4yIw+blArK6ERGeAdg+Sd2cJ9Mhe0FnRnEw=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=wUdPYMyuK6avp6YkssN/Twahu1l45AzmXpt7UKZiA8v74t6Iga2kGu16gM1puUNygO/G+aU99h9yMgO4ZwgtpiljJbG86wKNyS4PzTGJPOblgPxwpm9ijjfECminYafTToDvla95bP51MPNb8uUSqfqwfDtfEM39X9Rtmj5gxMI= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-6e6b54a28d0so846660b3a.2 for ; Thu, 14 Mar 2024 06:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710422757; x=1711027557; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=orBT0KACp4P6NJcM0hGDO37dKoUKitUWAjDw+8fd8Zs=; b=l0sSK+to64QEJS5uuEhzQudf2nu56t5bEwI+227ZhxKpeVK2EHavTlqtUmMXs20fXi BB72LQs9y54sb6FmJ4dquiaZxxlZBxF933jffD23EfIE89/XjiIp6mZ6opDm8xv7TGU1 q4UA+ZUhdNpEE6rFQGXV1F3G5eDMuh2jBN5ohgojdlfuZyHyLkpKqfOY+1vrDFyeRJEA uZkuzuY04mfqLo6YZthr9aVNODydyMZCXFqz1rysr9AisiovrhtsBO/CLZGBlX5bxmH5 nCvvd2cKv46lZ3MHNjzojIidYcNv0GJPnK5DzrYf0G2Id+axd1e3goWdDEp5KdWhRPFM x65g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710422757; x=1711027557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=orBT0KACp4P6NJcM0hGDO37dKoUKitUWAjDw+8fd8Zs=; b=Uh0/DMEgwI1NlbGKeiZnxvE20QeNlLhqWrnj7Rv0JzwcirRmKjMbzNnXu05fS+K79D IJdceDNPqO+1tYTvo4p5BDwNDzdLWLeMzpGnO1ia827y4DAlF8r3kfkOsw9RFuMXnk0K ksW/56CQnWKBzZnnjzsq4d9snM4bOcohkamyiZcieFvAQgTwre4k2HLTDEUeZ50EYJLA l/jTq2/kksPsQ4zMJTaBa8roOQ6v4S2+RVAzg74AWmJfGduzIz0sZd9doV3YNwCXWGaO dZB+Fl8tHE9E40aKw5tM5CLNCWfyPqDfJkJyJWqH57pr5E7QlNDeDV128lKxNtjs/cQh RCTg== X-Gm-Message-State: AOJu0YwKiYWVvahjO6vgG+66oquYgvUBa+/Ep5MGUSO2okBgHkO/uK7S afFXSbEfqicHnUoAQ3PmhRH0LFloYSCnJcRfz6Z97efdZ9GVrEIO X-Google-Smtp-Source: AGHT+IF8JK5bRLo90dwP1UIFpSv3GUbfrJX/kSXwd1FtC+fpzarWuJ1dd+3ndXPuIeQ0Q9gpK3YP6g== X-Received: by 2002:a05:6a00:21cf:b0:6e6:42ef:ed1b with SMTP id t15-20020a056a0021cf00b006e642efed1bmr149839pfj.31.1710422757035; Thu, 14 Mar 2024 06:25:57 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id x2-20020aa79182000000b006e5dc1b4866sm1474970pfa.144.2024.03.14.06.25.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Mar 2024 06:25:53 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id AD0827406C4; Thu, 14 Mar 2024 06:25:52 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v10 1/6] elf: Use mmap to map in read-only sections Date: Thu, 14 Mar 2024 06:25:47 -0700 Message-ID: <20240314132552.83916-2-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240314132552.83916-1-hjl.tools@gmail.com> References: <20240314132552.83916-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: There are many linker input files in LLVM debug build with huge string sections. All these string sections can be treated as read-only. But linker copies all of them into memory which consumes huge amount of memory and slows down linker significantly. Add _bfd_mmap_readonly_persistent and _bfd_mmap_readonly_temporary to mmap in reado-only sections with size >= 4 * page size. NB: All string sections in valid ELF inputs must be null terminated. There is no need to terminate it again and string sections are mmapped as read-only. * bfd.c (bfd_mmapped_entry): New. (bfd_mmapped): Likewise. (bfd): Add mmapped. * bfdwin.c (bfd_get_file_window): Use _bfd_pagesize. * cache.c (cache_bmmap): Remove pagesize_m1 and use pagesize_m1 instead. * elf.c (bfd_elf_get_str_section): Call _bfd_mmap_readonly_persistent instead of _bfd_alloc_and_read. Don't terminate the string section again. (get_hash_table_data): Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. (_bfd_elf_get_dynamic_symbols): Call _bfd_mmap_readonly_persistent instead of _bfd_alloc_and_read. Don't terminate the string section again. Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. (_bfd_elf_slurp_version_tables): Call _bfd_mmap_readonly_temporary and _bfd_munmap_readonly_temporary instead of _bfd_malloc_and_read and free. * elflink.c (bfd_elf_link_record_dynamic_symbol): Use bfd_malloc to get the unversioned symbol. * libbfd-in.h (_bfd_pagesize): New. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (_bfd_mmap_readonly_persistent): Likewise. (_bfd_mmap_readonly_temporary): Likewise. (_bfd_munmap_readonly_temporary): Likewise. * libbfd.c (bfd_allocate_mmapped_page): New. (_bfd_mmap_readonly_temporary): Likewise. (_bfd_munmap_readonly_temporary): Likewise. (_bfd_mmap_readonly_persistent): Likewise. (_bfd_pagesize): Likewise. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (bfd_init_pagesize): Likewise. * lynx-core.c (lynx_core_file_p): Use _bfd_pagesize. * opncls.c (_bfd_delete_bfd): Munmap tracked mmapped memories. * sysdep.h (MAP_ANONYMOUS): New. Define if undefined. * bfd-in2.h: Regenerated. * libbfd.h: Likewise. --- bfd/bfd-in2.h | 17 ++++++ bfd/bfd.c | 17 ++++++ bfd/bfdwin.c | 8 +-- bfd/cache.c | 7 +-- bfd/elf.c | 79 +++++++++++++++++-------- bfd/elflink.c | 16 ++--- bfd/libbfd-in.h | 29 ++++++++++ bfd/libbfd.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++ bfd/libbfd.h | 29 ++++++++++ bfd/lynx-core.c | 2 +- bfd/opncls.c | 12 ++++ bfd/sysdep.h | 4 ++ 12 files changed, 328 insertions(+), 43 deletions(-) diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 29602e054da..544422a9522 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -1951,6 +1951,20 @@ struct bfd_build_id bfd_byte data[1]; }; +struct bfd_mmapped_entry + { + void *addr; + size_t size; + }; + +struct bfd_mmapped + { + struct bfd_mmapped *next; + unsigned int max_entry; + unsigned int next_entry; + struct bfd_mmapped_entry entries[1]; + }; + struct bfd { /* The filename the application opened the BFD with. */ @@ -2280,6 +2294,9 @@ struct bfd /* For input BFDs, the build ID, if the object has one. */ const struct bfd_build_id *build_id; + + /* For input BFDs, mmapped entries. */ + struct bfd_mmapped *mmapped; }; static inline const char * diff --git a/bfd/bfd.c b/bfd/bfd.c index 54061a34240..8302013f1b4 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -74,6 +74,20 @@ EXTERNAL . bfd_byte data[1]; . }; . +.struct bfd_mmapped_entry +. { +. void *addr; +. size_t size; +. }; +. +.struct bfd_mmapped +. { +. struct bfd_mmapped *next; +. unsigned int max_entry; +. unsigned int next_entry; +. struct bfd_mmapped_entry entries[1]; +. }; +. CODE_FRAGMENT .struct bfd @@ -406,6 +420,9 @@ CODE_FRAGMENT . . {* For input BFDs, the build ID, if the object has one. *} . const struct bfd_build_id *build_id; +. +. {* For input BFDs, mmapped entries. *} +. struct bfd_mmapped *mmapped; .}; . diff --git a/bfd/bfdwin.c b/bfd/bfdwin.c index 2919c71b3cb..73e44635bcb 100644 --- a/bfd/bfdwin.c +++ b/bfd/bfdwin.c @@ -157,7 +157,7 @@ bfd_get_file_window (bfd *abfd, bool writable) { static int ok_to_map = 1; - static size_t pagesize; + size_t pagesize = _bfd_pagesize; bfd_window_internal *i = windowp->i; bfd_size_type size_to_alloc = size; @@ -167,12 +167,6 @@ bfd_get_file_window (bfd *abfd, windowp, windowp->data, (unsigned long) windowp->size, windowp->i, writable); - /* Make sure we know the page size, so we can be friendly to mmap. */ - if (pagesize == 0) - pagesize = getpagesize (); - if (pagesize == 0) - abort (); - if (i == NULL) { i = bfd_zmalloc (sizeof (bfd_window_internal)); diff --git a/bfd/cache.c b/bfd/cache.c index d0e7be293a5..0f994c74239 100644 --- a/bfd/cache.c +++ b/bfd/cache.c @@ -494,10 +494,10 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, #ifdef HAVE_MMAP else { - static uintptr_t pagesize_m1; + uintptr_t pagesize_m1 = _bfd_pagesize_m1; FILE *f; file_ptr pg_offset; - bfd_size_type pg_len; + size_t pg_len; f = bfd_cache_lookup (abfd, CACHE_NO_SEEK_ERROR); if (f == NULL) @@ -506,9 +506,6 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, return ret; } - if (pagesize_m1 == 0) - pagesize_m1 = getpagesize () - 1; - /* Align. */ pg_offset = offset & ~pagesize_m1; pg_len = (len + (offset - pg_offset) + pagesize_m1) & ~pagesize_m1; diff --git a/bfd/elf.c b/bfd/elf.c index 8bffd3c5141..c80fff47b45 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -289,16 +289,23 @@ bfd_elf_get_str_section (bfd *abfd, unsigned int shindex) in case the string table is not terminated. */ if (shstrtabsize + 1 <= 1 || bfd_seek (abfd, offset, SEEK_SET) != 0 - || (shstrtab = _bfd_alloc_and_read (abfd, shstrtabsize + 1, - shstrtabsize)) == NULL) + || (shstrtab + = _bfd_mmap_readonly_persistent (abfd, shstrtabsize)) == NULL) { /* Once we've failed to read it, make sure we don't keep trying. Otherwise, we'll keep allocating space for the string table over and over. */ i_shdrp[shindex]->sh_size = 0; } - else - shstrtab[shstrtabsize] = '\0'; + else if (shstrtab[shstrtabsize - 1] != '\0') + { + /* It is an error if a string table isn't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB(%pA): string table is corrupt"), + abfd, i_shdrp[shindex]->bfd_section); + return NULL; + } i_shdrp[shindex]->contents = shstrtab; } return (char *) shstrtab; @@ -1919,6 +1926,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, unsigned char *e_data = NULL; bfd_vma *i_data = NULL; bfd_size_type size; + void *e_data_addr; + size_t e_data_size ATTRIBUTE_UNUSED; if (ent_size != 4 && ent_size != 8) return NULL; @@ -1940,7 +1949,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, return NULL; } - e_data = _bfd_malloc_and_read (abfd, size, size); + e_data = _bfd_mmap_readonly_temporary (abfd, size, &e_data_addr, + &e_data_size); if (e_data == NULL) return NULL; @@ -1958,7 +1968,7 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, while (number--) i_data[number] = bfd_get_64 (abfd, e_data + number * ent_size); - free (e_data); + _bfd_munmap_readonly_temporary (e_data_addr, e_data_size); return i_data; } @@ -2007,6 +2017,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, size_t verneed_size = 0; size_t extsym_size; const struct elf_backend_data *bed; + void *dynbuf_addr = NULL; + void *esymbuf_addr = NULL; + size_t dynbuf_size = 0; + size_t esymbuf_size = 0; /* Return TRUE if symbol table is bad. */ if (elf_bad_symtab (abfd)) @@ -2024,7 +2038,9 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (bfd_seek (abfd, phdr->p_offset, SEEK_SET) != 0) goto error_return; - dynbuf = _bfd_malloc_and_read (abfd, phdr->p_filesz, phdr->p_filesz); + dynbuf_size = phdr->p_filesz; + dynbuf = _bfd_mmap_readonly_temporary (abfd, dynbuf_size, + &dynbuf_addr, &dynbuf_size); if (dynbuf == NULL) goto error_return; @@ -2102,11 +2118,17 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* Dynamic string table must be valid until ABFD is closed. */ - strbuf = (char *) _bfd_alloc_and_read (abfd, dt_strsz + 1, dt_strsz); + strbuf = (char *) _bfd_mmap_readonly_persistent (abfd, dt_strsz); if (strbuf == NULL) goto error_return; - /* Since this is a string table, make sure that it is terminated. */ - strbuf[dt_strsz] = 0; + if (strbuf[dt_strsz - 1] != 0) + { + /* It is an error if a string table is't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB: DT_STRTAB table is corrupt"), abfd); + goto error_return; + } /* Get the real symbol count from DT_HASH or DT_GNU_HASH. Prefer DT_HASH since it is simpler than DT_GNU_HASH. */ @@ -2281,7 +2303,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (filepos == (file_ptr) -1 || bfd_seek (abfd, filepos, SEEK_SET) != 0) goto error_return; - esymbuf = _bfd_malloc_and_read (abfd, amt, amt); + esymbuf_size = amt; + esymbuf = _bfd_mmap_readonly_temporary (abfd, esymbuf_size, + &esymbuf_addr, + &esymbuf_size); if (esymbuf == NULL) goto error_return; @@ -2325,7 +2350,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERSYM info must be valid until ABFD is closed. */ - versym = _bfd_alloc_and_read (abfd, amt, amt); + versym = _bfd_mmap_readonly_persistent (abfd, amt); if (dt_verdef) { @@ -2337,8 +2362,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERDEF info must be valid until ABFD is closed. */ - verdef = _bfd_alloc_and_read (abfd, verdef_size, - verdef_size); + verdef = _bfd_mmap_readonly_persistent (abfd, verdef_size); } if (dt_verneed) @@ -2351,8 +2375,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERNEED info must be valid until ABFD is closed. */ - verneed = _bfd_alloc_and_read (abfd, verneed_size, - verneed_size); + verneed = _bfd_mmap_readonly_persistent (abfd, verneed_size); } } @@ -2375,8 +2398,8 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, /* Restore file position for elf_object_p. */ if (bfd_seek (abfd, saved_filepos, SEEK_SET) != 0) res = false; - free (dynbuf); - free (esymbuf); + _bfd_munmap_readonly_temporary (dynbuf_addr, dynbuf_size); + _bfd_munmap_readonly_temporary (esymbuf_addr, esymbuf_size); free (gnubuckets); free (gnuchains); free (mipsxlat); @@ -9435,6 +9458,8 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) bfd_byte *contents = NULL; unsigned int freeidx = 0; size_t amt; + void *contents_addr = NULL; + size_t contents_size = 0; if (elf_dynverref (abfd) != 0 || elf_tdata (abfd)->dt_verneed != NULL) { @@ -9471,7 +9496,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verref; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_temporary (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verref; @@ -9604,8 +9632,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) elf_tdata (abfd)->cverrefs = i; if (contents != elf_tdata (abfd)->dt_verneed) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } if (elf_dynverdef (abfd) != 0 || elf_tdata (abfd)->dt_verdef != NULL) @@ -9646,7 +9675,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verdef; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_temporary (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verdef; @@ -9800,8 +9832,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) } if (contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } else if (default_imported_symver) { @@ -9857,7 +9890,7 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) error_return: if (contents != elf_tdata (abfd)->dt_verneed && contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_temporary (contents_addr, contents_size); return false; } diff --git a/bfd/elflink.c b/bfd/elflink.c index 5a6cb07b2ce..42029f29f7a 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -549,22 +549,24 @@ bfd_elf_link_record_dynamic_symbol (struct bfd_link_info *info, return false; } + char *unversioned_name = NULL; + /* We don't put any version information in the dynamic string table. */ name = h->root.root.string; p = strchr (name, ELF_VER_CHR); if (p != NULL) - /* We know that the p points into writable memory. In fact, - there are only a few symbols that have read-only names, being - those like _GLOBAL_OFFSET_TABLE_ that are created specially - by the backends. Most symbols will have names pointing into - an ELF string table read from a file, or to objalloc memory. */ - *p = 0; + { + unversioned_name = bfd_malloc (p - name + 1); + memcpy (unversioned_name, name, p - name); + unversioned_name[p - name] = 0; + name = unversioned_name; + } indx = _bfd_elf_strtab_add (dynstr, name, p != NULL); if (p != NULL) - *p = ELF_VER_CHR; + free (unversioned_name); if (indx == (size_t) -1) return false; diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index b8b2ce7ba09..c5a79cf932c 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -851,6 +851,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -888,6 +892,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_persistent + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_temporary + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_temporary + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_persistent(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -910,3 +927,15 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif diff --git a/bfd/libbfd.c b/bfd/libbfd.c index f8d148c9677..a79c814a0dc 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -1038,6 +1038,141 @@ bfd_get_bits (const void *p, int bits, bool big_p) return data; } +#ifdef USE_MMAP +/* Allocate a page to track mmapped memory and return the page and + the first entry. Return NULL if mmap fails. */ + +static struct bfd_mmapped * +bfd_allocate_mmapped_page (bfd *abfd, struct bfd_mmapped_entry **entry) +{ + struct bfd_mmapped * mmapped + = (struct bfd_mmapped *) mmap (NULL, _bfd_pagesize, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + if (mmapped == MAP_FAILED) + return NULL; + + mmapped->next = abfd->mmapped; + mmapped->max_entry + = ((_bfd_pagesize - offsetof (struct bfd_mmapped, entries)) + / sizeof (struct bfd_mmapped_entry)); + mmapped->next_entry = 1; + abfd->mmapped = mmapped; + *entry = mmapped->entries; + return mmapped; +} + +/* Mmap a memory region of RSIZE bytes with PROT at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and MAP_FAILED for mmap failure. */ + +static void * +bfd_mmap_local (bfd *abfd, size_t rsize, int prot, void **map_addr, + size_t *map_size) +{ + if (!_bfd_constant_p (rsize)) + { + ufile_ptr filesize = bfd_get_file_size (abfd); + if (filesize != 0 && rsize > filesize) + { + bfd_set_error (bfd_error_file_truncated); + return NULL; + } + } + + void *mem; + ufile_ptr offset = bfd_tell (abfd); + mem = bfd_mmap (abfd, NULL, rsize, prot, MAP_PRIVATE, offset, + map_addr, map_size); + return mem; +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and MAP_FAILED for mmap failure. */ + +void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + { + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + /* NB: Set *MAP_ADDR to MEM and *MAP_SIZE to 0 to indicate that + _bfd_malloc_and_read is called. */ + *map_addr = mem; + *map_size = 0; + return mem; + } + + return bfd_mmap_local (abfd, rsize, PROT_READ, map_addr, map_size); +} + +/* Munmap RSIZE bytes at PTR. */ + +void +_bfd_munmap_readonly_temporary (void *ptr, size_t rsize) +{ + /* NB: Since _bfd_munmap_readonly_temporary is called like free, PTR + may be NULL. Otherwise, PTR and RSIZE must be valid. If RSIZE is + 0, _bfd_malloc_and_read is called. */ + if (ptr == NULL) + return; + if (rsize != 0) + { + if (munmap (ptr, rsize) != 0) + abort (); + } + else + free (ptr); +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return NULL on invalid input or mmap failure. */ + +void * +_bfd_mmap_readonly_persistent (bfd *abfd, size_t rsize) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + void *mem, *map_addr; + size_t map_size; + mem = bfd_mmap_local (abfd, rsize, PROT_READ, &map_addr, &map_size); + if (mem == NULL) + return mem; + if (mem == MAP_FAILED) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + struct bfd_mmapped_entry *entry; + unsigned int next_entry; + struct bfd_mmapped *mmapped = abfd->mmapped; + if (mmapped != NULL + && (next_entry = mmapped->next_entry) < mmapped->max_entry) + { + entry = &mmapped->entries[next_entry]; + mmapped->next_entry++; + } + else + { + mmapped = bfd_allocate_mmapped_page (abfd, &entry); + if (mmapped == NULL) + { + munmap (map_addr, map_size); + return NULL; + } + } + + entry->addr = map_addr; + entry->size = map_size; + + return mem; +} +#endif + /* Default implementation */ bool @@ -1326,3 +1461,19 @@ _bfd_generic_init_private_section_data (bfd *ibfd ATTRIBUTE_UNUSED, { return true; } + +uintptr_t _bfd_pagesize; +uintptr_t _bfd_pagesize_m1; +uintptr_t _bfd_minimum_mmap_size; + +__attribute__ ((unused, constructor)) +static void +bfd_init_pagesize (void) +{ + _bfd_pagesize = getpagesize (); + if (_bfd_pagesize == 0) + abort (); + _bfd_pagesize_m1 = _bfd_pagesize - 1; + /* The minimum section size to use mmap. */ + _bfd_minimum_mmap_size = _bfd_pagesize * 4; +} diff --git a/bfd/libbfd.h b/bfd/libbfd.h index f15b5f27db8..47f40889a95 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -857,6 +857,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -894,6 +898,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_persistent + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_temporary + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_temporary + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_persistent(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_temporary(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -916,6 +933,18 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_temporary (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif /* Extracted from libbfd.c. */ void *bfd_malloc (bfd_size_type /*size*/) ATTRIBUTE_HIDDEN; diff --git a/bfd/lynx-core.c b/bfd/lynx-core.c index 44d94ad8745..9ec5a0d2028 100644 --- a/bfd/lynx-core.c +++ b/bfd/lynx-core.c @@ -96,7 +96,7 @@ lynx_core_file_p (bfd *abfd) asection *newsect; size_t amt; - pagesize = getpagesize (); /* Serious cross-target issue here... This + pagesize = _bfd_pagesize; /* Serious cross-target issue here... This really needs to come from a system-specific header file. */ diff --git a/bfd/opncls.c b/bfd/opncls.c index c764d204831..e6337b88e18 100644 --- a/bfd/opncls.c +++ b/bfd/opncls.c @@ -163,6 +163,18 @@ _bfd_new_bfd_contained_in (bfd *obfd) static void _bfd_delete_bfd (bfd *abfd) { +#ifdef USE_MMAP + struct bfd_mmapped *mmapped, *next; + for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next) + { + struct bfd_mmapped_entry *entries = mmapped->entries; + next = mmapped->next; + for (unsigned int i = 0; i < mmapped->next_entry; i++) + munmap (entries[i].addr, entries[i].size); + munmap (mmapped, _bfd_pagesize); + } +#endif + /* Give the target _bfd_free_cached_info a chance to free memory. */ if (abfd->memory && abfd->xvec) bfd_free_cached_info (abfd); diff --git a/bfd/sysdep.h b/bfd/sysdep.h index b907bc26a09..173f5804df3 100644 --- a/bfd/sysdep.h +++ b/bfd/sysdep.h @@ -98,6 +98,10 @@ #define MAP_FAILED ((void *) -1) #endif +#ifndef MAP_ANONYMOUS +#define MAP_ANONYMOUS MAP_ANON +#endif + #include "filenames.h" #if !HAVE_DECL_FFS -- 2.44.0