From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by sourceware.org (Postfix) with ESMTPS id 706B23858417 for ; Wed, 6 Mar 2024 23:24:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 706B23858417 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 706B23858417 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::62f ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709767450; cv=none; b=JpPbR2rgOyI03pzYgCQiWncvCPsMX7PQCmz7E1/UhDyFaJvgUmAR3qK1pE+Y0+UaI+AvlHfhNwOB/kc6onWRTEFKNKoY3E67IO3jaz7mDxtWvNH81uOhx8FXXcUbMZlDDy9KWK3Ee/9yKgEWXFlb+TS3DIs3RrNXrHMim0zdyCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709767450; c=relaxed/simple; bh=mKgbYi6aKClFKalm/ggPSZY+PZgsfaGcdWEd0TgUmHA=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=lMxj+ckFugij5VpzUBT5mCBroPls0CfX4rKPQ800bn1EVcAe1Kt8JuiflPWB6txBph8xCdhngg5R9Jf5pPKnGHEFfTWDdeDCP2Ckl1VB5aQXur4mtElu9rl0ZVwIqjqaBT8zDSNC46xbj/um35u7iUfkt5+B5CHehf4mJ6IkYO0= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1dc09556599so2666995ad.1 for ; Wed, 06 Mar 2024 15:24:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709767445; x=1710372245; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g+OqC+NOBR73cVhATX39ej/tEaPAzhrUTdBuxxIZ4cI=; b=UAqmJvqgh/yyO+ae97cbf8DCpiYMll4e5dTlYLaN4Nh7LbPaDa2jbE1pEqZwLQ7Af7 5lqqeyjwQzU4lGEa07pQ2qJao53kyPDkCcYmERkqBBCjGnFLqescLmXehFYhrO1oAnbz TH2mz9s2r9gjEbEFruraiPBsQoeqOVoouUje2Jf35Z5yC0L2RaE4UM2mEc/4mJfmD5jF mq5xvqyhBOeKltqISG01kbqoJluxDYm8Q2Ie8wI2zi5FJ+M7/lsZWgZJEETINwLb80AA /eCZom7dwDd/2K4mzs9qZhLDxpkwyL+l7TQVtft9qXHs+rYUHf4cDw8858/Q6kXBPVdQ mq7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709767445; x=1710372245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g+OqC+NOBR73cVhATX39ej/tEaPAzhrUTdBuxxIZ4cI=; b=J4sme7245F+6eidGQq8Efpwhzb7Yr8mWYWb/bBrP/Tf/C3pSO4cJqYGyAi5TP9u7Hs V6v0lSxjPDYVAhNz9njGm+iUBFZtK37QkvYPAVs8tLFpmo18axNyPRxRQ53AmBU51wuF ffyAB0TIXz7T6x4mjrkizhUocEXIeXNjxPRDbTFqLda7a3XuqCdCMMrIJuWGqymQ2UjM XAj0iI72dqKyEQ4+qn6cD1K5TjgKArmnrxdTPlIMlD+DnD8NSkO5KYUFF8L/Q9rqO5At /i0wDp4yopgByVFDTv+5O1D68ItrheCbmD9crJIEYvodOHvbrqbEe+BmpltacSz+eLf1 He5g== X-Gm-Message-State: AOJu0YwiRou3M/H6Lgoic9jeUaoWJeKNGe3pXyKSMwzsDIT/a1Ckh5oZ xIV1PRMf2KWTBVeHE3QE9qYcdGxQAADKvBBxurRCNRdXnzqZDvpg46Q3zaeZ X-Google-Smtp-Source: AGHT+IHzabqZ9VYTIavWiXTGUwpc+PV+jOEGGT2ecnJGvnA5o63D5pwEhP3CBXf/g+fTMUPXFV2ZPQ== X-Received: by 2002:a17:902:c081:b0:1d9:14fb:d142 with SMTP id j1-20020a170902c08100b001d914fbd142mr6085696pld.32.1709767445255; Wed, 06 Mar 2024 15:24:05 -0800 (PST) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id jb5-20020a170903258500b001dcc29b985csm13166765plb.134.2024.03.06.15.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Mar 2024 15:24:02 -0800 (PST) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id C9CB07406C5; Wed, 6 Mar 2024 15:24:01 -0800 (PST) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, amodra@gmail.com Subject: [PATCH v4 3/6] elf: Use mmap to map in read-only sections Date: Wed, 6 Mar 2024 15:23:58 -0800 Message-ID: <20240306232401.1408530-4-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240306232401.1408530-1-hjl.tools@gmail.com> References: <20240306232401.1408530-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: There are many linker input files in LLVM debug build with huge string sections. All these string sections can be treated as read-only. But linker copies all of them into memory which consumes huge amount of memory and slows down linker significantly. Change the size type in the BFD mmap interface from bfd_size_type to size_t to be consistent with the size type of the host mmap interface. Add _bfd_mmap_readonly_tracked and _bfd_mmap_readonly_untracked to mmap in reado-only sections with size >= 4 * page size. Change setup_group to map in the group section as read-only and allocate the memory for internal use separately. NB: All string sections in valid ELF inputs must be null terminated. There is no need to terminate it again and string sections are mmapped as read-only. * bfd.c (bfd_mmapped_entry): New. (bfd_mmapped): Likewise. (bfd): Add mmapped. * bfdio.c (bfd_iovec): Change the bmmap size type to size_t. (bfd_mmap): Likewise. (memory_bmmap): Likewise. * bfdwin.c (bfd_get_file_window): Use _bfd_pagesize. * cache.c (cache_bmmap): Change the bmmap size type to size_t. Remove pagesize_m1 and use pagesize_m1 instead. * elf.c (bfd_elf_get_str_section): Call _bfd_mmap_readonly_tracked instead of _bfd_alloc_and_read. Don't terminate the string section again. (setup_group): Call _bfd_mmap_readonly_tracked instead of _bfd_alloc_and_read to map in the group section as read-only and allocate the memory for internal use separately. (get_hash_table_data): Call _bfd_mmap_readonly_untracked and _bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read and free. (_bfd_elf_get_dynamic_symbols): Call _bfd_mmap_readonly_tracked instead of _bfd_alloc_and_read. Don't terminate the string section again. Call _bfd_mmap_readonly_untracked and _bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read and free. (_bfd_elf_slurp_version_tables): Call _bfd_mmap_readonly_untracked and _bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read and free. * elflink.c (bfd_elf_link_record_dynamic_symbol): Use bfd_malloc to get the unversioned symbol. * libbfd-in.h (_bfd_pagesize): New. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (_bfd_mmap_readonly_tracked): Likewise. (_bfd_mmap_readonly_untracked): Likewise. (_bfd_munmap_readonly_untracked): Likewise. * libbfd.c (MAP_ANONYMOUS): New. Define if not defined. (bfd_allocate_mmapped_page): New. (_bfd_mmap_readonly_untracked): Likewise. (_bfd_munmap_readonly_untracked): Likewise. (_bfd_mmap_readonly_tracked): Likewise. (_bfd_pagesize): Likewise. (_bfd_pagesize_m1): Likewise. (_bfd_minimum_mmap_size): Likewise. (bfd_init_pagesize): Likewise. * ynx-core.c (lynx_core_file_p): Use _bfd_pagesize. * opncls.c: Include . (_bfd_delete_bfd): Munmap tracked mmapped memories. (opncls_bmmap): Change the bmmap size type to size_t. * bfd-in2.h: Regenerated. * libbfd.h: Likewise. --- bfd/bfd-in2.h | 21 ++++++- bfd/bfd.c | 17 ++++++ bfd/bfdio.c | 16 ++--- bfd/bfdwin.c | 8 +-- bfd/cache.c | 11 ++-- bfd/elf.c | 117 +++++++++++++++++++++++++++--------- bfd/elflink.c | 16 ++--- bfd/libbfd-in.h | 29 +++++++++ bfd/libbfd.c | 157 ++++++++++++++++++++++++++++++++++++++++++++++++ bfd/libbfd.h | 33 +++++++++- bfd/lynx-core.c | 2 +- bfd/opncls.c | 20 +++++- 12 files changed, 382 insertions(+), 65 deletions(-) diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 79b24a7f6e9..544422a9522 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -1951,6 +1951,20 @@ struct bfd_build_id bfd_byte data[1]; }; +struct bfd_mmapped_entry + { + void *addr; + size_t size; + }; + +struct bfd_mmapped + { + struct bfd_mmapped *next; + unsigned int max_entry; + unsigned int next_entry; + struct bfd_mmapped_entry entries[1]; + }; + struct bfd { /* The filename the application opened the BFD with. */ @@ -2280,6 +2294,9 @@ struct bfd /* For input BFDs, the build ID, if the object has one. */ const struct bfd_build_id *build_id; + + /* For input BFDs, mmapped entries. */ + struct bfd_mmapped *mmapped; }; static inline const char * @@ -2768,9 +2785,9 @@ ufile_ptr bfd_get_size (bfd *abfd); ufile_ptr bfd_get_file_size (bfd *abfd); -void *bfd_mmap (bfd *abfd, void *addr, bfd_size_type len, +void *bfd_mmap (bfd *abfd, void *addr, size_t len, int prot, int flags, file_ptr offset, - void **map_addr, bfd_size_type *map_len) + void **map_addr, size_t *map_len) ATTRIBUTE_WARN_UNUSED_RESULT; time_t bfd_get_current_time (time_t now); diff --git a/bfd/bfd.c b/bfd/bfd.c index 71732a0f92b..a33f092a6b2 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -74,6 +74,20 @@ EXTERNAL . bfd_byte data[1]; . }; . +.struct bfd_mmapped_entry +. { +. void *addr; +. size_t size; +. }; +. +.struct bfd_mmapped +. { +. struct bfd_mmapped *next; +. unsigned int max_entry; +. unsigned int next_entry; +. struct bfd_mmapped_entry entries[1]; +. }; +. CODE_FRAGMENT .struct bfd @@ -406,6 +420,9 @@ CODE_FRAGMENT . . {* For input BFDs, the build ID, if the object has one. *} . const struct bfd_build_id *build_id; +. +. {* For input BFDs, mmapped entries. *} +. struct bfd_mmapped *mmapped; .}; . diff --git a/bfd/bfdio.c b/bfd/bfdio.c index ab3e28ea204..a6b88b55625 100644 --- a/bfd/bfdio.c +++ b/bfd/bfdio.c @@ -219,9 +219,9 @@ DESCRIPTION . Also write in MAP_ADDR the address of the page aligned buffer and in . MAP_LEN the size mapped (a page multiple). Use unmap with MAP_ADDR and . MAP_LEN to unmap. *} -. void *(*bmmap) (struct bfd *abfd, void *addr, bfd_size_type len, +. void *(*bmmap) (struct bfd *abfd, void *addr, size_t len, . int prot, int flags, file_ptr offset, -. void **map_addr, bfd_size_type *map_len); +. void **map_addr, size_t *map_len); .}; .extern const struct bfd_iovec _bfd_memory_iovec; @@ -638,9 +638,9 @@ FUNCTION bfd_mmap SYNOPSIS - void *bfd_mmap (bfd *abfd, void *addr, bfd_size_type len, + void *bfd_mmap (bfd *abfd, void *addr, size_t len, int prot, int flags, file_ptr offset, - void **map_addr, bfd_size_type *map_len) + void **map_addr, size_t *map_len) ATTRIBUTE_WARN_UNUSED_RESULT; DESCRIPTION @@ -651,9 +651,9 @@ DESCRIPTION */ void * -bfd_mmap (bfd *abfd, void *addr, bfd_size_type len, +bfd_mmap (bfd *abfd, void *addr, size_t len, int prot, int flags, file_ptr offset, - void **map_addr, bfd_size_type *map_len) + void **map_addr, size_t *map_len) { while (abfd->my_archive != NULL && !bfd_is_thin_archive (abfd->my_archive)) @@ -815,10 +815,10 @@ memory_bstat (bfd *abfd, struct stat *statbuf) static void * memory_bmmap (bfd *abfd ATTRIBUTE_UNUSED, void *addr ATTRIBUTE_UNUSED, - bfd_size_type len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED, + size_t len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED, int flags ATTRIBUTE_UNUSED, file_ptr offset ATTRIBUTE_UNUSED, void **map_addr ATTRIBUTE_UNUSED, - bfd_size_type *map_len ATTRIBUTE_UNUSED) + size_t *map_len ATTRIBUTE_UNUSED) { return (void *)-1; } diff --git a/bfd/bfdwin.c b/bfd/bfdwin.c index beb17398744..d9ce0d25e90 100644 --- a/bfd/bfdwin.c +++ b/bfd/bfdwin.c @@ -165,7 +165,7 @@ bfd_get_file_window (bfd *abfd, bool writable) { static int ok_to_map = 1; - static size_t pagesize; + size_t pagesize = _bfd_pagesize; bfd_window_internal *i = windowp->i; bfd_size_type size_to_alloc = size; @@ -175,12 +175,6 @@ bfd_get_file_window (bfd *abfd, windowp, windowp->data, (unsigned long) windowp->size, windowp->i, writable); - /* Make sure we know the page size, so we can be friendly to mmap. */ - if (pagesize == 0) - pagesize = getpagesize (); - if (pagesize == 0) - abort (); - if (i == NULL) { i = bfd_zmalloc (sizeof (bfd_window_internal)); diff --git a/bfd/cache.c b/bfd/cache.c index 4c00c00f8d5..e0bdb658c9e 100644 --- a/bfd/cache.c +++ b/bfd/cache.c @@ -482,12 +482,12 @@ cache_bstat (struct bfd *abfd, struct stat *sb) static void * cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, void *addr ATTRIBUTE_UNUSED, - bfd_size_type len ATTRIBUTE_UNUSED, + size_t len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED, int flags ATTRIBUTE_UNUSED, file_ptr offset ATTRIBUTE_UNUSED, void **map_addr ATTRIBUTE_UNUSED, - bfd_size_type *map_len ATTRIBUTE_UNUSED) + size_t *map_len ATTRIBUTE_UNUSED) { void *ret = (void *) -1; @@ -498,10 +498,10 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, #ifdef HAVE_MMAP else { - static uintptr_t pagesize_m1; + uintptr_t pagesize_m1 = _bfd_pagesize_m1; FILE *f; file_ptr pg_offset; - bfd_size_type pg_len; + size_t pg_len; f = bfd_cache_lookup (abfd, CACHE_NO_SEEK_ERROR); if (f == NULL) @@ -510,9 +510,6 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, return ret; } - if (pagesize_m1 == 0) - pagesize_m1 = getpagesize () - 1; - /* Align. */ pg_offset = offset & ~pagesize_m1; pg_len = (len + (offset - pg_offset) + pagesize_m1) & ~pagesize_m1; diff --git a/bfd/elf.c b/bfd/elf.c index 8bffd3c5141..1c905762704 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -289,16 +289,23 @@ bfd_elf_get_str_section (bfd *abfd, unsigned int shindex) in case the string table is not terminated. */ if (shstrtabsize + 1 <= 1 || bfd_seek (abfd, offset, SEEK_SET) != 0 - || (shstrtab = _bfd_alloc_and_read (abfd, shstrtabsize + 1, - shstrtabsize)) == NULL) + || (shstrtab + = _bfd_mmap_readonly_tracked (abfd, shstrtabsize)) == NULL) { /* Once we've failed to read it, make sure we don't keep trying. Otherwise, we'll keep allocating space for the string table over and over. */ i_shdrp[shindex]->sh_size = 0; } - else - shstrtab[shstrtabsize] = '\0'; + else if (shstrtab[shstrtabsize - 1] != '\0') + { + /* It is an error if a string table isn't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB(%pA): string table is corrupt"), + abfd, i_shdrp[shindex]->bfd_section); + return NULL; + } i_shdrp[shindex]->contents = shstrtab; } return (char *) shstrtab; @@ -653,8 +660,12 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect) if (IS_VALID_GROUP_SECTION_HEADER (shdr, 2 * GRP_ENTRY_SIZE)) { - unsigned char *src; + unsigned char *src, *contents; Elf_Internal_Group *dest; + size_t contents_size; +#ifdef USE_MMAP + void *contents_addr = NULL; +#endif /* Make sure the group section has a BFD section attached to it. */ @@ -667,12 +678,23 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect) /* Read the raw contents. */ BFD_ASSERT (sizeof (*dest) >= 4 && sizeof (*dest) % 4 == 0); + contents_size = shdr->sh_size; shdr->contents = NULL; - if (_bfd_mul_overflow (shdr->sh_size, + if (_bfd_mul_overflow (contents_size, sizeof (*dest) / 4, &amt) || bfd_seek (abfd, shdr->sh_offset, SEEK_SET) != 0 - || !(shdr->contents - = _bfd_alloc_and_read (abfd, amt, shdr->sh_size))) +#ifdef USE_MMAP + || !(contents + = _bfd_mmap_readonly_untracked (abfd, + contents_size, + &contents_addr, + &contents_size)) +#else + || !(contents + = _bfd_alloc_and_read (abfd, amt, + contents_size)) +#endif + ) { _bfd_error_handler /* xgettext:c-format */ @@ -684,11 +706,17 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect) continue; } +#ifdef USE_MMAP + shdr->contents = bfd_alloc (abfd, amt); +#else + shdr->contents = contents; +#endif + /* Translate raw contents, a flag word followed by an array of elf section indices all in target byte order, to the flag word followed by an array of elf section pointers. */ - src = shdr->contents + shdr->sh_size; + src = contents + shdr->sh_size; dest = (Elf_Internal_Group *) (shdr->contents + amt); while (1) @@ -698,7 +726,7 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect) src -= 4; --dest; idx = H_GET_32 (abfd, src); - if (src == shdr->contents) + if (src == contents) { dest->shdr = NULL; dest->flags = idx; @@ -726,6 +754,11 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect) dest->shdr = NULL; } } + +#ifdef USE_MMAP + _bfd_munmap_readonly_untracked (contents_addr, + contents_size); +#endif } } @@ -1919,6 +1952,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, unsigned char *e_data = NULL; bfd_vma *i_data = NULL; bfd_size_type size; + void *e_data_addr; + size_t e_data_size ATTRIBUTE_UNUSED; if (ent_size != 4 && ent_size != 8) return NULL; @@ -1940,7 +1975,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, return NULL; } - e_data = _bfd_malloc_and_read (abfd, size, size); + e_data = _bfd_mmap_readonly_untracked (abfd, size, &e_data_addr, + &e_data_size); if (e_data == NULL) return NULL; @@ -1958,7 +1994,7 @@ get_hash_table_data (bfd *abfd, bfd_size_type number, while (number--) i_data[number] = bfd_get_64 (abfd, e_data + number * ent_size); - free (e_data); + _bfd_munmap_readonly_untracked (e_data_addr, e_data_size); return i_data; } @@ -2007,6 +2043,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, size_t verneed_size = 0; size_t extsym_size; const struct elf_backend_data *bed; + void *dynbuf_addr = NULL; + void *esymbuf_addr = NULL; + size_t dynbuf_size = 0; + size_t esymbuf_size = 0; /* Return TRUE if symbol table is bad. */ if (elf_bad_symtab (abfd)) @@ -2024,7 +2064,9 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (bfd_seek (abfd, phdr->p_offset, SEEK_SET) != 0) goto error_return; - dynbuf = _bfd_malloc_and_read (abfd, phdr->p_filesz, phdr->p_filesz); + dynbuf_size = phdr->p_filesz; + dynbuf = _bfd_mmap_readonly_untracked (abfd, dynbuf_size, + &dynbuf_addr, &dynbuf_size); if (dynbuf == NULL) goto error_return; @@ -2102,11 +2144,17 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* Dynamic string table must be valid until ABFD is closed. */ - strbuf = (char *) _bfd_alloc_and_read (abfd, dt_strsz + 1, dt_strsz); + strbuf = (char *) _bfd_mmap_readonly_tracked (abfd, dt_strsz); if (strbuf == NULL) goto error_return; - /* Since this is a string table, make sure that it is terminated. */ - strbuf[dt_strsz] = 0; + if (strbuf[dt_strsz - 1] != 0) + { + /* It is an error if a string table is't terminated. */ + _bfd_error_handler + /* xgettext:c-format */ + (_("%pB: DT_STRTAB table is corrupt"), abfd); + goto error_return; + } /* Get the real symbol count from DT_HASH or DT_GNU_HASH. Prefer DT_HASH since it is simpler than DT_GNU_HASH. */ @@ -2281,7 +2329,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, if (filepos == (file_ptr) -1 || bfd_seek (abfd, filepos, SEEK_SET) != 0) goto error_return; - esymbuf = _bfd_malloc_and_read (abfd, amt, amt); + esymbuf_size = amt; + esymbuf = _bfd_mmap_readonly_untracked (abfd, esymbuf_size, + &esymbuf_addr, + &esymbuf_size); if (esymbuf == NULL) goto error_return; @@ -2325,7 +2376,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERSYM info must be valid until ABFD is closed. */ - versym = _bfd_alloc_and_read (abfd, amt, amt); + versym = _bfd_mmap_readonly_tracked (abfd, amt); if (dt_verdef) { @@ -2337,8 +2388,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERDEF info must be valid until ABFD is closed. */ - verdef = _bfd_alloc_and_read (abfd, verdef_size, - verdef_size); + verdef = _bfd_mmap_readonly_tracked (abfd, verdef_size); } if (dt_verneed) @@ -2351,8 +2401,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, goto error_return; /* DT_VERNEED info must be valid until ABFD is closed. */ - verneed = _bfd_alloc_and_read (abfd, verneed_size, - verneed_size); + verneed = _bfd_mmap_readonly_tracked (abfd, verneed_size); } } @@ -2375,8 +2424,8 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr, /* Restore file position for elf_object_p. */ if (bfd_seek (abfd, saved_filepos, SEEK_SET) != 0) res = false; - free (dynbuf); - free (esymbuf); + _bfd_munmap_readonly_untracked (dynbuf_addr, dynbuf_size); + _bfd_munmap_readonly_untracked (esymbuf_addr, esymbuf_size); free (gnubuckets); free (gnuchains); free (mipsxlat); @@ -9435,6 +9484,8 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) bfd_byte *contents = NULL; unsigned int freeidx = 0; size_t amt; + void *contents_addr = NULL; + size_t contents_size = 0; if (elf_dynverref (abfd) != 0 || elf_tdata (abfd)->dt_verneed != NULL) { @@ -9471,7 +9522,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verref; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_untracked (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verref; @@ -9604,8 +9658,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) elf_tdata (abfd)->cverrefs = i; if (contents != elf_tdata (abfd)->dt_verneed) - free (contents); + _bfd_munmap_readonly_untracked (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } if (elf_dynverdef (abfd) != 0 || elf_tdata (abfd)->dt_verdef != NULL) @@ -9646,7 +9701,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0) goto error_return_verdef; - contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size); + contents_size = hdr->sh_size; + contents = _bfd_mmap_readonly_untracked (abfd, contents_size, + &contents_addr, + &contents_size); if (contents == NULL) goto error_return_verdef; @@ -9800,8 +9858,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) } if (contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_untracked (contents_addr, contents_size); contents = NULL; + contents_addr = NULL; } else if (default_imported_symver) { @@ -9857,7 +9916,7 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver) error_return: if (contents != elf_tdata (abfd)->dt_verneed && contents != elf_tdata (abfd)->dt_verdef) - free (contents); + _bfd_munmap_readonly_untracked (contents_addr, contents_size); return false; } diff --git a/bfd/elflink.c b/bfd/elflink.c index 5a6cb07b2ce..42029f29f7a 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -549,22 +549,24 @@ bfd_elf_link_record_dynamic_symbol (struct bfd_link_info *info, return false; } + char *unversioned_name = NULL; + /* We don't put any version information in the dynamic string table. */ name = h->root.root.string; p = strchr (name, ELF_VER_CHR); if (p != NULL) - /* We know that the p points into writable memory. In fact, - there are only a few symbols that have read-only names, being - those like _GLOBAL_OFFSET_TABLE_ that are created specially - by the backends. Most symbols will have names pointing into - an ELF string table read from a file, or to objalloc memory. */ - *p = 0; + { + unversioned_name = bfd_malloc (p - name + 1); + memcpy (unversioned_name, name, p - name); + unversioned_name[p - name] = 0; + name = unversioned_name; + } indx = _bfd_elf_strtab_add (dynstr, name, p != NULL); if (p != NULL) - *p = ELF_VER_CHR; + free (unversioned_name); if (indx == (size_t) -1) return false; diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h index b8b2ce7ba09..7887fad9c92 100644 --- a/bfd/libbfd-in.h +++ b/bfd/libbfd-in.h @@ -851,6 +851,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -888,6 +892,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_tracked + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_untracked + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_untracked + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_tracked(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -910,3 +927,15 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif diff --git a/bfd/libbfd.c b/bfd/libbfd.c index f8d148c9677..237f91e5c97 100644 --- a/bfd/libbfd.c +++ b/bfd/libbfd.c @@ -1038,6 +1038,147 @@ bfd_get_bits (const void *p, int bits, bool big_p) return data; } +#ifdef USE_MMAP +#include + +#ifndef MAP_ANONYMOUS +#define MAP_ANONYMOUS MAP_ANON +#endif + +/* Allocate a page to track mmapped memory and return the page and + the first entry. Return NULL if mmap fails. */ + +static struct bfd_mmapped * +bfd_allocate_mmapped_page (bfd *abfd, struct bfd_mmapped_entry **entry) +{ + struct bfd_mmapped * mmapped + = (struct bfd_mmapped *) mmap (NULL, _bfd_pagesize, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + -1, 0); + if (mmapped == MAP_FAILED) + return NULL; + + mmapped->next = abfd->mmapped; + mmapped->max_entry + = ((_bfd_pagesize - offsetof (struct bfd_mmapped, entries)) + / sizeof (struct bfd_mmapped_entry)); + mmapped->next_entry = 1; + abfd->mmapped = mmapped; + *entry = mmapped->entries; + return mmapped; +} + +/* Mmap a memory region of RSIZE bytes with PROT at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and (void *) -1 for mmap failure. */ + +static void * +bfd_mmap_local (bfd *abfd, size_t rsize, int prot, void **map_addr, + size_t *map_size) +{ + if (!_bfd_constant_p (rsize)) + { + ufile_ptr filesize = bfd_get_file_size (abfd); + if (filesize != 0 && rsize > filesize) + { + bfd_set_error (bfd_error_file_truncated); + return NULL; + } + } + + void *mem; + ufile_ptr offset = bfd_tell (abfd); + mem = bfd_mmap (abfd, NULL, rsize, prot, MAP_PRIVATE, offset, + map_addr, map_size); + return mem; +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL + on invalid input and (void *) -1 for mmap failure. */ + +void * +_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + { + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + /* NB: Set *MAP_ADDR to MEM and *MAP_SIZE to 0 to indicate that + _bfd_malloc_and_read is called. */ + *map_addr = mem; + *map_size = 0; + return mem; + } + + return bfd_mmap_local (abfd, rsize, PROT_READ, map_addr, map_size); +} + +/* Munmap RSIZE bytes at PTR. */ + +void +_bfd_munmap_readonly_untracked (void *ptr, size_t rsize) +{ + /* NB: Since _bfd_munmap_readonly_untracked is called like free, PTR + may be NULL. Otherwise, PTR and RSIZE must be valid. If RSIZE is + 0, _bfd_malloc_and_read is called. */ + if (ptr == NULL) + return; + if (rsize != 0) + { + if (munmap (ptr, rsize) != 0) + abort (); + } + else + free (ptr); +} + +/* Mmap a readonly memory region of RSIZE bytes at the current offset. + Return NULL on invalid input or mmap failure. */ + +void * +_bfd_mmap_readonly_tracked (bfd *abfd, size_t rsize) +{ + /* Use mmap only if section size >= the minimum mmap section size. */ + if (rsize < _bfd_minimum_mmap_size) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + void *mem, *map_addr; + size_t map_size; + mem = bfd_mmap_local (abfd, rsize, PROT_READ, &map_addr, &map_size); + if (mem == NULL) + return mem; + if (mem == (void *) -1) + return _bfd_alloc_and_read (abfd, rsize, rsize); + + struct bfd_mmapped_entry *entry; + unsigned int next_entry; + struct bfd_mmapped *mmapped = abfd->mmapped; + if (mmapped != NULL + && (next_entry = mmapped->next_entry) < mmapped->max_entry) + { + entry = &mmapped->entries[next_entry]; + mmapped->next_entry++; + } + else + { + mmapped = bfd_allocate_mmapped_page (abfd, &entry); + if (mmapped == NULL) + { + munmap (map_addr, map_size); + return NULL; + } + } + + entry->addr = map_addr; + entry->size = map_size; + + return mem; +} +#endif + /* Default implementation */ bool @@ -1326,3 +1467,19 @@ _bfd_generic_init_private_section_data (bfd *ibfd ATTRIBUTE_UNUSED, { return true; } + +uintptr_t _bfd_pagesize; +uintptr_t _bfd_pagesize_m1; +uintptr_t _bfd_minimum_mmap_size; + +__attribute__ ((unused, constructor)) +static void +bfd_init_pagesize (void) +{ + _bfd_pagesize = getpagesize (); + if (_bfd_pagesize == 0) + abort (); + _bfd_pagesize_m1 = _bfd_pagesize - 1; + /* The minimum section size to use mmap. */ + _bfd_minimum_mmap_size = _bfd_pagesize * 4; +} diff --git a/bfd/libbfd.h b/bfd/libbfd.h index a60063b5ae1..1515a03b093 100644 --- a/bfd/libbfd.h +++ b/bfd/libbfd.h @@ -857,6 +857,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *) extern bool _bfd_link_keep_memory (struct bfd_link_info *) ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN; +extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN; + #if GCC_VERSION >= 7000 #define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res) #else @@ -894,6 +898,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) return NULL; } +#ifdef USE_MMAP +extern void *_bfd_mmap_readonly_tracked + (bfd *, size_t) ATTRIBUTE_HIDDEN; +extern void *_bfd_mmap_readonly_untracked + (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN; +extern void _bfd_munmap_readonly_untracked + (void *, size_t) ATTRIBUTE_HIDDEN; +#else +#define _bfd_mmap_readonly_tracked(abfd, rsize) \ + _bfd_alloc_and_read (abfd, rsize, rsize) +#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr) +#endif + static inline void * _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) { @@ -916,6 +933,18 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize) } return NULL; } + +#ifndef USE_MMAP +static inline void * +_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr, + size_t *map_size) +{ + void *mem = _bfd_malloc_and_read (abfd, rsize, rsize); + *map_addr = mem; + *map_size = rsize; + return mem; +} +#endif /* Extracted from libbfd.c. */ void *bfd_malloc (bfd_size_type /*size*/) ATTRIBUTE_HIDDEN; @@ -969,9 +998,9 @@ struct bfd_iovec Also write in MAP_ADDR the address of the page aligned buffer and in MAP_LEN the size mapped (a page multiple). Use unmap with MAP_ADDR and MAP_LEN to unmap. */ - void *(*bmmap) (struct bfd *abfd, void *addr, bfd_size_type len, + void *(*bmmap) (struct bfd *abfd, void *addr, size_t len, int prot, int flags, file_ptr offset, - void **map_addr, bfd_size_type *map_len); + void **map_addr, size_t *map_len); }; extern const struct bfd_iovec _bfd_memory_iovec; diff --git a/bfd/lynx-core.c b/bfd/lynx-core.c index 44d94ad8745..9ec5a0d2028 100644 --- a/bfd/lynx-core.c +++ b/bfd/lynx-core.c @@ -96,7 +96,7 @@ lynx_core_file_p (bfd *abfd) asection *newsect; size_t amt; - pagesize = getpagesize (); /* Serious cross-target issue here... This + pagesize = _bfd_pagesize; /* Serious cross-target issue here... This really needs to come from a system-specific header file. */ diff --git a/bfd/opncls.c b/bfd/opncls.c index a0a5c40fba8..ac74607bac0 100644 --- a/bfd/opncls.c +++ b/bfd/opncls.c @@ -27,6 +27,10 @@ #include "libiberty.h" #include "elf-bfd.h" +#ifdef USE_MMAP +#include +#endif + #ifndef S_IXUSR #define S_IXUSR 0100 /* Execute by owner. */ #endif @@ -163,6 +167,18 @@ _bfd_new_bfd_contained_in (bfd *obfd) static void _bfd_delete_bfd (bfd *abfd) { +#ifdef USE_MMAP + struct bfd_mmapped *mmapped, *next; + for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next) + { + struct bfd_mmapped_entry *entries = mmapped->entries; + next = mmapped->next; + for (unsigned int i = 0; i < mmapped->next_entry; i++) + munmap (entries[i].addr, entries[i].size); + munmap (mmapped, _bfd_pagesize); + } +#endif + /* Give the target _bfd_free_cached_info a chance to free memory. */ if (abfd->memory && abfd->xvec) bfd_free_cached_info (abfd); @@ -668,12 +684,12 @@ opncls_bstat (struct bfd *abfd, struct stat *sb) static void * opncls_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED, void *addr ATTRIBUTE_UNUSED, - bfd_size_type len ATTRIBUTE_UNUSED, + size_t len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED, int flags ATTRIBUTE_UNUSED, file_ptr offset ATTRIBUTE_UNUSED, void **map_addr ATTRIBUTE_UNUSED, - bfd_size_type *map_len ATTRIBUTE_UNUSED) + size_t *map_len ATTRIBUTE_UNUSED) { return (void *) -1; } -- 2.44.0