* [PATCH v3 0/6] elf: Use mmap to map in section contents
@ 2024-03-06 20:47 H.J. Lu
2024-03-06 20:47 ` [PATCH v3 1/6] bfd: Don't hard-code BFD_JUMP_TABLE_COPY H.J. Lu
` (6 more replies)
0 siblings, 7 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
Changes in v3:
1. Fix non-mmap build.
2. Change the argument name of bfd_mmap_local from flags to prot since
its values are PROT_XXX.
Changes in v2:
1. Don't hard-code BFD_JUMP_TABLE_COPY in bfd so that elf-bfd.h can be
included in libbfd.c.
2. Change the --with-mmap default to true.
3. Check USE_MMAP instead of HAVE_MMAP.
4. Remove the asize parameter to _bfd_mmap_readonly_tracked.
5. Add contents_addr and contents_size to bfd_elf_section_data.
6. Rename _bfd_link_keep_memory to _bfd_elf_link_keep_memory.
---
We can use mmap to map in ELF section contents, instead of copying them
into memory by hand. We don't need to cache symbol nor relocation tables
if they are mapped in. Data to link the 3.5GB clang executable in LLVM
17 debug build on Linux/x86-64 with 32GB RAM is:
stdio mmap improvement
user 86.73 87.02 -0.3%
system 9.55 9.21 3.6%
total 100.40 97.66 0.7%
maximum set(GB) 17.34 13.14 24%
page faults 4047667 3042877 25%
and data to link the 275M cc1plus executable in GCC 14 stage 1 build is:
user 5.41 5.44 -0.5%
system 0.80 0.76 5%
total 6.25 6.26 -0.2%
maximum set(MB) 1323 968 27%
page faults 323451 236371 27%
Data shows that these won't improve the single copy linker performance.
But they improve the overall system performance when linker is used by
reducing linker memory usage and page faults. They allow more parallel
linker jobs on LLVM debug build.
Here is a quote from Noah Goldstein: "on a large project they are an
extremely large speedup".
H.J. Lu (6):
bfd: Don't hard-code BFD_JUMP_TABLE_COPY
bfd: Change the --with-mmap default to true
elf: Use mmap to map in read-only sections
elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents
elf: Use mmap to map in symbol and relocation tables
elf: Don't cache symbol nor relocation tables with mmap
bfd/aout-target.h | 3 +
bfd/bfd-in2.h | 30 +++++-
bfd/bfd.c | 17 +++
bfd/bfdio.c | 16 +--
bfd/bfdwin.c | 8 +-
bfd/cache.c | 11 +-
bfd/coff-rs6000.c | 2 +
bfd/coffcode.h | 2 +
bfd/compress.c | 2 +-
bfd/configure | 2 +-
bfd/configure.ac | 2 +-
bfd/elf-bfd.h | 25 ++++-
bfd/elf-eh-frame.c | 4 +-
bfd/elf-sframe.c | 4 +-
bfd/elf.c | 253 ++++++++++++++++++++++++++++++++++++--------
bfd/elf32-i386.c | 8 +-
bfd/elf64-x86-64.c | 12 +--
bfd/elfcode.h | 7 +-
bfd/elflink.c | 134 ++++++++++++++++-------
bfd/elfxx-target.h | 8 +-
bfd/elfxx-x86.c | 7 +-
bfd/elfxx-x86.h | 1 +
bfd/libbfd-in.h | 33 +++++-
bfd/libbfd.c | 249 ++++++++++++++++++++++++++++++++++++++++++-
bfd/libbfd.h | 37 ++++++-
bfd/libecoff.h | 3 +
bfd/linker.c | 35 ------
bfd/lynx-core.c | 2 +-
bfd/mach-o-target.c | 1 +
bfd/mmo.c | 1 +
bfd/opncls.c | 29 ++++-
bfd/plugin.c | 1 +
bfd/ppcboot.c | 1 +
bfd/section.c | 9 +-
bfd/som.c | 1 +
bfd/targets.c | 2 +-
bfd/vms-alpha.c | 1 +
37 files changed, 776 insertions(+), 187 deletions(-)
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 1/6] bfd: Don't hard-code BFD_JUMP_TABLE_COPY
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 20:47 ` [PATCH v3 2/6] bfd: Change the --with-mmap default to true H.J. Lu
` (5 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
In BFD_JUMP_TABLE_COPY, replace _bfd_generic_init_private_section_data
with NAME##_init_private_section_data so that ELF targets can properly
replace it with _bfd_elf_init_private_section_data.
* aout-target.h (MY_init_private_section_data): New.
* coff-rs6000.c (_bfd_xcoff_init_private_section_data): New.
* coffcode.h (coff_init_private_section_data): New.
* elfxx-target.h (bfd_elfNN_init_private_section_data): New.
* libecoff.h (_bfd_ecoff_init_private_section_data): New.
* mach-o-target.c (bfd_mach_o_init_private_section_data): New.
* mmo.c (mmo_init_private_section_data): New.
* plugin.c (bfd_plugin_init_private_section_data): New.
* ppcboot.c (ppcboot_init_private_section_data): New.
* som.c (som_init_private_section_data): New.
* targets.c (BFD_JUMP_TABLE_COPY): Replace
_bfd_generic_init_private_section_data with
NAME##_init_private_section_data.
* vms-alpha.c (vms_init_private_section_data): New.
* elf-bfd.h (_bfd_generic_init_private_section_data): Removed.
* bfd-in2.h: Regenerated.
---
bfd/aout-target.h | 3 +++
bfd/bfd-in2.h | 2 +-
bfd/coff-rs6000.c | 2 ++
bfd/coffcode.h | 2 ++
bfd/elf-bfd.h | 2 --
bfd/elfxx-target.h | 2 ++
bfd/libecoff.h | 3 +++
bfd/mach-o-target.c | 1 +
bfd/mmo.c | 1 +
bfd/plugin.c | 1 +
bfd/ppcboot.c | 1 +
bfd/som.c | 1 +
bfd/targets.c | 2 +-
bfd/vms-alpha.c | 1 +
14 files changed, 20 insertions(+), 4 deletions(-)
diff --git a/bfd/aout-target.h b/bfd/aout-target.h
index 4a05c602435..47fe49fe04a 100644
--- a/bfd/aout-target.h
+++ b/bfd/aout-target.h
@@ -569,6 +569,9 @@ MY_bfd_final_link (bfd *abfd, struct bfd_link_info *info)
#define MY_bfd_merge_private_bfd_data _bfd_generic_bfd_merge_private_bfd_data
#endif
+#define MY_init_private_section_data \
+ _bfd_generic_init_private_section_data
+
#ifndef MY_bfd_copy_private_symbol_data
#define MY_bfd_copy_private_symbol_data _bfd_generic_bfd_copy_private_symbol_data
#endif
diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h
index a335df522d2..79b24a7f6e9 100644
--- a/bfd/bfd-in2.h
+++ b/bfd/bfd-in2.h
@@ -7685,7 +7685,7 @@ typedef struct bfd_target
#define BFD_JUMP_TABLE_COPY(NAME) \
NAME##_bfd_copy_private_bfd_data, \
NAME##_bfd_merge_private_bfd_data, \
- _bfd_generic_init_private_section_data, \
+ NAME##_init_private_section_data, \
NAME##_bfd_copy_private_section_data, \
NAME##_bfd_copy_private_symbol_data, \
NAME##_bfd_copy_private_header_data, \
diff --git a/bfd/coff-rs6000.c b/bfd/coff-rs6000.c
index 87feb672bf1..7d04739c8a9 100644
--- a/bfd/coff-rs6000.c
+++ b/bfd/coff-rs6000.c
@@ -4471,6 +4471,8 @@ const struct xcoff_dwsect_name xcoff_dwsect_names[] = {
_bfd_generic_get_section_contents_in_window
/* For copy private data entry points. */
+#define _bfd_xcoff_init_private_section_data \
+ _bfd_generic_init_private_section_data
#define _bfd_xcoff_bfd_copy_private_bfd_data \
_bfd_xcoff_copy_private_bfd_data
#define _bfd_xcoff_bfd_merge_private_bfd_data \
diff --git a/bfd/coffcode.h b/bfd/coffcode.h
index 4170b630b4d..ebf4f513fde 100644
--- a/bfd/coffcode.h
+++ b/bfd/coffcode.h
@@ -5968,6 +5968,8 @@ static const bfd_coff_backend_data bigobj_swap_table =
#define coff_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
#endif
+#define coff_init_private_section_data _bfd_generic_init_private_section_data
+
#ifndef coff_bfd_copy_private_section_data
#define coff_bfd_copy_private_section_data _bfd_generic_bfd_copy_private_section_data
#endif
diff --git a/bfd/elf-bfd.h b/bfd/elf-bfd.h
index 3ed22fa6c52..c5d325435b6 100644
--- a/bfd/elf-bfd.h
+++ b/bfd/elf-bfd.h
@@ -2359,8 +2359,6 @@ extern bool _bfd_elf_copy_private_header_data
(bfd *, bfd *);
extern bool _bfd_elf_copy_private_symbol_data
(bfd *, asymbol *, bfd *, asymbol *);
-#define _bfd_generic_init_private_section_data \
- _bfd_elf_init_private_section_data
extern bool _bfd_elf_init_private_section_data
(bfd *, asection *, bfd *, asection *, struct bfd_link_info *);
extern bool _bfd_elf_copy_private_section_data
diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h
index a7f2fc6e320..1e6992b5793 100644
--- a/bfd/elfxx-target.h
+++ b/bfd/elfxx-target.h
@@ -230,6 +230,8 @@
_bfd_elf_copy_private_symbol_data
#endif
+#define bfd_elfNN_init_private_section_data \
+ _bfd_elf_init_private_section_data
#ifndef bfd_elfNN_bfd_copy_private_section_data
#define bfd_elfNN_bfd_copy_private_section_data \
_bfd_elf_copy_private_section_data
diff --git a/bfd/libecoff.h b/bfd/libecoff.h
index 5e6de3ccebf..50a0af7c7a2 100644
--- a/bfd/libecoff.h
+++ b/bfd/libecoff.h
@@ -250,6 +250,9 @@ extern bool _bfd_ecoff_get_section_contents
#define _bfd_ecoff_bfd_link_split_section _bfd_generic_link_split_section
#define _bfd_ecoff_bfd_link_check_relocs _bfd_generic_link_check_relocs
+#define _bfd_ecoff_init_private_section_data \
+ _bfd_generic_init_private_section_data
+
extern bool _bfd_ecoff_bfd_copy_private_bfd_data
(bfd *, bfd *);
#define _bfd_ecoff_bfd_copy_private_section_data \
diff --git a/bfd/mach-o-target.c b/bfd/mach-o-target.c
index 01666d3aa5c..6edb09f4347 100644
--- a/bfd/mach-o-target.c
+++ b/bfd/mach-o-target.c
@@ -75,6 +75,7 @@
#define bfd_mach_o_write_armap _bfd_noarchive_write_armap
#define bfd_mach_o_get_elt_at_index _bfd_noarchive_get_elt_at_index
#define bfd_mach_o_update_armap_timestamp _bfd_noarchive_update_armap_timestamp
+#define bfd_mach_o_init_private_section_data _bfd_generic_init_private_section_data
#define TARGET_NAME_BACKEND XCONCAT2(TARGET_NAME,_backend)
diff --git a/bfd/mmo.c b/bfd/mmo.c
index 485a1c3ca33..528de1d10e0 100644
--- a/bfd/mmo.c
+++ b/bfd/mmo.c
@@ -3360,6 +3360,7 @@ mmo_write_object_contents (bfd *abfd)
#define mmo_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
#define mmo_bfd_set_private_flags _bfd_generic_bfd_set_private_flags
#define mmo_bfd_print_private_bfd_data _bfd_generic_bfd_print_private_bfd_data
+#define mmo_init_private_section_data _bfd_generic_init_private_section_data
const bfd_target mmix_mmo_vec =
{
diff --git a/bfd/plugin.c b/bfd/plugin.c
index 784a3c1fd9d..8226c264733 100644
--- a/bfd/plugin.c
+++ b/bfd/plugin.c
@@ -74,6 +74,7 @@ dlerror (void)
#define bfd_plugin_new_section_hook _bfd_generic_new_section_hook
#define bfd_plugin_get_section_contents _bfd_generic_get_section_contents
#define bfd_plugin_get_section_contents_in_window _bfd_generic_get_section_contents_in_window
+#define bfd_plugin_init_private_section_data _bfd_generic_init_private_section_data
#define bfd_plugin_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
#define bfd_plugin_bfd_merge_private_bfd_data _bfd_generic_bfd_merge_private_bfd_data
#define bfd_plugin_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
diff --git a/bfd/ppcboot.c b/bfd/ppcboot.c
index 461050c668b..2e947779885 100644
--- a/bfd/ppcboot.c
+++ b/bfd/ppcboot.c
@@ -476,6 +476,7 @@ ppcboot_bfd_print_private_bfd_data (bfd *abfd, void * farg)
#define ppcboot_bfd_copy_private_bfd_data _bfd_generic_bfd_copy_private_bfd_data
#define ppcboot_bfd_merge_private_bfd_data _bfd_generic_bfd_merge_private_bfd_data
+#define ppcboot_init_private_section_data _bfd_generic_init_private_section_data
#define ppcboot_bfd_copy_private_section_data _bfd_generic_bfd_copy_private_section_data
#define ppcboot_bfd_copy_private_symbol_data _bfd_generic_bfd_copy_private_symbol_data
#define ppcboot_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
diff --git a/bfd/som.c b/bfd/som.c
index 17c9d713585..dc641e7857b 100644
--- a/bfd/som.c
+++ b/bfd/som.c
@@ -6782,6 +6782,7 @@ som_bfd_link_split_section (bfd *abfd ATTRIBUTE_UNUSED, asection *sec)
#define som_bfd_link_hide_symbol _bfd_generic_link_hide_symbol
#define som_bfd_define_start_stop bfd_generic_define_start_stop
#define som_bfd_merge_private_bfd_data _bfd_generic_bfd_merge_private_bfd_data
+#define som_init_private_section_data _bfd_generic_init_private_section_data
#define som_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
#define som_bfd_set_private_flags _bfd_generic_bfd_set_private_flags
#define som_find_inliner_info _bfd_nosymbols_find_inliner_info
diff --git a/bfd/targets.c b/bfd/targets.c
index 3addf2fe373..f94992d031d 100644
--- a/bfd/targets.c
+++ b/bfd/targets.c
@@ -285,7 +285,7 @@ BFD_JUMP_TABLE macros.
.#define BFD_JUMP_TABLE_COPY(NAME) \
. NAME##_bfd_copy_private_bfd_data, \
. NAME##_bfd_merge_private_bfd_data, \
-. _bfd_generic_init_private_section_data, \
+. NAME##_init_private_section_data, \
. NAME##_bfd_copy_private_section_data, \
. NAME##_bfd_copy_private_symbol_data, \
. NAME##_bfd_copy_private_header_data, \
diff --git a/bfd/vms-alpha.c b/bfd/vms-alpha.c
index f5ecdcb3e0c..ace5a24c631 100644
--- a/bfd/vms-alpha.c
+++ b/bfd/vms-alpha.c
@@ -10129,6 +10129,7 @@ bfd_vms_get_data (bfd *abfd)
#define vms_bfd_copy_private_bfd_data _bfd_generic_bfd_copy_private_bfd_data
#define vms_bfd_merge_private_bfd_data _bfd_generic_bfd_merge_private_bfd_data
+#define vms_init_private_section_data _bfd_generic_init_private_section_data
#define vms_bfd_copy_private_section_data _bfd_generic_bfd_copy_private_section_data
#define vms_bfd_copy_private_symbol_data _bfd_generic_bfd_copy_private_symbol_data
#define vms_bfd_copy_private_header_data _bfd_generic_bfd_copy_private_header_data
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 2/6] bfd: Change the --with-mmap default to true
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
2024-03-06 20:47 ` [PATCH v3 1/6] bfd: Don't hard-code BFD_JUMP_TABLE_COPY H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 20:47 ` [PATCH v3 3/6] elf: Use mmap to map in read-only sections H.J. Lu
` (4 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
Change the configure default to using mmap.
* configure.ac: Change the --with-mmap default to true.
* configure: Regenerated.
---
bfd/configure | 2 +-
bfd/configure.ac | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/bfd/configure b/bfd/configure
index 23ffa6da332..210550ae042 100755
--- a/bfd/configure
+++ b/bfd/configure
@@ -12018,7 +12018,7 @@ if test "${with_mmap+set}" = set; then :
*) as_fn_error $? "bad value ${withval} for BFD with-mmap option" "$LINENO" 5 ;;
esac
else
- want_mmap=false
+ want_mmap=true
fi
# Check whether --enable-secureplt was given.
diff --git a/bfd/configure.ac b/bfd/configure.ac
index 7fcc5d4a947..02d8ba96318 100644
--- a/bfd/configure.ac
+++ b/bfd/configure.ac
@@ -113,7 +113,7 @@ AC_ARG_WITH(mmap,
yes) want_mmap=true ;;
no) want_mmap=false ;;
*) AC_MSG_ERROR(bad value ${withval} for BFD with-mmap option) ;;
-esac],[want_mmap=false])dnl
+esac],[want_mmap=true])dnl
AC_ARG_ENABLE(secureplt,
[ --enable-secureplt Default to creating read-only plt entries],
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 3/6] elf: Use mmap to map in read-only sections
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
2024-03-06 20:47 ` [PATCH v3 1/6] bfd: Don't hard-code BFD_JUMP_TABLE_COPY H.J. Lu
2024-03-06 20:47 ` [PATCH v3 2/6] bfd: Change the --with-mmap default to true H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 20:47 ` [PATCH v3 4/6] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents H.J. Lu
` (3 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
There are many linker input files in LLVM debug build with huge string
sections. All these string sections can be treated as read-only. But
linker copies all of them into memory which consumes huge amount of
memory and slows down linker significantly.
Change the size type in the BFD mmap interface from bfd_size_type to
size_t to be consistent with the size type of the host mmap interface.
Add _bfd_mmap_readonly_tracked and _bfd_mmap_readonly_untracked to mmap
in reado-only sections with size >= 4 * page size.
Change setup_group to map in the group section as read-only and allocate
the memory for internal use separately.
NB: All string sections in valid ELF inputs must be null terminated.
There is no need to terminate it again and string sections are mmapped
as read-only.
* bfd.c (bfd_mmapped_entry): New.
(bfd_mmapped): Likewise.
(bfd): Add mmapped.
* bfdio.c (bfd_iovec): Change the bmmap size type to size_t.
(bfd_mmap): Likewise.
(memory_bmmap): Likewise.
* bfdwin.c (bfd_get_file_window): Use _bfd_pagesize.
* cache.c (cache_bmmap): Change the bmmap size type to size_t.
Remove pagesize_m1 and use pagesize_m1 instead.
* elf.c (bfd_elf_get_str_section): Call _bfd_mmap_readonly_tracked
instead of _bfd_alloc_and_read. Don't terminate the string
section again.
(setup_group): Call _bfd_mmap_readonly_tracked instead of
_bfd_alloc_and_read to map in the group section as read-only
and allocate the memory for internal use separately.
(get_hash_table_data): Call _bfd_mmap_readonly_untracked and
_bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read
and free.
(_bfd_elf_get_dynamic_symbols): Call _bfd_mmap_readonly_tracked
instead of _bfd_alloc_and_read. Don't terminate the string
section again. Call _bfd_mmap_readonly_untracked and
_bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read
and free.
(_bfd_elf_slurp_version_tables): Call _bfd_mmap_readonly_untracked
and _bfd_munmap_readonly_untracked instead of _bfd_malloc_and_read
and free.
* elflink.c (bfd_elf_link_record_dynamic_symbol): Use bfd_malloc
to get the unversioned symbol.
* libbfd-in.h (_bfd_pagesize): New.
(_bfd_pagesize_m1): Likewise.
(_bfd_minimum_mmap_size): Likewise.
(_bfd_mmap_readonly_tracked): Likewise.
(_bfd_mmap_readonly_untracked): Likewise.
(_bfd_munmap_readonly_untracked): Likewise.
* libbfd.c (MAP_ANONYMOUS): New. Define if not defined.
(bfd_allocate_mmapped_page): New.
(_bfd_mmap_readonly_untracked): Likewise.
(_bfd_munmap_readonly_untracked): Likewise.
(_bfd_mmap_readonly_tracked): Likewise.
(_bfd_pagesize): Likewise.
(_bfd_pagesize_m1): Likewise.
(_bfd_minimum_mmap_size): Likewise.
(bfd_init_pagesize): Likewise.
* ynx-core.c (lynx_core_file_p): Use _bfd_pagesize.
* opncls.c: Include <sys/mman.h>.
(_bfd_delete_bfd): Munmap tracked mmapped memories.
(opncls_bmmap): Change the bmmap size type to size_t.
* bfd-in2.h: Regenerated.
* libbfd.h: Likewise.
---
bfd/bfd-in2.h | 21 ++++++-
bfd/bfd.c | 17 ++++++
bfd/bfdio.c | 16 ++---
bfd/bfdwin.c | 8 +--
bfd/cache.c | 11 ++--
bfd/elf.c | 117 +++++++++++++++++++++++++++---------
bfd/elflink.c | 16 ++---
bfd/libbfd-in.h | 29 +++++++++
bfd/libbfd.c | 157 ++++++++++++++++++++++++++++++++++++++++++++++++
bfd/libbfd.h | 33 +++++++++-
bfd/lynx-core.c | 2 +-
bfd/opncls.c | 20 +++++-
12 files changed, 382 insertions(+), 65 deletions(-)
diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h
index 79b24a7f6e9..544422a9522 100644
--- a/bfd/bfd-in2.h
+++ b/bfd/bfd-in2.h
@@ -1951,6 +1951,20 @@ struct bfd_build_id
bfd_byte data[1];
};
+struct bfd_mmapped_entry
+ {
+ void *addr;
+ size_t size;
+ };
+
+struct bfd_mmapped
+ {
+ struct bfd_mmapped *next;
+ unsigned int max_entry;
+ unsigned int next_entry;
+ struct bfd_mmapped_entry entries[1];
+ };
+
struct bfd
{
/* The filename the application opened the BFD with. */
@@ -2280,6 +2294,9 @@ struct bfd
/* For input BFDs, the build ID, if the object has one. */
const struct bfd_build_id *build_id;
+
+ /* For input BFDs, mmapped entries. */
+ struct bfd_mmapped *mmapped;
};
static inline const char *
@@ -2768,9 +2785,9 @@ ufile_ptr bfd_get_size (bfd *abfd);
ufile_ptr bfd_get_file_size (bfd *abfd);
-void *bfd_mmap (bfd *abfd, void *addr, bfd_size_type len,
+void *bfd_mmap (bfd *abfd, void *addr, size_t len,
int prot, int flags, file_ptr offset,
- void **map_addr, bfd_size_type *map_len)
+ void **map_addr, size_t *map_len)
ATTRIBUTE_WARN_UNUSED_RESULT;
time_t bfd_get_current_time (time_t now);
diff --git a/bfd/bfd.c b/bfd/bfd.c
index 71732a0f92b..a33f092a6b2 100644
--- a/bfd/bfd.c
+++ b/bfd/bfd.c
@@ -74,6 +74,20 @@ EXTERNAL
. bfd_byte data[1];
. };
.
+.struct bfd_mmapped_entry
+. {
+. void *addr;
+. size_t size;
+. };
+.
+.struct bfd_mmapped
+. {
+. struct bfd_mmapped *next;
+. unsigned int max_entry;
+. unsigned int next_entry;
+. struct bfd_mmapped_entry entries[1];
+. };
+.
CODE_FRAGMENT
.struct bfd
@@ -406,6 +420,9 @@ CODE_FRAGMENT
.
. {* For input BFDs, the build ID, if the object has one. *}
. const struct bfd_build_id *build_id;
+.
+. {* For input BFDs, mmapped entries. *}
+. struct bfd_mmapped *mmapped;
.};
.
diff --git a/bfd/bfdio.c b/bfd/bfdio.c
index ab3e28ea204..a6b88b55625 100644
--- a/bfd/bfdio.c
+++ b/bfd/bfdio.c
@@ -219,9 +219,9 @@ DESCRIPTION
. Also write in MAP_ADDR the address of the page aligned buffer and in
. MAP_LEN the size mapped (a page multiple). Use unmap with MAP_ADDR and
. MAP_LEN to unmap. *}
-. void *(*bmmap) (struct bfd *abfd, void *addr, bfd_size_type len,
+. void *(*bmmap) (struct bfd *abfd, void *addr, size_t len,
. int prot, int flags, file_ptr offset,
-. void **map_addr, bfd_size_type *map_len);
+. void **map_addr, size_t *map_len);
.};
.extern const struct bfd_iovec _bfd_memory_iovec;
@@ -638,9 +638,9 @@ FUNCTION
bfd_mmap
SYNOPSIS
- void *bfd_mmap (bfd *abfd, void *addr, bfd_size_type len,
+ void *bfd_mmap (bfd *abfd, void *addr, size_t len,
int prot, int flags, file_ptr offset,
- void **map_addr, bfd_size_type *map_len)
+ void **map_addr, size_t *map_len)
ATTRIBUTE_WARN_UNUSED_RESULT;
DESCRIPTION
@@ -651,9 +651,9 @@ DESCRIPTION
*/
void *
-bfd_mmap (bfd *abfd, void *addr, bfd_size_type len,
+bfd_mmap (bfd *abfd, void *addr, size_t len,
int prot, int flags, file_ptr offset,
- void **map_addr, bfd_size_type *map_len)
+ void **map_addr, size_t *map_len)
{
while (abfd->my_archive != NULL
&& !bfd_is_thin_archive (abfd->my_archive))
@@ -815,10 +815,10 @@ memory_bstat (bfd *abfd, struct stat *statbuf)
static void *
memory_bmmap (bfd *abfd ATTRIBUTE_UNUSED, void *addr ATTRIBUTE_UNUSED,
- bfd_size_type len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED,
+ size_t len ATTRIBUTE_UNUSED, int prot ATTRIBUTE_UNUSED,
int flags ATTRIBUTE_UNUSED, file_ptr offset ATTRIBUTE_UNUSED,
void **map_addr ATTRIBUTE_UNUSED,
- bfd_size_type *map_len ATTRIBUTE_UNUSED)
+ size_t *map_len ATTRIBUTE_UNUSED)
{
return (void *)-1;
}
diff --git a/bfd/bfdwin.c b/bfd/bfdwin.c
index beb17398744..d9ce0d25e90 100644
--- a/bfd/bfdwin.c
+++ b/bfd/bfdwin.c
@@ -165,7 +165,7 @@ bfd_get_file_window (bfd *abfd,
bool writable)
{
static int ok_to_map = 1;
- static size_t pagesize;
+ size_t pagesize = _bfd_pagesize;
bfd_window_internal *i = windowp->i;
bfd_size_type size_to_alloc = size;
@@ -175,12 +175,6 @@ bfd_get_file_window (bfd *abfd,
windowp, windowp->data, (unsigned long) windowp->size,
windowp->i, writable);
- /* Make sure we know the page size, so we can be friendly to mmap. */
- if (pagesize == 0)
- pagesize = getpagesize ();
- if (pagesize == 0)
- abort ();
-
if (i == NULL)
{
i = bfd_zmalloc (sizeof (bfd_window_internal));
diff --git a/bfd/cache.c b/bfd/cache.c
index 4c00c00f8d5..e0bdb658c9e 100644
--- a/bfd/cache.c
+++ b/bfd/cache.c
@@ -482,12 +482,12 @@ cache_bstat (struct bfd *abfd, struct stat *sb)
static void *
cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED,
void *addr ATTRIBUTE_UNUSED,
- bfd_size_type len ATTRIBUTE_UNUSED,
+ size_t len ATTRIBUTE_UNUSED,
int prot ATTRIBUTE_UNUSED,
int flags ATTRIBUTE_UNUSED,
file_ptr offset ATTRIBUTE_UNUSED,
void **map_addr ATTRIBUTE_UNUSED,
- bfd_size_type *map_len ATTRIBUTE_UNUSED)
+ size_t *map_len ATTRIBUTE_UNUSED)
{
void *ret = (void *) -1;
@@ -498,10 +498,10 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED,
#ifdef HAVE_MMAP
else
{
- static uintptr_t pagesize_m1;
+ uintptr_t pagesize_m1 = _bfd_pagesize_m1;
FILE *f;
file_ptr pg_offset;
- bfd_size_type pg_len;
+ size_t pg_len;
f = bfd_cache_lookup (abfd, CACHE_NO_SEEK_ERROR);
if (f == NULL)
@@ -510,9 +510,6 @@ cache_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED,
return ret;
}
- if (pagesize_m1 == 0)
- pagesize_m1 = getpagesize () - 1;
-
/* Align. */
pg_offset = offset & ~pagesize_m1;
pg_len = (len + (offset - pg_offset) + pagesize_m1) & ~pagesize_m1;
diff --git a/bfd/elf.c b/bfd/elf.c
index 8bffd3c5141..1c905762704 100644
--- a/bfd/elf.c
+++ b/bfd/elf.c
@@ -289,16 +289,23 @@ bfd_elf_get_str_section (bfd *abfd, unsigned int shindex)
in case the string table is not terminated. */
if (shstrtabsize + 1 <= 1
|| bfd_seek (abfd, offset, SEEK_SET) != 0
- || (shstrtab = _bfd_alloc_and_read (abfd, shstrtabsize + 1,
- shstrtabsize)) == NULL)
+ || (shstrtab
+ = _bfd_mmap_readonly_tracked (abfd, shstrtabsize)) == NULL)
{
/* Once we've failed to read it, make sure we don't keep
trying. Otherwise, we'll keep allocating space for
the string table over and over. */
i_shdrp[shindex]->sh_size = 0;
}
- else
- shstrtab[shstrtabsize] = '\0';
+ else if (shstrtab[shstrtabsize - 1] != '\0')
+ {
+ /* It is an error if a string table isn't terminated. */
+ _bfd_error_handler
+ /* xgettext:c-format */
+ (_("%pB(%pA): string table is corrupt"),
+ abfd, i_shdrp[shindex]->bfd_section);
+ return NULL;
+ }
i_shdrp[shindex]->contents = shstrtab;
}
return (char *) shstrtab;
@@ -653,8 +660,12 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect)
if (IS_VALID_GROUP_SECTION_HEADER (shdr, 2 * GRP_ENTRY_SIZE))
{
- unsigned char *src;
+ unsigned char *src, *contents;
Elf_Internal_Group *dest;
+ size_t contents_size;
+#ifdef USE_MMAP
+ void *contents_addr = NULL;
+#endif
/* Make sure the group section has a BFD section
attached to it. */
@@ -667,12 +678,23 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect)
/* Read the raw contents. */
BFD_ASSERT (sizeof (*dest) >= 4 && sizeof (*dest) % 4 == 0);
+ contents_size = shdr->sh_size;
shdr->contents = NULL;
- if (_bfd_mul_overflow (shdr->sh_size,
+ if (_bfd_mul_overflow (contents_size,
sizeof (*dest) / 4, &amt)
|| bfd_seek (abfd, shdr->sh_offset, SEEK_SET) != 0
- || !(shdr->contents
- = _bfd_alloc_and_read (abfd, amt, shdr->sh_size)))
+#ifdef USE_MMAP
+ || !(contents
+ = _bfd_mmap_readonly_untracked (abfd,
+ contents_size,
+ &contents_addr,
+ &contents_size))
+#else
+ || !(contents
+ = _bfd_alloc_and_read (abfd, amt,
+ contents_size))
+#endif
+ )
{
_bfd_error_handler
/* xgettext:c-format */
@@ -684,11 +706,17 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect)
continue;
}
+#ifdef USE_MMAP
+ shdr->contents = bfd_alloc (abfd, amt);
+#else
+ shdr->contents = contents;
+#endif
+
/* Translate raw contents, a flag word followed by an
array of elf section indices all in target byte order,
to the flag word followed by an array of elf section
pointers. */
- src = shdr->contents + shdr->sh_size;
+ src = contents + shdr->sh_size;
dest = (Elf_Internal_Group *) (shdr->contents + amt);
while (1)
@@ -698,7 +726,7 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect)
src -= 4;
--dest;
idx = H_GET_32 (abfd, src);
- if (src == shdr->contents)
+ if (src == contents)
{
dest->shdr = NULL;
dest->flags = idx;
@@ -726,6 +754,11 @@ setup_group (bfd *abfd, Elf_Internal_Shdr *hdr, asection *newsect)
dest->shdr = NULL;
}
}
+
+#ifdef USE_MMAP
+ _bfd_munmap_readonly_untracked (contents_addr,
+ contents_size);
+#endif
}
}
@@ -1919,6 +1952,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number,
unsigned char *e_data = NULL;
bfd_vma *i_data = NULL;
bfd_size_type size;
+ void *e_data_addr;
+ size_t e_data_size ATTRIBUTE_UNUSED;
if (ent_size != 4 && ent_size != 8)
return NULL;
@@ -1940,7 +1975,8 @@ get_hash_table_data (bfd *abfd, bfd_size_type number,
return NULL;
}
- e_data = _bfd_malloc_and_read (abfd, size, size);
+ e_data = _bfd_mmap_readonly_untracked (abfd, size, &e_data_addr,
+ &e_data_size);
if (e_data == NULL)
return NULL;
@@ -1958,7 +1994,7 @@ get_hash_table_data (bfd *abfd, bfd_size_type number,
while (number--)
i_data[number] = bfd_get_64 (abfd, e_data + number * ent_size);
- free (e_data);
+ _bfd_munmap_readonly_untracked (e_data_addr, e_data_size);
return i_data;
}
@@ -2007,6 +2043,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
size_t verneed_size = 0;
size_t extsym_size;
const struct elf_backend_data *bed;
+ void *dynbuf_addr = NULL;
+ void *esymbuf_addr = NULL;
+ size_t dynbuf_size = 0;
+ size_t esymbuf_size = 0;
/* Return TRUE if symbol table is bad. */
if (elf_bad_symtab (abfd))
@@ -2024,7 +2064,9 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
if (bfd_seek (abfd, phdr->p_offset, SEEK_SET) != 0)
goto error_return;
- dynbuf = _bfd_malloc_and_read (abfd, phdr->p_filesz, phdr->p_filesz);
+ dynbuf_size = phdr->p_filesz;
+ dynbuf = _bfd_mmap_readonly_untracked (abfd, dynbuf_size,
+ &dynbuf_addr, &dynbuf_size);
if (dynbuf == NULL)
goto error_return;
@@ -2102,11 +2144,17 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
goto error_return;
/* Dynamic string table must be valid until ABFD is closed. */
- strbuf = (char *) _bfd_alloc_and_read (abfd, dt_strsz + 1, dt_strsz);
+ strbuf = (char *) _bfd_mmap_readonly_tracked (abfd, dt_strsz);
if (strbuf == NULL)
goto error_return;
- /* Since this is a string table, make sure that it is terminated. */
- strbuf[dt_strsz] = 0;
+ if (strbuf[dt_strsz - 1] != 0)
+ {
+ /* It is an error if a string table is't terminated. */
+ _bfd_error_handler
+ /* xgettext:c-format */
+ (_("%pB: DT_STRTAB table is corrupt"), abfd);
+ goto error_return;
+ }
/* Get the real symbol count from DT_HASH or DT_GNU_HASH. Prefer
DT_HASH since it is simpler than DT_GNU_HASH. */
@@ -2281,7 +2329,10 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
if (filepos == (file_ptr) -1
|| bfd_seek (abfd, filepos, SEEK_SET) != 0)
goto error_return;
- esymbuf = _bfd_malloc_and_read (abfd, amt, amt);
+ esymbuf_size = amt;
+ esymbuf = _bfd_mmap_readonly_untracked (abfd, esymbuf_size,
+ &esymbuf_addr,
+ &esymbuf_size);
if (esymbuf == NULL)
goto error_return;
@@ -2325,7 +2376,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
goto error_return;
/* DT_VERSYM info must be valid until ABFD is closed. */
- versym = _bfd_alloc_and_read (abfd, amt, amt);
+ versym = _bfd_mmap_readonly_tracked (abfd, amt);
if (dt_verdef)
{
@@ -2337,8 +2388,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
goto error_return;
/* DT_VERDEF info must be valid until ABFD is closed. */
- verdef = _bfd_alloc_and_read (abfd, verdef_size,
- verdef_size);
+ verdef = _bfd_mmap_readonly_tracked (abfd, verdef_size);
}
if (dt_verneed)
@@ -2351,8 +2401,7 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
goto error_return;
/* DT_VERNEED info must be valid until ABFD is closed. */
- verneed = _bfd_alloc_and_read (abfd, verneed_size,
- verneed_size);
+ verneed = _bfd_mmap_readonly_tracked (abfd, verneed_size);
}
}
@@ -2375,8 +2424,8 @@ _bfd_elf_get_dynamic_symbols (bfd *abfd, Elf_Internal_Phdr *phdr,
/* Restore file position for elf_object_p. */
if (bfd_seek (abfd, saved_filepos, SEEK_SET) != 0)
res = false;
- free (dynbuf);
- free (esymbuf);
+ _bfd_munmap_readonly_untracked (dynbuf_addr, dynbuf_size);
+ _bfd_munmap_readonly_untracked (esymbuf_addr, esymbuf_size);
free (gnubuckets);
free (gnuchains);
free (mipsxlat);
@@ -9435,6 +9484,8 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
bfd_byte *contents = NULL;
unsigned int freeidx = 0;
size_t amt;
+ void *contents_addr = NULL;
+ size_t contents_size = 0;
if (elf_dynverref (abfd) != 0 || elf_tdata (abfd)->dt_verneed != NULL)
{
@@ -9471,7 +9522,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0)
goto error_return_verref;
- contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size);
+ contents_size = hdr->sh_size;
+ contents = _bfd_mmap_readonly_untracked (abfd, contents_size,
+ &contents_addr,
+ &contents_size);
if (contents == NULL)
goto error_return_verref;
@@ -9604,8 +9658,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
elf_tdata (abfd)->cverrefs = i;
if (contents != elf_tdata (abfd)->dt_verneed)
- free (contents);
+ _bfd_munmap_readonly_untracked (contents_addr, contents_size);
contents = NULL;
+ contents_addr = NULL;
}
if (elf_dynverdef (abfd) != 0 || elf_tdata (abfd)->dt_verdef != NULL)
@@ -9646,7 +9701,10 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
if (bfd_seek (abfd, hdr->sh_offset, SEEK_SET) != 0)
goto error_return_verdef;
- contents = _bfd_malloc_and_read (abfd, hdr->sh_size, hdr->sh_size);
+ contents_size = hdr->sh_size;
+ contents = _bfd_mmap_readonly_untracked (abfd, contents_size,
+ &contents_addr,
+ &contents_size);
if (contents == NULL)
goto error_return_verdef;
@@ -9800,8 +9858,9 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
}
if (contents != elf_tdata (abfd)->dt_verdef)
- free (contents);
+ _bfd_munmap_readonly_untracked (contents_addr, contents_size);
contents = NULL;
+ contents_addr = NULL;
}
else if (default_imported_symver)
{
@@ -9857,7 +9916,7 @@ _bfd_elf_slurp_version_tables (bfd *abfd, bool default_imported_symver)
error_return:
if (contents != elf_tdata (abfd)->dt_verneed
&& contents != elf_tdata (abfd)->dt_verdef)
- free (contents);
+ _bfd_munmap_readonly_untracked (contents_addr, contents_size);
return false;
}
\f
diff --git a/bfd/elflink.c b/bfd/elflink.c
index 5a6cb07b2ce..42029f29f7a 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -549,22 +549,24 @@ bfd_elf_link_record_dynamic_symbol (struct bfd_link_info *info,
return false;
}
+ char *unversioned_name = NULL;
+
/* We don't put any version information in the dynamic string
table. */
name = h->root.root.string;
p = strchr (name, ELF_VER_CHR);
if (p != NULL)
- /* We know that the p points into writable memory. In fact,
- there are only a few symbols that have read-only names, being
- those like _GLOBAL_OFFSET_TABLE_ that are created specially
- by the backends. Most symbols will have names pointing into
- an ELF string table read from a file, or to objalloc memory. */
- *p = 0;
+ {
+ unversioned_name = bfd_malloc (p - name + 1);
+ memcpy (unversioned_name, name, p - name);
+ unversioned_name[p - name] = 0;
+ name = unversioned_name;
+ }
indx = _bfd_elf_strtab_add (dynstr, name, p != NULL);
if (p != NULL)
- *p = ELF_VER_CHR;
+ free (unversioned_name);
if (indx == (size_t) -1)
return false;
diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h
index b8b2ce7ba09..7887fad9c92 100644
--- a/bfd/libbfd-in.h
+++ b/bfd/libbfd-in.h
@@ -851,6 +851,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *)
extern bool _bfd_link_keep_memory (struct bfd_link_info *)
ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN;
+
#if GCC_VERSION >= 7000
#define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res)
#else
@@ -888,6 +892,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
return NULL;
}
+#ifdef USE_MMAP
+extern void *_bfd_mmap_readonly_tracked
+ (bfd *, size_t) ATTRIBUTE_HIDDEN;
+extern void *_bfd_mmap_readonly_untracked
+ (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN;
+extern void _bfd_munmap_readonly_untracked
+ (void *, size_t) ATTRIBUTE_HIDDEN;
+#else
+#define _bfd_mmap_readonly_tracked(abfd, rsize) \
+ _bfd_alloc_and_read (abfd, rsize, rsize)
+#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr)
+#endif
+
static inline void *
_bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
{
@@ -910,3 +927,15 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
}
return NULL;
}
+
+#ifndef USE_MMAP
+static inline void *
+_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr,
+ size_t *map_size)
+{
+ void *mem = _bfd_malloc_and_read (abfd, rsize, rsize);
+ *map_addr = mem;
+ *map_size = rsize;
+ return mem;
+}
+#endif
diff --git a/bfd/libbfd.c b/bfd/libbfd.c
index f8d148c9677..237f91e5c97 100644
--- a/bfd/libbfd.c
+++ b/bfd/libbfd.c
@@ -1038,6 +1038,147 @@ bfd_get_bits (const void *p, int bits, bool big_p)
return data;
}
\f
+#ifdef USE_MMAP
+#include <sys/mman.h>
+
+#ifndef MAP_ANONYMOUS
+#define MAP_ANONYMOUS MAP_ANON
+#endif
+
+/* Allocate a page to track mmapped memory and return the page and
+ the first entry. Return NULL if mmap fails. */
+
+static struct bfd_mmapped *
+bfd_allocate_mmapped_page (bfd *abfd, struct bfd_mmapped_entry **entry)
+{
+ struct bfd_mmapped * mmapped
+ = (struct bfd_mmapped *) mmap (NULL, _bfd_pagesize,
+ PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS,
+ -1, 0);
+ if (mmapped == MAP_FAILED)
+ return NULL;
+
+ mmapped->next = abfd->mmapped;
+ mmapped->max_entry
+ = ((_bfd_pagesize - offsetof (struct bfd_mmapped, entries))
+ / sizeof (struct bfd_mmapped_entry));
+ mmapped->next_entry = 1;
+ abfd->mmapped = mmapped;
+ *entry = mmapped->entries;
+ return mmapped;
+}
+
+/* Mmap a memory region of RSIZE bytes with PROT at the current offset.
+ Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL
+ on invalid input and (void *) -1 for mmap failure. */
+
+static void *
+bfd_mmap_local (bfd *abfd, size_t rsize, int prot, void **map_addr,
+ size_t *map_size)
+{
+ if (!_bfd_constant_p (rsize))
+ {
+ ufile_ptr filesize = bfd_get_file_size (abfd);
+ if (filesize != 0 && rsize > filesize)
+ {
+ bfd_set_error (bfd_error_file_truncated);
+ return NULL;
+ }
+ }
+
+ void *mem;
+ ufile_ptr offset = bfd_tell (abfd);
+ mem = bfd_mmap (abfd, NULL, rsize, prot, MAP_PRIVATE, offset,
+ map_addr, map_size);
+ return mem;
+}
+
+/* Mmap a readonly memory region of RSIZE bytes at the current offset.
+ Return mmap address and size in MAP_ADDR and MAP_SIZE. Return NULL
+ on invalid input and (void *) -1 for mmap failure. */
+
+void *
+_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr,
+ size_t *map_size)
+{
+ /* Use mmap only if section size >= the minimum mmap section size. */
+ if (rsize < _bfd_minimum_mmap_size)
+ {
+ void *mem = _bfd_malloc_and_read (abfd, rsize, rsize);
+ /* NB: Set *MAP_ADDR to MEM and *MAP_SIZE to 0 to indicate that
+ _bfd_malloc_and_read is called. */
+ *map_addr = mem;
+ *map_size = 0;
+ return mem;
+ }
+
+ return bfd_mmap_local (abfd, rsize, PROT_READ, map_addr, map_size);
+}
+
+/* Munmap RSIZE bytes at PTR. */
+
+void
+_bfd_munmap_readonly_untracked (void *ptr, size_t rsize)
+{
+ /* NB: Since _bfd_munmap_readonly_untracked is called like free, PTR
+ may be NULL. Otherwise, PTR and RSIZE must be valid. If RSIZE is
+ 0, _bfd_malloc_and_read is called. */
+ if (ptr == NULL)
+ return;
+ if (rsize != 0)
+ {
+ if (munmap (ptr, rsize) != 0)
+ abort ();
+ }
+ else
+ free (ptr);
+}
+
+/* Mmap a readonly memory region of RSIZE bytes at the current offset.
+ Return NULL on invalid input or mmap failure. */
+
+void *
+_bfd_mmap_readonly_tracked (bfd *abfd, size_t rsize)
+{
+ /* Use mmap only if section size >= the minimum mmap section size. */
+ if (rsize < _bfd_minimum_mmap_size)
+ return _bfd_alloc_and_read (abfd, rsize, rsize);
+
+ void *mem, *map_addr;
+ size_t map_size;
+ mem = bfd_mmap_local (abfd, rsize, PROT_READ, &map_addr, &map_size);
+ if (mem == NULL)
+ return mem;
+ if (mem == (void *) -1)
+ return _bfd_alloc_and_read (abfd, rsize, rsize);
+
+ struct bfd_mmapped_entry *entry;
+ unsigned int next_entry;
+ struct bfd_mmapped *mmapped = abfd->mmapped;
+ if (mmapped != NULL
+ && (next_entry = mmapped->next_entry) < mmapped->max_entry)
+ {
+ entry = &mmapped->entries[next_entry];
+ mmapped->next_entry++;
+ }
+ else
+ {
+ mmapped = bfd_allocate_mmapped_page (abfd, &entry);
+ if (mmapped == NULL)
+ {
+ munmap (map_addr, map_size);
+ return NULL;
+ }
+ }
+
+ entry->addr = map_addr;
+ entry->size = map_size;
+
+ return mem;
+}
+#endif
+
/* Default implementation */
bool
@@ -1326,3 +1467,19 @@ _bfd_generic_init_private_section_data (bfd *ibfd ATTRIBUTE_UNUSED,
{
return true;
}
+
+uintptr_t _bfd_pagesize;
+uintptr_t _bfd_pagesize_m1;
+uintptr_t _bfd_minimum_mmap_size;
+
+__attribute__ ((unused, constructor))
+static void
+bfd_init_pagesize (void)
+{
+ _bfd_pagesize = getpagesize ();
+ if (_bfd_pagesize == 0)
+ abort ();
+ _bfd_pagesize_m1 = _bfd_pagesize - 1;
+ /* The minimum section size to use mmap. */
+ _bfd_minimum_mmap_size = _bfd_pagesize * 4;
+}
diff --git a/bfd/libbfd.h b/bfd/libbfd.h
index a60063b5ae1..1515a03b093 100644
--- a/bfd/libbfd.h
+++ b/bfd/libbfd.h
@@ -857,6 +857,10 @@ extern struct bfd_link_info *_bfd_get_link_info (bfd *)
extern bool _bfd_link_keep_memory (struct bfd_link_info *)
ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN;
+extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN;
+
#if GCC_VERSION >= 7000
#define _bfd_mul_overflow(a, b, res) __builtin_mul_overflow (a, b, res)
#else
@@ -894,6 +898,19 @@ _bfd_alloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
return NULL;
}
+#ifdef USE_MMAP
+extern void *_bfd_mmap_readonly_tracked
+ (bfd *, size_t) ATTRIBUTE_HIDDEN;
+extern void *_bfd_mmap_readonly_untracked
+ (bfd *, size_t, void **, size_t *) ATTRIBUTE_HIDDEN;
+extern void _bfd_munmap_readonly_untracked
+ (void *, size_t) ATTRIBUTE_HIDDEN;
+#else
+#define _bfd_mmap_readonly_tracked(abfd, rsize) \
+ _bfd_alloc_and_read (abfd, rsize, rsize)
+#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr)
+#endif
+
static inline void *
_bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
{
@@ -916,6 +933,18 @@ _bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
}
return NULL;
}
+
+#ifndef USE_MMAP
+static inline void *
+_bfd_mmap_readonly_untracked (bfd *abfd, size_t rsize, void **map_addr,
+ size_t *map_size)
+{
+ void *mem = _bfd_malloc_and_read (abfd, rsize, rsize);
+ *map_addr = mem;
+ *map_size = rsize;
+ return mem;
+}
+#endif
/* Extracted from libbfd.c. */
void *bfd_malloc (bfd_size_type /*size*/) ATTRIBUTE_HIDDEN;
@@ -969,9 +998,9 @@ struct bfd_iovec
Also write in MAP_ADDR the address of the page aligned buffer and in
MAP_LEN the size mapped (a page multiple). Use unmap with MAP_ADDR and
MAP_LEN to unmap. */
- void *(*bmmap) (struct bfd *abfd, void *addr, bfd_size_type len,
+ void *(*bmmap) (struct bfd *abfd, void *addr, size_t len,
int prot, int flags, file_ptr offset,
- void **map_addr, bfd_size_type *map_len);
+ void **map_addr, size_t *map_len);
};
extern const struct bfd_iovec _bfd_memory_iovec;
diff --git a/bfd/lynx-core.c b/bfd/lynx-core.c
index 44d94ad8745..9ec5a0d2028 100644
--- a/bfd/lynx-core.c
+++ b/bfd/lynx-core.c
@@ -96,7 +96,7 @@ lynx_core_file_p (bfd *abfd)
asection *newsect;
size_t amt;
- pagesize = getpagesize (); /* Serious cross-target issue here... This
+ pagesize = _bfd_pagesize; /* Serious cross-target issue here... This
really needs to come from a system-specific
header file. */
diff --git a/bfd/opncls.c b/bfd/opncls.c
index a0a5c40fba8..ac74607bac0 100644
--- a/bfd/opncls.c
+++ b/bfd/opncls.c
@@ -27,6 +27,10 @@
#include "libiberty.h"
#include "elf-bfd.h"
+#ifdef USE_MMAP
+#include <sys/mman.h>
+#endif
+
#ifndef S_IXUSR
#define S_IXUSR 0100 /* Execute by owner. */
#endif
@@ -163,6 +167,18 @@ _bfd_new_bfd_contained_in (bfd *obfd)
static void
_bfd_delete_bfd (bfd *abfd)
{
+#ifdef USE_MMAP
+ struct bfd_mmapped *mmapped, *next;
+ for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next)
+ {
+ struct bfd_mmapped_entry *entries = mmapped->entries;
+ next = mmapped->next;
+ for (unsigned int i = 0; i < mmapped->next_entry; i++)
+ munmap (entries[i].addr, entries[i].size);
+ munmap (mmapped, _bfd_pagesize);
+ }
+#endif
+
/* Give the target _bfd_free_cached_info a chance to free memory. */
if (abfd->memory && abfd->xvec)
bfd_free_cached_info (abfd);
@@ -668,12 +684,12 @@ opncls_bstat (struct bfd *abfd, struct stat *sb)
static void *
opncls_bmmap (struct bfd *abfd ATTRIBUTE_UNUSED,
void *addr ATTRIBUTE_UNUSED,
- bfd_size_type len ATTRIBUTE_UNUSED,
+ size_t len ATTRIBUTE_UNUSED,
int prot ATTRIBUTE_UNUSED,
int flags ATTRIBUTE_UNUSED,
file_ptr offset ATTRIBUTE_UNUSED,
void **map_addr ATTRIBUTE_UNUSED,
- bfd_size_type *map_len ATTRIBUTE_UNUSED)
+ size_t *map_len ATTRIBUTE_UNUSED)
{
return (void *) -1;
}
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 4/6] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
` (2 preceding siblings ...)
2024-03-06 20:47 ` [PATCH v3 3/6] elf: Use mmap to map in read-only sections H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 20:47 ` [PATCH v3 5/6] elf: Use mmap to map in symbol and relocation tables H.J. Lu
` (2 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents to avoid
reading input sections twice. A backend must opt-in to use mmap. It
should replace bfd_malloc_and_get_section with _bfd_elf_mmap_section and
free with _bfd_elf_munmap_section_contents on section contents.
* compress.c (bfd_get_full_section_contents): Don't allocate
buffer if mmapped_p is true.
* elf-bfd.h (elf_backend_data): Add use_mmap.
(bfd_elf_section_data): Add contents_addr and contents_size.
(_bfd_elf_mmap_section): New.
(_bfd_elf_munmap_section_contents): Likewise.
* elf-eh-frame.c (_bfd_elf_parse_eh_frame): Replace
bfd_malloc_and_get_section and free with
_bfd_elf_mmap_section and _bfd_elf_munmap_section_contents on
section contents.
* elf-sframe.c (_bfd_elf_parse_sframe): Likewise.
* elf.c: Include <sys/mman.h> if HAVE_MMAP is defined.
(_bfd_elf_make_section_from_shdr): Replace
bfd_malloc_and_get_section and free with
_bfd_elf_mmap_section and _bfd_elf_munmap_section_contents on
section contents.
(_bfd_elf_print_private_bfd_data): Likewise.
(_bfd_elf_mmap_section): New.
(_bfd_elf_munmap_section_contents): Likewise.
* elf32-i386.c (elf_i386_scan_relocs): Replace
bfd_malloc_and_get_section and free with
_bfd_elf_mmap_section and _bfd_elf_munmap_section_contents on
section contents.
* elf64-x86-64.c (elf_x86_64_scan_relocs): Likewise.
(elf_x86_64_get_synthetic_symtab): Likewise.
* elfcode.h (elf_checksum_contents): Likewise.
* elflink.c (elf_link_add_object_symbols): Likewise.
(bfd_elf_get_bfd_needed_list): Likewise.
* elfxx-target.h (elf_backend_use_mmap): New.
(elfNN_bed): Add elf_backend_use_mmap.
* elfxx-x86.c (elf_x86_size_or_finish_relative_reloc): Replace
bfd_malloc_and_get_section and free with
_bfd_elf_mmap_section and _bfd_elf_munmap_section_contents on
section contents.
(_bfd_x86_elf_get_synthetic_symtab): Replace free with
_bfd_elf_munmap_section_contents.
* elfxx-x86.h (elf_backend_use_mmap): New.
* libbfd.c: Include "elf-bfd.h".
(_bfd_generic_get_section_contents): Call bfd_mmap_local for
mmapped_p.
* opncls.c (_bfd_delete_bfd): Also munmap ELF section contents.
* section.c (asection): Add mmapped_p.
(BFD_FAKE_SECTION): Updated.
(bfd_malloc_and_get_section): Add a sanity check for not
mmapped_p.
* bfd-in2.h: Regenerated.
---
bfd/bfd-in2.h | 7 ++--
bfd/compress.c | 2 +-
bfd/elf-bfd.h | 20 +++++++++++
bfd/elf-eh-frame.c | 4 +--
bfd/elf-sframe.c | 4 +--
bfd/elf.c | 89 +++++++++++++++++++++++++++++++++++++++++++---
bfd/elf32-i386.c | 6 ++--
bfd/elf64-x86-64.c | 10 +++---
bfd/elfcode.h | 7 ++--
bfd/elflink.c | 12 +++----
bfd/elfxx-target.h | 6 +++-
bfd/elfxx-x86.c | 7 ++--
bfd/elfxx-x86.h | 1 +
bfd/libbfd.c | 59 ++++++++++++++++++++++++++++--
bfd/opncls.c | 9 +++++
bfd/section.c | 9 +++--
16 files changed, 213 insertions(+), 39 deletions(-)
diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h
index 544422a9522..dc6bceb6a96 100644
--- a/bfd/bfd-in2.h
+++ b/bfd/bfd-in2.h
@@ -688,6 +688,9 @@ typedef struct bfd_section
/* Nonzero if this section uses RELA relocations, rather than REL. */
unsigned int use_rela_p:1;
+ /* Nonzero if this section contents are mmapped, rather than malloced. */
+ unsigned int mmapped_p:1;
+
/* Bits used by various backends. The generic code doesn't touch
these fields. */
@@ -975,8 +978,8 @@ discarded_section (const asection *sec)
/* linker_mark, linker_has_input, gc_mark, decompress_status, */ \
0, 0, 1, 0, \
\
- /* segment_mark, sec_info_type, use_rela_p, */ \
- 0, 0, 0, \
+ /* segment_mark, sec_info_type, use_rela_p, mmapped_p, */ \
+ 0, 0, 0, 0, \
\
/* sec_flg0, sec_flg1, sec_flg2, sec_flg3, sec_flg4, sec_flg5, */ \
0, 0, 0, 0, 0, 0, \
diff --git a/bfd/compress.c b/bfd/compress.c
index 8bc44de813b..6c211843e60 100644
--- a/bfd/compress.c
+++ b/bfd/compress.c
@@ -749,7 +749,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr)
switch (compress_status)
{
case COMPRESS_SECTION_NONE:
- if (p == NULL)
+ if (p == NULL && !sec->mmapped_p)
{
p = (bfd_byte *) bfd_malloc (allocsz);
if (p == NULL)
diff --git a/bfd/elf-bfd.h b/bfd/elf-bfd.h
index c5d325435b6..da7c5208017 100644
--- a/bfd/elf-bfd.h
+++ b/bfd/elf-bfd.h
@@ -1775,6 +1775,12 @@ struct elf_backend_data
/* True if the 64-bit Linux PRPSINFO structure's `pr_uid' and `pr_gid'
members use a 16-bit data type. */
unsigned linux_prpsinfo64_ugid16 : 1;
+
+ /* True if the backend can use mmap to map in all input section
+ contents. All bfd_malloc_and_get_section and free usages on
+ section contents must be replaced by _bfd_elf_mmap_section and
+ _bfd_elf_munmap_section_contents. */
+ unsigned use_mmap : 1;
};
/* Information about reloc sections associated with a bfd_elf_section_data
@@ -1856,6 +1862,15 @@ struct bfd_elf_section_data
/* Link from a text section to its .eh_frame_entry section. */
asection *eh_frame_entry;
+ /* If the mmapped_p flag is set, this points to the actual mmapped
+ address of contents. If it is set to NULL, contents isn't
+ mmapped. */
+ void *contents_addr;
+
+ /* If the mmapped_p flag is set, this is the actual mmapped size of
+ contents. */
+ size_t contents_size;
+
/* TRUE if the section has secondary reloc sections associated with it.
FIXME: In the future it might be better to change this into a list
of secondary reloc sections, making lookup easier and faster. */
@@ -3124,6 +3139,11 @@ extern bool _bfd_elf_maybe_set_textrel
extern bool _bfd_elf_add_dynamic_tags
(bfd *, struct bfd_link_info *, bool);
+extern bool _bfd_elf_mmap_section
+ (bfd *abfd, asection *section, bfd_byte **buf);
+extern void _bfd_elf_munmap_section_contents
+ (asection *, void *);
+
/* Large common section. */
extern asection _bfd_elf_large_com_section;
diff --git a/bfd/elf-eh-frame.c b/bfd/elf-eh-frame.c
index 9a504234163..f9836f3ea14 100644
--- a/bfd/elf-eh-frame.c
+++ b/bfd/elf-eh-frame.c
@@ -618,7 +618,7 @@ _bfd_elf_parse_eh_frame (bfd *abfd, struct bfd_link_info *info,
/* Read the frame unwind information from abfd. */
- REQUIRE (bfd_malloc_and_get_section (abfd, sec, &ehbuf));
+ REQUIRE (_bfd_elf_mmap_section (abfd, sec, &ehbuf));
/* If .eh_frame section size doesn't fit into int, we cannot handle
it (it would need to use 64-bit .eh_frame format anyway). */
@@ -1052,7 +1052,7 @@ _bfd_elf_parse_eh_frame (bfd *abfd, struct bfd_link_info *info,
hdr_info->u.dwarf.table = false;
free (sec_info);
success:
- free (ehbuf);
+ _bfd_elf_munmap_section_contents (sec, ehbuf);
free (local_cies);
#undef REQUIRE
}
diff --git a/bfd/elf-sframe.c b/bfd/elf-sframe.c
index bfc875cd9fc..9a271d414b9 100644
--- a/bfd/elf-sframe.c
+++ b/bfd/elf-sframe.c
@@ -208,7 +208,7 @@ _bfd_elf_parse_sframe (bfd *abfd,
}
/* Read the SFrame stack trace information from abfd. */
- if (!bfd_malloc_and_get_section (abfd, sec, &sfbuf))
+ if (!_bfd_elf_mmap_section (abfd, sec, &sfbuf))
goto fail_no_free;
/* Decode the buffer and keep decoded contents for later use.
@@ -241,7 +241,7 @@ fail_no_free:
abfd, sec);
return false;
success:
- free (sfbuf);
+ _bfd_elf_munmap_section_contents (sec, sfbuf);
return true;
}
diff --git a/bfd/elf.c b/bfd/elf.c
index 1c905762704..e2e31a93950 100644
--- a/bfd/elf.c
+++ b/bfd/elf.c
@@ -45,6 +45,10 @@ SECTION
#include "safe-ctype.h"
#include "elf-linux-core.h"
+#ifdef USE_MMAP
+#include <sys/mman.h>
+#endif
+
#ifdef CORE_HEADER
#include CORE_HEADER
#endif
@@ -1149,12 +1153,12 @@ _bfd_elf_make_section_from_shdr (bfd *abfd,
{
bfd_byte *contents;
- if (!bfd_malloc_and_get_section (abfd, newsect, &contents))
+ if (!_bfd_elf_mmap_section (abfd, newsect, &contents))
return false;
elf_parse_notes (abfd, (char *) contents, hdr->sh_size,
hdr->sh_offset, hdr->sh_addralign);
- free (contents);
+ _bfd_elf_munmap_section_contents (newsect, contents);
}
if ((newsect->flags & SEC_ALLOC) != 0)
@@ -1734,7 +1738,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg)
fprintf (f, _("\nDynamic Section:\n"));
- if (!bfd_malloc_and_get_section (abfd, s, &dynbuf))
+ if (!_bfd_elf_mmap_section (abfd, s, &dynbuf))
goto error_return;
elfsec = _bfd_elf_section_from_bfd_section (abfd, s);
@@ -1856,7 +1860,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg)
fprintf (f, "\n");
}
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
dynbuf = NULL;
}
@@ -1913,7 +1917,7 @@ _bfd_elf_print_private_bfd_data (bfd *abfd, void *farg)
return true;
error_return:
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
return false;
}
@@ -14418,3 +14422,78 @@ _bfd_elf_write_secondary_reloc_section (bfd *abfd, asection *sec)
return result;
}
+
+/* Mmap in section contents. */
+
+bool
+_bfd_elf_mmap_section (bfd *abfd, sec_ptr sec, bfd_byte **buf)
+{
+#ifdef USE_MMAP
+ const struct elf_backend_data *bed = get_elf_backend_data (abfd);
+ if (bed->use_mmap
+ && sec->compress_status == COMPRESS_SECTION_NONE
+ && (sec->flags & SEC_LINKER_CREATED) == 0)
+ {
+ /* Use mmap only if section size >= the minimum mmap section
+ size. */
+ size_t readsz = bfd_get_section_limit_octets (abfd, sec);
+ size_t allocsz = bfd_get_section_alloc_size (abfd, sec);
+ if (readsz == allocsz && readsz >= _bfd_minimum_mmap_size)
+ {
+ if (sec->contents != NULL)
+ {
+ if (!sec->mmapped_p)
+ abort ();
+ *buf = sec->contents;
+ return true;
+ }
+ if (sec->mmapped_p)
+ abort ();
+ sec->mmapped_p = 1;
+ }
+ }
+#endif
+ *buf = NULL;
+ bool ret = bfd_get_full_section_contents (abfd, sec, buf);
+ if (ret && sec->mmapped_p)
+ *buf = sec->contents;
+ return ret;
+}
+
+/* Munmap section contents. */
+
+void
+_bfd_elf_munmap_section_contents (asection *sec ATTRIBUTE_UNUSED,
+ void *contents)
+{
+ /* NB: Since _bfd_elf_munmap_section_contents is called like free,
+ CONTENTS may be NULL. */
+ if (contents == NULL)
+ return;
+
+#ifdef USE_MMAP
+ if (sec->mmapped_p)
+ {
+ /* NB: Don't free CONTENTS if it has been cached. */
+ if (elf_section_data (sec)->this_hdr.contents == contents)
+ return;
+
+ /* When bfd_mmap_local returns (void *) -1 on an unknown input,
+ CONTENTS is malloced and CONTENTS_ADDR is set to NULL. */
+ if (elf_section_data (sec)->contents_addr != NULL)
+ {
+ /* NB: CONTENTS_ADDR and CONTENTS_SIZE must be valid. */
+ if (munmap (elf_section_data (sec)->contents_addr,
+ elf_section_data (sec)->contents_size) != 0)
+ abort ();
+ sec->mmapped_p = 0;
+ sec->contents = NULL;
+ elf_section_data (sec)->contents_addr = NULL;
+ elf_section_data (sec)->contents_size = 0;
+ return;
+ }
+ }
+#endif
+
+ free (contents);
+}
diff --git a/bfd/elf32-i386.c b/bfd/elf32-i386.c
index 703a48c2c0a..8aa1533d0ba 100644
--- a/bfd/elf32-i386.c
+++ b/bfd/elf32-i386.c
@@ -1499,7 +1499,7 @@ elf_i386_scan_relocs (bfd *abfd,
/* Get the section contents. */
if (elf_section_data (sec)->this_hdr.contents != NULL)
contents = elf_section_data (sec)->this_hdr.contents;
- else if (!bfd_malloc_and_get_section (abfd, sec, &contents))
+ else if (!_bfd_elf_mmap_section (abfd, sec, &contents))
{
sec->check_relocs_failed = 1;
return false;
@@ -1933,7 +1933,7 @@ elf_i386_scan_relocs (bfd *abfd,
if (elf_section_data (sec)->this_hdr.contents != contents)
{
if (!converted && !_bfd_link_keep_memory (info))
- free (contents);
+ _bfd_elf_munmap_section_contents (sec, contents);
else
{
/* Cache the section contents for elf_link_input_bfd if any
@@ -1951,7 +1951,7 @@ elf_i386_scan_relocs (bfd *abfd,
error_return:
if (elf_section_data (sec)->this_hdr.contents != contents)
- free (contents);
+ _bfd_elf_munmap_section_contents (sec, contents);
sec->check_relocs_failed = 1;
return false;
}
diff --git a/bfd/elf64-x86-64.c b/bfd/elf64-x86-64.c
index 3300a2017bd..f1e06040a5a 100644
--- a/bfd/elf64-x86-64.c
+++ b/bfd/elf64-x86-64.c
@@ -2057,7 +2057,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info,
/* Get the section contents. */
if (elf_section_data (sec)->this_hdr.contents != NULL)
contents = elf_section_data (sec)->this_hdr.contents;
- else if (!bfd_malloc_and_get_section (abfd, sec, &contents))
+ else if (!_bfd_elf_mmap_section (abfd, sec, &contents))
{
sec->check_relocs_failed = 1;
return false;
@@ -2591,7 +2591,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info,
if (elf_section_data (sec)->this_hdr.contents != contents)
{
if (!converted && !_bfd_link_keep_memory (info))
- free (contents);
+ _bfd_elf_munmap_section_contents (sec, contents);
else
{
/* Cache the section contents for elf_link_input_bfd if any
@@ -2609,7 +2609,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info,
error_return:
if (elf_section_data (sec)->this_hdr.contents != contents)
- free (contents);
+ _bfd_elf_munmap_section_contents (sec, contents);
sec->check_relocs_failed = 1;
return false;
}
@@ -5274,7 +5274,7 @@ elf_x86_64_get_synthetic_symtab (bfd *abfd,
continue;
/* Get the PLT section contents. */
- if (!bfd_malloc_and_get_section (abfd, plt, &plt_contents))
+ if (!_bfd_elf_mmap_section (abfd, plt, &plt_contents))
break;
/* Check what kind of PLT it is. */
@@ -5367,7 +5367,7 @@ elf_x86_64_get_synthetic_symtab (bfd *abfd,
if (plt_type == plt_unknown)
{
- free (plt_contents);
+ _bfd_elf_munmap_section_contents (plt, plt_contents);
continue;
}
diff --git a/bfd/elfcode.h b/bfd/elfcode.h
index 1e0784611bc..6c2ad904fc0 100644
--- a/bfd/elfcode.h
+++ b/bfd/elfcode.h
@@ -1195,6 +1195,7 @@ elf_checksum_contents (bfd *abfd,
Elf_Internal_Shdr i_shdr;
Elf_External_Shdr x_shdr;
bfd_byte *contents, *free_contents;
+ asection *sec = NULL;
i_shdr = *i_shdrp[count];
i_shdr.sh_offset = 0;
@@ -1210,8 +1211,6 @@ elf_checksum_contents (bfd *abfd,
contents = i_shdr.contents;
if (contents == NULL)
{
- asection *sec;
-
sec = bfd_section_from_elf_index (abfd, count);
if (sec != NULL)
{
@@ -1220,7 +1219,7 @@ elf_checksum_contents (bfd *abfd,
{
/* Force rereading from file. */
sec->flags &= ~SEC_IN_MEMORY;
- if (!bfd_malloc_and_get_section (abfd, sec, &free_contents))
+ if (!_bfd_elf_mmap_section (abfd, sec, &free_contents))
continue;
contents = free_contents;
}
@@ -1229,7 +1228,7 @@ elf_checksum_contents (bfd *abfd,
if (contents != NULL)
{
(*process) (contents, i_shdr.sh_size, arg);
- free (free_contents);
+ _bfd_elf_munmap_section_contents (sec, free_contents);
}
}
diff --git a/bfd/elflink.c b/bfd/elflink.c
index 42029f29f7a..47fb890f94f 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -4426,10 +4426,10 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info)
unsigned int elfsec;
unsigned long shlink;
- if (!bfd_malloc_and_get_section (abfd, s, &dynbuf))
+ if (!_bfd_elf_mmap_section (abfd, s, &dynbuf))
{
error_free_dyn:
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
goto error_return;
}
@@ -4535,7 +4535,7 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info)
elf_tdata (abfd)->is_pie = (dyn.d_un.d_val & DF_1_PIE) != 0;
}
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
}
/* DT_RUNPATH overrides DT_RPATH. Do _NOT_ bfd_release, as that
@@ -8283,7 +8283,7 @@ bfd_elf_get_bfd_needed_list (bfd *abfd,
if (s == NULL || s->size == 0 || (s->flags & SEC_HAS_CONTENTS) == 0)
return true;
- if (!bfd_malloc_and_get_section (abfd, s, &dynbuf))
+ if (!_bfd_elf_mmap_section (abfd, s, &dynbuf))
goto error_return;
elfsec = _bfd_elf_section_from_bfd_section (abfd, s);
@@ -8329,12 +8329,12 @@ bfd_elf_get_bfd_needed_list (bfd *abfd,
}
}
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
return true;
error_return:
- free (dynbuf);
+ _bfd_elf_munmap_section_contents (s, dynbuf);
return false;
}
diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h
index 1e6992b5793..89e3d36adb2 100644
--- a/bfd/elfxx-target.h
+++ b/bfd/elfxx-target.h
@@ -148,6 +148,9 @@
#ifndef elf_backend_strtab_flags
#define elf_backend_strtab_flags 0
#endif
+#ifndef elf_backend_use_mmap
+#define elf_backend_use_mmap false
+#endif
#define bfd_elfNN_bfd_debug_info_start _bfd_void_bfd
#define bfd_elfNN_bfd_debug_info_end _bfd_void_bfd
@@ -974,7 +977,8 @@ static const struct elf_backend_data elfNN_bed =
elf_backend_extern_protected_data,
elf_backend_always_renumber_dynsyms,
elf_backend_linux_prpsinfo32_ugid16,
- elf_backend_linux_prpsinfo64_ugid16
+ elf_backend_linux_prpsinfo64_ugid16,
+ elf_backend_use_mmap
};
/* Forward declaration for use when initialising alternative_target field. */
diff --git a/bfd/elfxx-x86.c b/bfd/elfxx-x86.c
index 508fd771da3..ed9822628a6 100644
--- a/bfd/elfxx-x86.c
+++ b/bfd/elfxx-x86.c
@@ -1566,9 +1566,8 @@ elf_x86_size_or_finish_relative_reloc
= elf_section_data (sec)->this_hdr.contents;
else
{
- if (!bfd_malloc_and_get_section (sec->owner,
- sec,
- &contents))
+ if (!_bfd_elf_mmap_section (sec->owner,
+ sec, &contents))
info->callbacks->einfo
/* xgettext:c-format */
(_("%F%P: %pB: failed to allocate memory for section `%pA'\n"),
@@ -3789,7 +3788,7 @@ _bfd_x86_elf_get_synthetic_symtab (bfd *abfd,
count = n;
for (j = 0; plts[j].name != NULL; j++)
- free (plts[j].contents);
+ _bfd_elf_munmap_section_contents (plts[j].sec, plts[j].contents);
free (dynrelbuf);
diff --git a/bfd/elfxx-x86.h b/bfd/elfxx-x86.h
index b3af9b841ba..8680e3c29b5 100644
--- a/bfd/elfxx-x86.h
+++ b/bfd/elfxx-x86.h
@@ -960,6 +960,7 @@ extern void _bfd_x86_elf_link_report_relative_reloc
_bfd_elf_x86_size_relative_relocs
#define elf_backend_finish_relative_relocs \
_bfd_elf_x86_finish_relative_relocs
+#define elf_backend_use_mmap true
#define ELF_P_ALIGN ELF_MINPAGESIZE
diff --git a/bfd/libbfd.c b/bfd/libbfd.c
index 237f91e5c97..c847c4f0180 100644
--- a/bfd/libbfd.c
+++ b/bfd/libbfd.c
@@ -21,6 +21,7 @@
#include "sysdep.h"
#include "bfd.h"
+#include "elf-bfd.h"
#include "libbfd.h"
#include "objalloc.h"
@@ -1202,6 +1203,19 @@ _bfd_generic_get_section_contents (bfd *abfd,
return false;
}
+#ifdef USE_MMAP
+ if (section->mmapped_p
+ && (section->contents != NULL || location != NULL))
+ {
+ _bfd_error_handler
+ /* xgettext:c-format */
+ (_("%pB: mapped section %pA has non-NULLL buffer"),
+ abfd, section);
+ bfd_set_error (bfd_error_invalid_operation);
+ return false;
+ }
+#endif
+
sz = bfd_get_section_limit_octets (abfd, section);
if (offset + count < count
|| offset + count > sz
@@ -1214,8 +1228,49 @@ _bfd_generic_get_section_contents (bfd *abfd,
return false;
}
- if (bfd_seek (abfd, section->filepos + offset, SEEK_SET) != 0
- || bfd_read (location, count, abfd) != count)
+ if (bfd_seek (abfd, section->filepos + offset, SEEK_SET) != 0)
+ return false;
+
+#ifdef USE_MMAP
+ if (section->mmapped_p)
+ {
+ if (location != 0
+ || bfd_get_flavour (abfd) != bfd_target_elf_flavour)
+ abort ();
+
+ int prot = ((section->reloc_count == 0)
+ ? PROT_READ : PROT_READ | PROT_WRITE);
+
+ location = bfd_mmap_local
+ (abfd, count, prot, &elf_section_data (section)->contents_addr,
+ &elf_section_data (section)->contents_size);
+
+ if (location == NULL)
+ return false;
+
+ /* Check for the unknown input file. */
+ if (location != (void *) -1)
+ {
+ section->contents = location;
+ return true;
+ }
+
+ /* Malloc the buffer and call bfd_read. */
+ location = (bfd_byte *) bfd_malloc (count);
+ if (location == NULL)
+ {
+ if (bfd_get_error () == bfd_error_no_memory)
+ _bfd_error_handler
+ /* xgettext:c-format */
+ (_("error: %pB(%pA) is too large (%#" PRIx64 " bytes)"),
+ abfd, section, (uint64_t) count);
+ return false;
+ }
+ section->contents = location;
+ }
+#endif
+
+ if (bfd_read (location, count, abfd) != count)
return false;
return true;
diff --git a/bfd/opncls.c b/bfd/opncls.c
index ac74607bac0..582828ab27f 100644
--- a/bfd/opncls.c
+++ b/bfd/opncls.c
@@ -168,6 +168,15 @@ static void
_bfd_delete_bfd (bfd *abfd)
{
#ifdef USE_MMAP
+ if (bfd_get_flavour (abfd) == bfd_target_elf_flavour)
+ {
+ asection *sec;
+ for (sec = abfd->sections; sec != NULL; sec = sec->next)
+ if (sec->mmapped_p)
+ munmap (elf_section_data (sec)->contents_addr,
+ elf_section_data (sec)->contents_size);
+ }
+
struct bfd_mmapped *mmapped, *next;
for (mmapped = abfd->mmapped; mmapped != NULL; mmapped = next)
{
diff --git a/bfd/section.c b/bfd/section.c
index 8cd30e80f2b..604105b39c4 100644
--- a/bfd/section.c
+++ b/bfd/section.c
@@ -422,6 +422,9 @@ CODE_FRAGMENT
. {* Nonzero if this section uses RELA relocations, rather than REL. *}
. unsigned int use_rela_p:1;
.
+. {* Nonzero if this section contents are mmapped, rather than malloced. *}
+. unsigned int mmapped_p:1;
+.
. {* Bits used by various backends. The generic code doesn't touch
. these fields. *}
.
@@ -711,8 +714,8 @@ EXTERNAL
. {* linker_mark, linker_has_input, gc_mark, decompress_status, *} \
. 0, 0, 1, 0, \
. \
-. {* segment_mark, sec_info_type, use_rela_p, *} \
-. 0, 0, 0, \
+. {* segment_mark, sec_info_type, use_rela_p, mmapped_p, *} \
+. 0, 0, 0, 0, \
. \
. {* sec_flg0, sec_flg1, sec_flg2, sec_flg3, sec_flg4, sec_flg5, *} \
. 0, 0, 0, 0, 0, 0, \
@@ -1625,6 +1628,8 @@ DESCRIPTION
bool
bfd_malloc_and_get_section (bfd *abfd, sec_ptr sec, bfd_byte **buf)
{
+ if (sec->mmapped_p)
+ abort ();
*buf = NULL;
return bfd_get_full_section_contents (abfd, sec, buf);
}
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 5/6] elf: Use mmap to map in symbol and relocation tables
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
` (3 preceding siblings ...)
2024-03-06 20:47 ` [PATCH v3 4/6] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 20:47 ` [PATCH v3 6/6] elf: Don't cache symbol nor relocation tables with mmap H.J. Lu
2024-03-06 23:00 ` [PATCH v3 0/6] elf: Use mmap to map in section contents Alan Modra
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
Add _bfd_mmap_read_untracked to mmap in symbol tables and relocations
whose sizes >= 4 * page size. Don't cache external relocations when
mmap is used.
When mmap is used to map in all ELF sections, data to link the 3.5GB
clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM
is:
stdio mmap improvement
user 84.28 85.04 -0.9%
system 12.46 10.16 14%
total 96 95.35 0.7%
page faults 4837944 4047667 16%
and data to link the 275M cc1plus executable in GCC 14 stage 1 build
is:
user 5.22 5.27 -1%
system 0.94 0.84 11%
total 6.20 6.13 0.7%
page faults 361272 323377 10%
* elf.c (bfd_elf_get_elf_syms): Replace bfd_read with
_bfd_mmap_read_untracked.
* elflink.c (elf_link_read_relocs_from_section): Add 2 arguments
to return mmap memory address and size.
(_bfd_elf_link_info_read_relocs); Replace bfd_read with
_bfd_mmap_read_untracked.
(bfd_elf_final_link): Don't cache external relocations when mmap
is used.
* libbfd.c (_bfd_mmap_read_untracked ): New.
* libbfd-in.h (_bfd_mmap_read_untracked): Likewise.
* libbfd.h: Regenerated.
---
bfd/elf.c | 47 +++++++++++++++++++++++++++++++++++------------
bfd/elflink.c | 44 ++++++++++++++++++++++++++++++++------------
bfd/libbfd-in.h | 3 +++
bfd/libbfd.c | 33 +++++++++++++++++++++++++++++++++
bfd/libbfd.h | 3 +++
5 files changed, 106 insertions(+), 24 deletions(-)
diff --git a/bfd/elf.c b/bfd/elf.c
index e2e31a93950..7427dca2ba0 100644
--- a/bfd/elf.c
+++ b/bfd/elf.c
@@ -464,19 +464,30 @@ bfd_elf_get_elf_syms (bfd *ibfd,
goto out;
}
pos = symtab_hdr->sh_offset + symoffset * extsym_size;
+ size_t alloc_ext_size = amt;
if (extsym_buf == NULL)
{
- alloc_ext = bfd_malloc (amt);
- extsym_buf = alloc_ext;
+#ifdef USE_MMAP
+ if ((ibfd->flags & BFD_PLUGIN) != 0
+ || amt < _bfd_minimum_mmap_size)
+ {
+#endif
+ alloc_ext = bfd_malloc (amt);
+ extsym_buf = alloc_ext;
+#ifdef USE_MMAP
+ }
+#endif
}
- if (extsym_buf == NULL
- || bfd_seek (ibfd, pos, SEEK_SET) != 0
- || bfd_read (extsym_buf, amt, ibfd) != amt)
+
+ if (bfd_seek (ibfd, pos, SEEK_SET) != 0
+ || !_bfd_mmap_read_untracked (&extsym_buf, &alloc_ext_size,
+ &alloc_ext, ibfd))
{
intsym_buf = NULL;
goto out;
}
+ size_t alloc_extshndx_size = 0;
if (shndx_hdr == NULL || shndx_hdr->sh_size == 0)
extshndx_buf = NULL;
else
@@ -487,15 +498,27 @@ bfd_elf_get_elf_syms (bfd *ibfd,
intsym_buf = NULL;
goto out;
}
+ alloc_extshndx_size = amt;
pos = shndx_hdr->sh_offset + symoffset * sizeof (Elf_External_Sym_Shndx);
if (extshndx_buf == NULL)
{
- alloc_extshndx = (Elf_External_Sym_Shndx *) bfd_malloc (amt);
- extshndx_buf = alloc_extshndx;
+#ifdef USE_MMAP
+ if ((ibfd->flags & BFD_PLUGIN) != 0
+ || amt < _bfd_minimum_mmap_size)
+ {
+#endif
+ alloc_extshndx
+ = (Elf_External_Sym_Shndx *) bfd_malloc (amt);
+ extshndx_buf = alloc_extshndx;
+#ifdef USE_MMAP
+ }
+#endif
}
- if (extshndx_buf == NULL
- || bfd_seek (ibfd, pos, SEEK_SET) != 0
- || bfd_read (extshndx_buf, amt, ibfd) != amt)
+ if (bfd_seek (ibfd, pos, SEEK_SET) != 0
+ || !_bfd_mmap_read_untracked ((void **) &extshndx_buf,
+ &alloc_extshndx_size,
+ (void **) &alloc_extshndx,
+ ibfd))
{
intsym_buf = NULL;
goto out;
@@ -534,8 +557,8 @@ bfd_elf_get_elf_syms (bfd *ibfd,
}
out:
- free (alloc_ext);
- free (alloc_extshndx);
+ _bfd_munmap_readonly_untracked (alloc_ext, alloc_ext_size);
+ _bfd_munmap_readonly_untracked (alloc_extshndx, alloc_extshndx_size);
return intsym_buf;
}
diff --git a/bfd/elflink.c b/bfd/elflink.c
index 47fb890f94f..4602fb3d10f 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -2644,8 +2644,11 @@ _bfd_elf_link_assign_sym_version (struct elf_link_hash_entry *h, void *data)
may be either a REL or a RELA section. The relocations are
translated into RELA relocations and stored in INTERNAL_RELOCS,
which should have already been allocated to contain enough space.
- The EXTERNAL_RELOCS are a buffer where the external form of the
- relocations should be stored.
+ The *EXTERNAL_RELOCS_P are a buffer where the external form of the
+ relocations should be stored. If *EXTERNAL_RELOCS_P is NULL,
+ *EXTERNAL_RELOCS_P and *EXTERNAL_RELOCS_SIZE_P returns the mmap
+ memory address and size. Otherwise, *EXTERNAL_RELOCS_SIZE_P is
+ unchanged and EXTERNAL_RELOCS_SIZE_P returns 0.
Returns FALSE if something goes wrong. */
@@ -2653,7 +2656,8 @@ static bool
elf_link_read_relocs_from_section (bfd *abfd,
asection *sec,
Elf_Internal_Shdr *shdr,
- void *external_relocs,
+ void **external_relocs_addr,
+ size_t *external_relocs_size_addr,
Elf_Internal_Rela *internal_relocs)
{
const struct elf_backend_data *bed;
@@ -2663,13 +2667,17 @@ elf_link_read_relocs_from_section (bfd *abfd,
Elf_Internal_Rela *irela;
Elf_Internal_Shdr *symtab_hdr;
size_t nsyms;
+ void *external_relocs = *external_relocs_addr;
/* Position ourselves at the start of the section. */
if (bfd_seek (abfd, shdr->sh_offset, SEEK_SET) != 0)
return false;
/* Read the relocations. */
- if (bfd_read (external_relocs, shdr->sh_size, abfd) != shdr->sh_size)
+ *external_relocs_size_addr = shdr->sh_size;
+ if (!_bfd_mmap_read_untracked (&external_relocs,
+ external_relocs_size_addr,
+ external_relocs_addr, abfd))
return false;
symtab_hdr = &elf_tdata (abfd)->symtab_hdr;
@@ -2754,6 +2762,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd,
bool keep_memory)
{
void *alloc1 = NULL;
+ size_t alloc1_size;
Elf_Internal_Rela *alloc2 = NULL;
const struct elf_backend_data *bed = get_elf_backend_data (abfd);
struct bfd_elf_section_data *esdo = elf_section_data (o);
@@ -2791,17 +2800,26 @@ _bfd_elf_link_info_read_relocs (bfd *abfd,
if (esdo->rela.hdr)
size += esdo->rela.hdr->sh_size;
- alloc1 = bfd_malloc (size);
- if (alloc1 == NULL)
- goto error_return;
- external_relocs = alloc1;
+#ifdef USE_MMAP
+ if (size < _bfd_minimum_mmap_size)
+ {
+#endif
+ alloc1 = bfd_malloc (size);
+ if (alloc1 == NULL)
+ goto error_return;
+ external_relocs = alloc1;
+#ifdef USE_MMAP
+ }
+#endif
}
+ else
+ alloc1 = external_relocs;
internal_rela_relocs = internal_relocs;
if (esdo->rel.hdr)
{
if (!elf_link_read_relocs_from_section (abfd, o, esdo->rel.hdr,
- external_relocs,
+ &alloc1, &alloc1_size,
internal_relocs))
goto error_return;
external_relocs = (((bfd_byte *) external_relocs)
@@ -2812,7 +2830,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd,
if (esdo->rela.hdr
&& (!elf_link_read_relocs_from_section (abfd, o, esdo->rela.hdr,
- external_relocs,
+ &alloc1, &alloc1_size,
internal_rela_relocs)))
goto error_return;
@@ -2820,7 +2838,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd,
if (keep_memory)
esdo->relocs = internal_relocs;
- free (alloc1);
+ _bfd_munmap_readonly_untracked (alloc1, alloc1_size);
/* Don't free alloc2, since if it was allocated we are passing it
back (under the name of internal_relocs). */
@@ -2828,7 +2846,7 @@ _bfd_elf_link_info_read_relocs (bfd *abfd,
return internal_relocs;
error_return:
- free (alloc1);
+ _bfd_munmap_readonly_untracked (alloc1, alloc1_size);
if (alloc2 != NULL)
{
if (keep_memory)
@@ -12741,12 +12759,14 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info)
goto error_return;
}
+#ifndef USE_MMAP
if (max_external_reloc_size != 0)
{
flinfo.external_relocs = bfd_malloc (max_external_reloc_size);
if (flinfo.external_relocs == NULL)
goto error_return;
}
+#endif
if (max_internal_reloc_count != 0)
{
diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h
index 7887fad9c92..effe1b86b53 100644
--- a/bfd/libbfd-in.h
+++ b/bfd/libbfd-in.h
@@ -905,6 +905,9 @@ extern void _bfd_munmap_readonly_untracked
#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr)
#endif
+extern bool _bfd_mmap_read_untracked
+ (void **, size_t *, void **, bfd *) ATTRIBUTE_HIDDEN;
+
static inline void *
_bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
{
diff --git a/bfd/libbfd.c b/bfd/libbfd.c
index c847c4f0180..0a31f113999 100644
--- a/bfd/libbfd.c
+++ b/bfd/libbfd.c
@@ -1180,6 +1180,39 @@ _bfd_mmap_readonly_tracked (bfd *abfd, size_t rsize)
}
#endif
+/* Attempt to read *SIZE_ADDR bytes from ABFD's iostream to *PTR_P.
+ Return true if the full the amount has been read. If *PTR_P is
+ NULL, mmap should be used, return the memory address at the
+ current offset in *PTR_P as well as return mmap address and size
+ in *PTR_ADDR and *SIZE_ADDR. Otherwise, return NULL in *PTR_ADDR
+ and 0 in *SIZE_ADDR. */
+
+bool
+_bfd_mmap_read_untracked (void **ptr_p, size_t *size_addr,
+ void **ptr_addr, bfd *abfd)
+{
+ void *ptr = *ptr_p;
+ size_t size = *size_addr;
+
+#ifdef USE_MMAP
+ if (ptr == NULL)
+ {
+ ptr = _bfd_mmap_readonly_untracked (abfd, size, ptr_addr,
+ size_addr);
+ if (ptr == NULL || ptr == (void *) -1)
+ abort ();
+ *ptr_p = ptr;
+ return true;
+ }
+ else
+#endif
+ {
+ *ptr_addr = NULL;
+ *size_addr = 0;
+ return bfd_read (ptr, size, abfd) == size;
+ }
+}
+
/* Default implementation */
bool
diff --git a/bfd/libbfd.h b/bfd/libbfd.h
index 1515a03b093..c80f5a86ed1 100644
--- a/bfd/libbfd.h
+++ b/bfd/libbfd.h
@@ -911,6 +911,9 @@ extern void _bfd_munmap_readonly_untracked
#define _bfd_munmap_readonly_untracked(ptr, rsize) free (ptr)
#endif
+extern bool _bfd_mmap_read_untracked
+ (void **, size_t *, void **, bfd *) ATTRIBUTE_HIDDEN;
+
static inline void *
_bfd_malloc_and_read (bfd *abfd, bfd_size_type asize, bfd_size_type rsize)
{
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 6/6] elf: Don't cache symbol nor relocation tables with mmap
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
` (4 preceding siblings ...)
2024-03-06 20:47 ` [PATCH v3 5/6] elf: Use mmap to map in symbol and relocation tables H.J. Lu
@ 2024-03-06 20:47 ` H.J. Lu
2024-03-06 23:00 ` [PATCH v3 0/6] elf: Use mmap to map in section contents Alan Modra
6 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 20:47 UTC (permalink / raw)
To: binutils; +Cc: goldstein.w.n, amodra
During a "-j 8" LLVM 17 debug build on a machine with 32GB RAM and 16GB
swap, ld was killed by kernel because of out of memory:
[79437.949336] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-9.scope,task=ld,pid=797431,uid=1000
[79437.949349] Out of memory: Killed process 797431 (ld) total-vm:9219600kB, anon-rss:6558156kB, file-rss:1792kB, shmem-rss:0kB, UID:1000 pgtables:17552kB oom_score_adj:0
Don't cache symbol nor relocation tables if they are mapped in. Data to
link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64
with 32GB RAM is:
stdio mmap improvement
user 86.73 87.02 -0.3%
system 9.55 9.21 3.6%
total 100.40 97.66 0.7%
maximum set(GB) 17.34 13.14 24%
page faults 4047667 3042877 25%
and data to link the 275M cc1plus executable in GCC 14 stage 1 build is:
user 5.41 5.44 -0.5%
system 0.80 0.76 5%
total 6.25 6.26 -0.2%
maximum set(MB) 1323 968 27%
page faults 323451 236371 27%
These improve the overall system performance for parallel build by
reducing memory usage and page faults.
Also rename _bfd_link_keep_memory to _bfd_elf_link_keep_memory.
* elf-bfd.h (_bfd_elf_link_keep_memory): New.
* elf32-i386.c (elf_i386_scan_relocs): Replace
_bfd_link_keep_memory with
_bfd_elf_link_keep_memory.
* elf64-x86-64.c (elf_x86_64_scan_relocs): Likewise.
* elflink.c (_bfd_elf_link_iterate_on_relocs): Likewise.
(elf_link_add_object_symbols): Likewise.
(init_reloc_cookie): Likewise.
(_bfd_elf_link_keep_memory): New.
* libbfd-in.h (_bfd_link_keep_memory): Removed.
* linker.c (_bfd_link_keep_memory): Likewise.
* libbfd.h: Regenerated.
---
bfd/elf-bfd.h | 3 +++
bfd/elf32-i386.c | 2 +-
bfd/elf64-x86-64.c | 2 +-
bfd/elflink.c | 62 +++++++++++++++++++++++++++++++++++++---------
bfd/libbfd-in.h | 3 ---
bfd/libbfd.h | 3 ---
bfd/linker.c | 35 --------------------------
7 files changed, 55 insertions(+), 55 deletions(-)
diff --git a/bfd/elf-bfd.h b/bfd/elf-bfd.h
index da7c5208017..6ed6f13cba2 100644
--- a/bfd/elf-bfd.h
+++ b/bfd/elf-bfd.h
@@ -3144,6 +3144,9 @@ extern bool _bfd_elf_mmap_section
extern void _bfd_elf_munmap_section_contents
(asection *, void *);
+extern bool _bfd_elf_link_keep_memory
+ (struct bfd_link_info *);
+
/* Large common section. */
extern asection _bfd_elf_large_com_section;
diff --git a/bfd/elf32-i386.c b/bfd/elf32-i386.c
index 8aa1533d0ba..606e38b472c 100644
--- a/bfd/elf32-i386.c
+++ b/bfd/elf32-i386.c
@@ -1932,7 +1932,7 @@ elf_i386_scan_relocs (bfd *abfd,
if (elf_section_data (sec)->this_hdr.contents != contents)
{
- if (!converted && !_bfd_link_keep_memory (info))
+ if (!converted && !_bfd_elf_link_keep_memory (info))
_bfd_elf_munmap_section_contents (sec, contents);
else
{
diff --git a/bfd/elf64-x86-64.c b/bfd/elf64-x86-64.c
index f1e06040a5a..e52fa932ffc 100644
--- a/bfd/elf64-x86-64.c
+++ b/bfd/elf64-x86-64.c
@@ -2590,7 +2590,7 @@ elf_x86_64_scan_relocs (bfd *abfd, struct bfd_link_info *info,
if (elf_section_data (sec)->this_hdr.contents != contents)
{
- if (!converted && !_bfd_link_keep_memory (info))
+ if (!converted && !_bfd_elf_link_keep_memory (info))
_bfd_elf_munmap_section_contents (sec, contents);
else
{
diff --git a/bfd/elflink.c b/bfd/elflink.c
index 4602fb3d10f..019d033805d 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -4205,10 +4205,9 @@ _bfd_elf_link_iterate_on_relocs
|| bfd_is_abs_section (o->output_section))
continue;
- internal_relocs = _bfd_elf_link_info_read_relocs (abfd, info,
- o, NULL,
- NULL,
- _bfd_link_keep_memory (info));
+ internal_relocs = _bfd_elf_link_info_read_relocs
+ (abfd, info, o, NULL, NULL,
+ _bfd_elf_link_keep_memory (info));
if (internal_relocs == NULL)
return false;
@@ -5574,10 +5573,9 @@ elf_link_add_object_symbols (bfd *abfd, struct bfd_link_info *info)
&& (s->flags & SEC_DEBUGGING) != 0))
continue;
- internal_relocs = _bfd_elf_link_info_read_relocs (abfd, info,
- s, NULL,
- NULL,
- _bfd_link_keep_memory (info));
+ internal_relocs = _bfd_elf_link_info_read_relocs
+ (abfd, info, s, NULL, NULL,
+ _bfd_elf_link_keep_memory (info));
if (internal_relocs == NULL)
goto error_free_vers;
@@ -13630,7 +13628,7 @@ init_reloc_cookie (struct elf_reloc_cookie *cookie,
info->callbacks->einfo (_("%P%X: can not read symbols: %E\n"));
return false;
}
- if (_bfd_link_keep_memory (info) )
+ if (_bfd_elf_link_keep_memory (info) )
{
symtab_hdr->contents = (bfd_byte *) cookie->locsyms;
info->cache_size += (cookie->locsymcount
@@ -13667,9 +13665,9 @@ init_reloc_cookie_rels (struct elf_reloc_cookie *cookie,
}
else
{
- cookie->rels = _bfd_elf_link_info_read_relocs (abfd, info, sec,
- NULL, NULL,
- _bfd_link_keep_memory (info));
+ cookie->rels = _bfd_elf_link_info_read_relocs
+ (abfd, info, sec, NULL, NULL,
+ _bfd_elf_link_keep_memory (info));
if (cookie->rels == NULL)
return false;
cookie->rel = cookie->rels;
@@ -15599,3 +15597,43 @@ _bfd_elf_add_dynamic_tags (bfd *output_bfd, struct bfd_link_info *info,
return true;
}
+
+/* Return false if linker should avoid caching relocation information
+ and symbol tables of input files in memory. */
+
+bool
+_bfd_elf_link_keep_memory (struct bfd_link_info *info ATTRIBUTE_UNUSED)
+{
+#ifdef USE_MMAP
+ /* Don't cache symbol nor relocation tables if they are mapped in. */
+ return false;
+#else
+ bfd *abfd;
+ bfd_size_type size;
+
+ if (!info->keep_memory)
+ return false;
+
+ if (info->max_cache_size == (bfd_size_type) -1)
+ return true;
+
+ abfd = info->input_bfds;
+ size = info->cache_size;
+ do
+ {
+ if (size >= info->max_cache_size)
+ {
+ /* Over the limit. Reduce the memory usage. */
+ info->keep_memory = false;
+ return false;
+ }
+ if (!abfd)
+ break;
+ size += abfd->alloc_size;
+ abfd = abfd->link.next;
+ }
+ while (1);
+
+ return true;
+#endif
+}
diff --git a/bfd/libbfd-in.h b/bfd/libbfd-in.h
index effe1b86b53..f89de00b551 100644
--- a/bfd/libbfd-in.h
+++ b/bfd/libbfd-in.h
@@ -848,9 +848,6 @@ extern bfd_byte * _bfd_write_unsigned_leb128
extern struct bfd_link_info *_bfd_get_link_info (bfd *)
ATTRIBUTE_HIDDEN;
-extern bool _bfd_link_keep_memory (struct bfd_link_info *)
- ATTRIBUTE_HIDDEN;
-
extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN;
extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN;
extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN;
diff --git a/bfd/libbfd.h b/bfd/libbfd.h
index c80f5a86ed1..3ed2e55a34b 100644
--- a/bfd/libbfd.h
+++ b/bfd/libbfd.h
@@ -854,9 +854,6 @@ extern bfd_byte * _bfd_write_unsigned_leb128
extern struct bfd_link_info *_bfd_get_link_info (bfd *)
ATTRIBUTE_HIDDEN;
-extern bool _bfd_link_keep_memory (struct bfd_link_info *)
- ATTRIBUTE_HIDDEN;
-
extern uintptr_t _bfd_pagesize ATTRIBUTE_HIDDEN;
extern uintptr_t _bfd_pagesize_m1 ATTRIBUTE_HIDDEN;
extern uintptr_t _bfd_minimum_mmap_size ATTRIBUTE_HIDDEN;
diff --git a/bfd/linker.c b/bfd/linker.c
index 36cca9624c2..eb42a78b622 100644
--- a/bfd/linker.c
+++ b/bfd/linker.c
@@ -3556,38 +3556,3 @@ _bfd_nolink_bfd_define_start_stop (struct bfd_link_info *info ATTRIBUTE_UNUSED,
{
return (struct bfd_link_hash_entry *) _bfd_ptr_bfd_null_error (sec->owner);
}
-
-/* Return false if linker should avoid caching relocation infomation
- and symbol tables of input files in memory. */
-
-bool
-_bfd_link_keep_memory (struct bfd_link_info * info)
-{
- bfd *abfd;
- bfd_size_type size;
-
- if (!info->keep_memory)
- return false;
-
- if (info->max_cache_size == (bfd_size_type) -1)
- return true;
-
- abfd = info->input_bfds;
- size = info->cache_size;
- do
- {
- if (size >= info->max_cache_size)
- {
- /* Over the limit. Reduce the memory usage. */
- info->keep_memory = false;
- return false;
- }
- if (!abfd)
- break;
- size += abfd->alloc_size;
- abfd = abfd->link.next;
- }
- while (1);
-
- return true;
-}
--
2.44.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/6] elf: Use mmap to map in section contents
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
` (5 preceding siblings ...)
2024-03-06 20:47 ` [PATCH v3 6/6] elf: Don't cache symbol nor relocation tables with mmap H.J. Lu
@ 2024-03-06 23:00 ` Alan Modra
2024-03-06 23:26 ` H.J. Lu
6 siblings, 1 reply; 9+ messages in thread
From: Alan Modra @ 2024-03-06 23:00 UTC (permalink / raw)
To: H.J. Lu; +Cc: binutils
On Wed, Mar 06, 2024 at 12:47:25PM -0800, H.J. Lu wrote:
> Changes in v3:
I ran my target tests against this version and found two regressions.
mipsisa32r2el-elf +FAIL: MIPS eh-frame 3
mipstx39-elf +FAIL: MIPS eh-frame 3
The keep_memory change causes this code in
bfd/elfxx-mips.c:_bfd_mips_elf_eh_frame_address_size to fail:
if (sec->reloc_count > 0
&& elf_section_data (sec)->relocs != NULL
&& (ELF32_R_TYPE (elf_section_data (sec)->relocs[0].r_info)
== R_MIPS_64))
return 8;
Yes, the test also fails with --no-keep-memory so this is really a bug
in the mips backend, but we should try to not break this code.
--
Alan Modra
Australia Development Lab, IBM
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v3 0/6] elf: Use mmap to map in section contents
2024-03-06 23:00 ` [PATCH v3 0/6] elf: Use mmap to map in section contents Alan Modra
@ 2024-03-06 23:26 ` H.J. Lu
0 siblings, 0 replies; 9+ messages in thread
From: H.J. Lu @ 2024-03-06 23:26 UTC (permalink / raw)
To: Alan Modra; +Cc: binutils
On Wed, Mar 6, 2024 at 3:00 PM Alan Modra <amodra@gmail.com> wrote:
>
> On Wed, Mar 06, 2024 at 12:47:25PM -0800, H.J. Lu wrote:
> > Changes in v3:
>
> I ran my target tests against this version and found two regressions.
> mipsisa32r2el-elf +FAIL: MIPS eh-frame 3
> mipstx39-elf +FAIL: MIPS eh-frame 3
>
> The keep_memory change causes this code in
> bfd/elfxx-mips.c:_bfd_mips_elf_eh_frame_address_size to fail:
>
> if (sec->reloc_count > 0
> && elf_section_data (sec)->relocs != NULL
> && (ELF32_R_TYPE (elf_section_data (sec)->relocs[0].r_info)
> == R_MIPS_64))
> return 8;
>
> Yes, the test also fails with --no-keep-memory so this is really a bug
> in the mips backend, but we should try to not break this code.
>
I changed it to
bool
_bfd_elf_link_keep_memory (struct bfd_link_info *info)
{
#ifdef USE_MMAP
/* Don't cache symbol nor relocation tables if they are mapped in.
NB: Since the --no-keep-memory linker option causes
FAIL: MIPS eh-frame 3
in the linker testsuite for mipsisa32r2el-elf target, this is
opt-in by each backend. */
const struct elf_backend_data *bed
= get_elf_backend_data (info->output_bfd);
if (bed->use_mmap)
return false;
#endif
The v4 patch is at
https://patchwork.sourceware.org/project/binutils/list/?series=31668
--
H.J.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-03-06 23:27 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-06 20:47 [PATCH v3 0/6] elf: Use mmap to map in section contents H.J. Lu
2024-03-06 20:47 ` [PATCH v3 1/6] bfd: Don't hard-code BFD_JUMP_TABLE_COPY H.J. Lu
2024-03-06 20:47 ` [PATCH v3 2/6] bfd: Change the --with-mmap default to true H.J. Lu
2024-03-06 20:47 ` [PATCH v3 3/6] elf: Use mmap to map in read-only sections H.J. Lu
2024-03-06 20:47 ` [PATCH v3 4/6] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents H.J. Lu
2024-03-06 20:47 ` [PATCH v3 5/6] elf: Use mmap to map in symbol and relocation tables H.J. Lu
2024-03-06 20:47 ` [PATCH v3 6/6] elf: Don't cache symbol nor relocation tables with mmap H.J. Lu
2024-03-06 23:00 ` [PATCH v3 0/6] elf: Use mmap to map in section contents Alan Modra
2024-03-06 23:26 ` H.J. Lu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).