From: "H.J. Lu" <hjl.tools@gmail.com>
To: binutils@sourceware.org
Cc: goldstein.w.n@gmail.com
Subject: [PATCH 4/4] elf: Don't cache symbol nor relocation tables with mmap
Date: Tue, 5 Mar 2024 09:00:57 -0800 [thread overview]
Message-ID: <20240305170057.292800-5-hjl.tools@gmail.com> (raw)
In-Reply-To: <20240305170057.292800-1-hjl.tools@gmail.com>
During a "-j 8" LLVM 17 debug build on a machine with 32GB RAM and 16GB
swap, ld was killed by kernel because of out of memory:
[79437.949336] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-9.scope,task=ld,pid=797431,uid=1000
[79437.949349] Out of memory: Killed process 797431 (ld) total-vm:9219600kB, anon-rss:6558156kB, file-rss:1792kB, shmem-rss:0kB, UID:1000 pgtables:17552kB oom_score_adj:0
Don't cache symbol nor relocation tables if they are mapped in. Data to
link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64
with 32GB RAM is:
stdio mmap improvement
user 86.73 87.02 -0.3%
system 9.55 9.21 3.6%
total 100.40 97.66 0.7%
maximum set(GB) 17.34 13.14 24%
page faults 4047667 3042877 25%
and data to link the 275M cc1plus executable in GCC 14 stage 1 build is:
user 5.41 5.44 -0.5%
system 0.80 0.76 5%
total 6.25 6.26 -0.2%
maximum set(MB) 1323 968 27%
page faults 323451 236371 27%
These improve the overall system performance for parallel build by
reducing memory usage and page faults.
* linker.c (_bfd_link_keep_memory): Return false if HAVE_MMAP
is defined.
---
bfd/linker.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/bfd/linker.c b/bfd/linker.c
index 36cca9624c2..d6145b3272e 100644
--- a/bfd/linker.c
+++ b/bfd/linker.c
@@ -3561,8 +3561,12 @@ _bfd_nolink_bfd_define_start_stop (struct bfd_link_info *info ATTRIBUTE_UNUSED,
and symbol tables of input files in memory. */
bool
-_bfd_link_keep_memory (struct bfd_link_info * info)
+_bfd_link_keep_memory (struct bfd_link_info * info ATTRIBUTE_UNUSED)
{
+#ifdef HAVE_MMAP
+ /* Don't cache symbol nor relocation tables if they are mapped in. */
+ return false;
+#else
bfd *abfd;
bfd_size_type size;
@@ -3590,4 +3594,5 @@ _bfd_link_keep_memory (struct bfd_link_info * info)
while (1);
return true;
+#endif
}
--
2.44.0
next prev parent reply other threads:[~2024-03-05 17:01 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-05 17:00 [PATCH 0/4] elf: Use mmap to map in section contents H.J. Lu
2024-03-05 17:00 ` [PATCH 1/4] elf: Use mmap to map in read-only sections H.J. Lu
2024-03-05 17:00 ` [PATCH 2/4] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents H.J. Lu
2024-03-05 17:00 ` [PATCH 3/4] elf: Use mmap to map in symbol and relocation tables H.J. Lu
2024-03-05 17:00 ` H.J. Lu [this message]
2024-03-05 17:03 ` [PATCH 0/4] elf: Use mmap to map in section contents Noah Goldstein
2024-03-06 0:50 ` Alan Modra
2024-03-06 16:16 ` H.J. Lu
2024-03-06 20:51 ` H.J. Lu
2024-03-06 4:45 ` Sam James
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240305170057.292800-5-hjl.tools@gmail.com \
--to=hjl.tools@gmail.com \
--cc=binutils@sourceware.org \
--cc=goldstein.w.n@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).