public inbox for binutils@sourceware.org
 help / color / mirror / Atom feed
From: "H.J. Lu" <hjl.tools@gmail.com>
To: binutils@sourceware.org
Cc: goldstein.w.n@gmail.com
Subject: [PATCH 4/4] elf: Don't cache symbol nor relocation tables with mmap
Date: Tue,  5 Mar 2024 09:00:57 -0800	[thread overview]
Message-ID: <20240305170057.292800-5-hjl.tools@gmail.com> (raw)
In-Reply-To: <20240305170057.292800-1-hjl.tools@gmail.com>

During a "-j 8" LLVM 17 debug build on a machine with 32GB RAM and 16GB
swap, ld was killed by kernel because of out of memory:

[79437.949336] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-9.scope,task=ld,pid=797431,uid=1000
[79437.949349] Out of memory: Killed process 797431 (ld) total-vm:9219600kB, anon-rss:6558156kB, file-rss:1792kB, shmem-rss:0kB, UID:1000 pgtables:17552kB oom_score_adj:0

Don't cache symbol nor relocation tables if they are mapped in.  Data to
link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64
with 32GB RAM is:

		stdio		mmap		improvement
user		86.73		87.02		-0.3%
system		9.55		9.21		3.6%
total		100.40		97.66		0.7%
maximum set(GB)	17.34		13.14		24%
page faults	4047667		3042877		25%

and data to link the 275M cc1plus executable in GCC 14 stage 1 build is:

user		5.41		5.44		-0.5%
system		0.80		0.76		5%
total		6.25		6.26		-0.2%
maximum set(MB)	1323		968		27%
page faults	323451		236371		27%

These improve the overall system performance for parallel build by
reducing memory usage and page faults.

	* linker.c (_bfd_link_keep_memory): Return false if HAVE_MMAP
	is defined.
---
 bfd/linker.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bfd/linker.c b/bfd/linker.c
index 36cca9624c2..d6145b3272e 100644
--- a/bfd/linker.c
+++ b/bfd/linker.c
@@ -3561,8 +3561,12 @@ _bfd_nolink_bfd_define_start_stop (struct bfd_link_info *info ATTRIBUTE_UNUSED,
    and symbol tables of input files in memory.  */
 
 bool
-_bfd_link_keep_memory (struct bfd_link_info * info)
+_bfd_link_keep_memory (struct bfd_link_info * info ATTRIBUTE_UNUSED)
 {
+#ifdef HAVE_MMAP
+  /* Don't cache symbol nor relocation tables if they are mapped in.  */
+  return false;
+#else
   bfd *abfd;
   bfd_size_type size;
 
@@ -3590,4 +3594,5 @@ _bfd_link_keep_memory (struct bfd_link_info * info)
   while (1);
 
   return true;
+#endif
 }
-- 
2.44.0


  parent reply	other threads:[~2024-03-05 17:01 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-03-05 17:00 [PATCH 0/4] elf: Use mmap to map in section contents H.J. Lu
2024-03-05 17:00 ` [PATCH 1/4] elf: Use mmap to map in read-only sections H.J. Lu
2024-03-05 17:00 ` [PATCH 2/4] elf: Add _bfd_elf_mmap_section and _bfd_elf_munmap_section_contents H.J. Lu
2024-03-05 17:00 ` [PATCH 3/4] elf: Use mmap to map in symbol and relocation tables H.J. Lu
2024-03-05 17:00 ` H.J. Lu [this message]
2024-03-05 17:03 ` [PATCH 0/4] elf: Use mmap to map in section contents Noah Goldstein
2024-03-06  0:50 ` Alan Modra
2024-03-06 16:16   ` H.J. Lu
2024-03-06 20:51     ` H.J. Lu
2024-03-06  4:45 ` Sam James

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240305170057.292800-5-hjl.tools@gmail.com \
    --to=hjl.tools@gmail.com \
    --cc=binutils@sourceware.org \
    --cc=goldstein.w.n@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).