From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by sourceware.org (Postfix) with ESMTPS id 6F799385828A for ; Tue, 5 Mar 2024 17:01:02 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6F799385828A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 6F799385828A Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::630 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709658065; cv=none; b=bxjhkrQnJlGWtl7+h+fE+Yopn3qqCw0pxQwV2xiWcQ0s2SLG+DwttQajzlRVUhdBCtCoNSKEwj2/DaQ4OaJmBQEbp8gOeGH2PDwTx3JpBTfo6NkNT48SENabT/yebCihfRzBbqSRKpIZ44PEqMgqyXdNLLN9oVt61+7QPOtH1cA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709658065; c=relaxed/simple; bh=uBi59mDsFsM9Q1UjiWOt+TLY04ZPGasOV+gTPH470NA=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=a4KpltWWhRIn8Gdv8k20dIaelwRpql2n1MQdtdOo+NpSVxCEp+q2EyM+Y8QzevomlmSJo7Lag643QEchXkv5dDqYDR3FrCFLK5ZRqJRJG0QC/vrJx6Z17GHIlK4moFxZ5z4i+gd6iI3sdSMbbzPeLBBwuBZQMUbtf76VXm0qxKQ= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1dbae7b8ff2so28792935ad.3 for ; Tue, 05 Mar 2024 09:01:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709658061; x=1710262861; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Xm4doBABAihMhsOyWbtEbfo5woMRB8YQ6cNpUkDOhE=; b=AXUzcScYZcibAjl5RdH2AUqSbyT3+dhi7SKgPJqbV3OgxgVM1koLIS/kW+VY503irG xcFNIZJxz0B4kquBM7n70foWgm1/LXI54Ivj7G5yGEGMGCqNU1PXtIy+ca1wUmsPC2T5 wZAVJR0uEPOt/OteZG+gnjMAF1kGndnGNFzYcfByalEdL8C2gqr4ScA6hBOfKmGt5EPi km1XxcwDWhW5q4rSFUsSeXk2s39x7kELiG5O6LGSxCDPsSEIGKwbXYDLuyIaXtb7Aqjq CDnZmLvh7tQBUCpYQ8Ky/e4KmoTQhiQKlnE8q0Iy1IZ5fVXlH1mzVjLSVcLA3y5PIVnu bpGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709658061; x=1710262861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Xm4doBABAihMhsOyWbtEbfo5woMRB8YQ6cNpUkDOhE=; b=erku1+CnhtTqNNZ3xoIsX48EMkiUdZ43cyxm7gslaFENYMYhdKyqMhw6VleNdOrkyx M1luKDqe8mq9laZ1kQBuL5J8btKub7K5vLo75fPGxJ9bKI5zldhaIfr20SoxnmjwUNTK 7i8oK0y4keRRIlv+32iTaD344c/1RUGZuYme79MerrREaPPatohX1BmMVCZNwGJoe87n 0khNOmc+K75m/B67LVYwnY5mFhqyjQHCVaSWiN9B6BCEbPsETW6U5ph65U3FI8Kf0JZt U32LBsp+RY1UcMAFsnhVpRKryIyqtl5d1o8W215JhsPvnDZA8U8FxbRYf+PHUn5J9yjt gt9Q== X-Gm-Message-State: AOJu0YwXyDLISu+EwQNo9CoTNmHKQZXs7Ikw9mvB4uchEfjqVALEf1ao 2/2PYWWFpkloF5+9nX359OJQEyG4lKGYDg7nP4G/JWFY0OA6/1xmAh5xp0X6 X-Google-Smtp-Source: AGHT+IGOk/rT+DvaLyl3XWYyha7HRrcGoCV8TdPjdsw+K+Pek9Sr7DPxs7teDmsFQb9QniuepdFElA== X-Received: by 2002:a17:903:41c8:b0:1dd:803:978e with SMTP id u8-20020a17090341c800b001dd0803978emr2716721ple.24.1709658059868; Tue, 05 Mar 2024 09:00:59 -0800 (PST) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id u11-20020a056a00098b00b006e629bd793esm56065pfg.108.2024.03.05.09.00.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Mar 2024 09:00:59 -0800 (PST) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 4E9167406AC; Tue, 5 Mar 2024 09:00:58 -0800 (PST) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com Subject: [PATCH 4/4] elf: Don't cache symbol nor relocation tables with mmap Date: Tue, 5 Mar 2024 09:00:57 -0800 Message-ID: <20240305170057.292800-5-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240305170057.292800-1-hjl.tools@gmail.com> References: <20240305170057.292800-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: During a "-j 8" LLVM 17 debug build on a machine with 32GB RAM and 16GB swap, ld was killed by kernel because of out of memory: [79437.949336] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-9.scope,task=ld,pid=797431,uid=1000 [79437.949349] Out of memory: Killed process 797431 (ld) total-vm:9219600kB, anon-rss:6558156kB, file-rss:1792kB, shmem-rss:0kB, UID:1000 pgtables:17552kB oom_score_adj:0 Don't cache symbol nor relocation tables if they are mapped in. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: stdio mmap improvement user 86.73 87.02 -0.3% system 9.55 9.21 3.6% total 100.40 97.66 0.7% maximum set(GB) 17.34 13.14 24% page faults 4047667 3042877 25% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.41 5.44 -0.5% system 0.80 0.76 5% total 6.25 6.26 -0.2% maximum set(MB) 1323 968 27% page faults 323451 236371 27% These improve the overall system performance for parallel build by reducing memory usage and page faults. * linker.c (_bfd_link_keep_memory): Return false if HAVE_MMAP is defined. --- bfd/linker.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/bfd/linker.c b/bfd/linker.c index 36cca9624c2..d6145b3272e 100644 --- a/bfd/linker.c +++ b/bfd/linker.c @@ -3561,8 +3561,12 @@ _bfd_nolink_bfd_define_start_stop (struct bfd_link_info *info ATTRIBUTE_UNUSED, and symbol tables of input files in memory. */ bool -_bfd_link_keep_memory (struct bfd_link_info * info) +_bfd_link_keep_memory (struct bfd_link_info * info ATTRIBUTE_UNUSED) { +#ifdef HAVE_MMAP + /* Don't cache symbol nor relocation tables if they are mapped in. */ + return false; +#else bfd *abfd; bfd_size_type size; @@ -3590,4 +3594,5 @@ _bfd_link_keep_memory (struct bfd_link_info * info) while (1); return true; +#endif } -- 2.44.0