From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by sourceware.org (Postfix) with ESMTPS id 5475B385780D for ; Wed, 13 Mar 2024 15:52:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5475B385780D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5475B385780D Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::634 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710345157; cv=none; b=M4GfkigleHsHrbXarLykeSBlDomUVGJNcjTbqvA191lg7u2LUBuLPpKp9cFBuifMZkMCUgCQcoDdb6NERXZVBsfBUfLXYgePige+OJ9egI6AwaljI9kMP5M1K6D2e0UtSvbGQLB7nrMLZ3MIlRLfZO+3nKayC0ZllEraz98NBFk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710345157; c=relaxed/simple; bh=L8i/aMjC5QJZJydIoAdWjrJWELUUGO8L63dBeJ9haZM=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=JPoWv5UqsLQKpeWbbX1rg+2KVJ+9LqsRmdxygc99bMrFF/OQo0ED1odTMqHc9gaSXpt6oCsOw2xgo2LE6IJE7Oay4nbU4dfFs1LP7mP7RrvZ0xBMSTkzibheGxwnmTIy0BkEQGszw57MG6facTowi2mivxrX2v9wQwA80sQCkvs= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1dd8dd198d0so29565845ad.3 for ; Wed, 13 Mar 2024 08:52:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710345153; x=1710949953; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V9Y/bKXMbr2CQjnTshkQ5pGpcjzO3QVIZUHaU3ulY7U=; b=Hj5Aqaoc5KoMwUILXGiGcQS6w0qUm349Lb5cDGvkZhoO2IME9KVLM7W70bRpYGlpTO 8V/l5DNrgdty3lS1TslhUptHFv3ORRlRUxWF5u+xO4xzumoOG+Cry9eBeoUMaNCq3Ueq gGDcm+MT3354vmS9WkupjMpCv3V+YGKTYB8ODdbT+fEo0Cu/8vBnVcNtuevx8r+jqHbn Uw9qmzaYU8069nviDs+ooU5yvvZcWbLrUtEVYpheG3k2QW/o0523+3JYmYzWHl1W2vsa nxOvZE2pRdABRtW5l4GA5OdzAVJ3HNfS4BZ9f1Yiup0wqYHfny+G8tVYk+2IS/AKGQ+Y pq1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710345153; x=1710949953; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V9Y/bKXMbr2CQjnTshkQ5pGpcjzO3QVIZUHaU3ulY7U=; b=LoyKjBYAJIqUiuvyvxyhobvPoQr2vJCegyR/cnhYKsLVK0PLgJCUdrM/1bA7gdZ1f1 eDX1znuAFMvo9hznuWY4gLTw0iz2COkhJSX+2wobwCig6VhZfhH+Mvg1/OtxEw1jbWq1 Hyp7gzivaI1VHQzXw+tSLVg9Ea3+qb2ZcCFsqM+aexnGO5m6/C8VJ1jXmvpVlBWoyzxY O+nBI0kuG6LMiiRn7PNcWDsUCnfciBp+qiauanZzujVgUkfuzj6k63ohuvlofY0LtE2K Hlv1zhuFOTtaPWnTRhnR+vEM53J81gRlVu1gNAaKb4JVuI9D+iYBJb17+7pPGFjrIMdZ 64Uw== X-Gm-Message-State: AOJu0YzNfWFXHvIvM32xeiOli8bbBSooTMSIrOCd0tneGC798UkUzod8 oUjqMwJfzot6lhQnvvcCDQvrdDWpx2N/YzRzI6mH12wFK73OjGyW X-Google-Smtp-Source: AGHT+IEe8pAZdat10viWwwedKCdmfbzoPgZYXNjh9iQMLvbQqMIHnQGr1fWHnyux/Anr1sd9RzgxNA== X-Received: by 2002:a17:903:2282:b0:1dc:b691:85f4 with SMTP id b2-20020a170903228200b001dcb69185f4mr16337204plh.16.1710345153213; Wed, 13 Mar 2024 08:52:33 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id y18-20020a170902b49200b001dd99fe365dsm6420758plr.42.2024.03.13.08.52.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Mar 2024 08:52:32 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 1871274071B; Wed, 13 Mar 2024 08:52:28 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v8 5/6] elf: Always keep symbol table and relocation info for eh_frame Date: Wed, 13 Mar 2024 08:52:26 -0700 Message-ID: <20240313155227.513873-6-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240313155227.513873-1-hjl.tools@gmail.com> References: <20240313155227.513873-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: When --no-keep-memory is used, the symbol table and relocation info for eh_frame are freed after they are retrieved for each text section in the input object. If an input object has many text sections, the same data is retrieved and freed many times which can take a very long time. Update _bfd_elf_gc_mark to keep the symbol table and relocation info for eh_frame to avoid it. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: before after improvement user 86.31 86.44 -0.2% system 8.77 8.63 1.6% total 95.58 96.81 -1.3% maximum set(GB) 13.1 13.1 0% page faults 3024752 3028699 -1.3% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.49 5.46 -0.5% system 0.73 0.73 0% total 6.26 6.25 0.3% maximum set(MB) 964 964 0% page faults 235173 235796 -0.3% The memory usage impact is minimum and the link time of the Rust binary in https://sourceware.org/bugzilla/show_bug.cgi?id=31466 is reduced from 500+ seconds to 1.44 seconds, a 300x speedup. PR ld/31466 * elflink.c (init_reloc_cookie): Add a bool argument, keep_memory, for keeping memory. Always keep memory if keep_memory is true. (init_reloc_cookie_rels): Likewise (init_reloc_cookie_for_section): Add a bool argument for keeping memory and pass it to init_reloc_cookie and init_reloc_cookie_rels. (_bfd_elf_gc_mark_reloc): Pass false to _bfd_elf_gc_mark. (_bfd_elf_gc_mark): Pass true to init_reloc_cookie_for_section for the eh_frame section. Pass false to init_reloc_cookie_for_section for other sections. (_bfd_elf_gc_mark_extra_sections): Add Add a bool argument for keeping memory and pass it to _bfd_elf_gc_mark. (bfd_elf_parse_eh_frame_entries): Pass false to init_reloc_cookie and init_reloc_cookie_rels. (bfd_elf_gc_sections): Pass false to init_reloc_cookie_for_section and _bfd_elf_gc_mark. (bfd_elf_discard_info): Pass false to init_reloc_cookie_for_section and init_reloc_cookie. --- bfd/elflink.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/bfd/elflink.c b/bfd/elflink.c index 9321eeda684..7cb9269f101 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -13590,7 +13590,8 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) static bool init_reloc_cookie (struct elf_reloc_cookie *cookie, - struct bfd_link_info *info, bfd *abfd) + struct bfd_link_info *info, bfd *abfd, + bool keep_memory) { Elf_Internal_Shdr *symtab_hdr; const struct elf_backend_data *bed; @@ -13628,7 +13629,7 @@ init_reloc_cookie (struct elf_reloc_cookie *cookie, info->callbacks->einfo (_("%P%X: can not read symbols: %E\n")); return false; } - if (_bfd_elf_link_keep_memory (info) ) + if (keep_memory || _bfd_elf_link_keep_memory (info)) { symtab_hdr->contents = (bfd_byte *) cookie->locsyms; info->cache_size += (cookie->locsymcount @@ -13656,7 +13657,7 @@ fini_reloc_cookie (struct elf_reloc_cookie *cookie, bfd *abfd) static bool init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, bfd *abfd, - asection *sec) + asection *sec, bool keep_memory) { if (sec->reloc_count == 0) { @@ -13667,7 +13668,7 @@ init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, { cookie->rels = _bfd_elf_link_info_read_relocs (abfd, info, sec, NULL, NULL, - _bfd_elf_link_keep_memory (info)); + keep_memory || _bfd_elf_link_keep_memory (info)); if (cookie->rels == NULL) return false; cookie->rel = cookie->rels; @@ -13693,11 +13694,12 @@ fini_reloc_cookie_rels (struct elf_reloc_cookie *cookie, static bool init_reloc_cookie_for_section (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, - asection *sec) + asection *sec, bool keep_memory) { - if (!init_reloc_cookie (cookie, info, sec->owner)) + if (!init_reloc_cookie (cookie, info, sec->owner, keep_memory)) goto error1; - if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec)) + if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec, + keep_memory)) goto error2; return true; @@ -13908,7 +13910,7 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, sec)) + if (!init_reloc_cookie_for_section (&cookie, info, sec, false)) ret = false; else { @@ -13926,7 +13928,14 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, eh_frame)) + /* NB: When --no-keep-memory is used, the symbol table and + relocation info for eh_frame are freed after they are retrieved + for each text section in the input object. If an input object + has many text sections, the same data is retrieved and freed + many times which can take a very long time. Always keep the + symbol table and relocation info for eh_frame to avoid it. */ + if (!init_reloc_cookie_for_section (&cookie, info, eh_frame, + true)) ret = false; else { @@ -14366,13 +14375,14 @@ bfd_elf_parse_eh_frame_entries (bfd *abfd ATTRIBUTE_UNUSED, if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; - if (!init_reloc_cookie (&cookie, info, ibfd)) + if (!init_reloc_cookie (&cookie, info, ibfd, false)) return false; for (sec = ibfd->sections; sec; sec = sec->next) { if (startswith (bfd_section_name (sec), ".eh_frame_entry") - && init_reloc_cookie_rels (&cookie, info, ibfd, sec)) + && init_reloc_cookie_rels (&cookie, info, ibfd, sec, + false)) { _bfd_elf_parse_eh_frame_entry (info, sec, &cookie); fini_reloc_cookie_rels (&cookie, sec); @@ -14417,7 +14427,8 @@ bfd_elf_gc_sections (bfd *abfd, struct bfd_link_info *info) if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; sec = bfd_get_section_by_name (sub, ".eh_frame"); - while (sec && init_reloc_cookie_for_section (&cookie, info, sec)) + while (sec && init_reloc_cookie_for_section (&cookie, info, sec, + false)) { _bfd_elf_parse_eh_frame (sub, info, sec, &cookie); if (elf_section_data (sec)->sec_info @@ -14929,7 +14940,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_discard_section_stabs (abfd, i, @@ -14960,7 +14971,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; _bfd_elf_parse_eh_frame (abfd, info, i, &cookie); @@ -15025,7 +15036,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_elf_parse_sframe (abfd, info, i, &cookie)) @@ -15061,7 +15072,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bed->elf_backend_discard_info != NULL) { - if (!init_reloc_cookie (&cookie, info, abfd)) + if (!init_reloc_cookie (&cookie, info, abfd, false)) return -1; if ((*bed->elf_backend_discard_info) (abfd, &cookie, info)) -- 2.44.0