From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by sourceware.org (Postfix) with ESMTPS id 28D153858402 for ; Sun, 17 Mar 2024 12:19:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 28D153858402 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 28D153858402 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::633 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; cv=none; b=xtC43fjeX+VuOLHdGeb1zUYK9qYQAHM9RdgAYbAQit41jkhtCq3MUeW32tAtzSjcXiZ/XizwD6ATTPcknHhWS/LMsLZ/kYvLBWGBJ2FbLH+RGalRfFYg6fTFrUTqL23yQ22ciGt328A0s+hoLq+3axAg1Zf0trH1YF1hUbd22eI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1710677961; c=relaxed/simple; bh=MvL8soQye2pxrdrAiBempMnRCuSeLTuxLLbzefAxfb8=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=RGBk1u+G++mlZg2Ak4GMqGRu8sUR7WYiDU8wSnZgfJJaVMN0HVH4JyWRrV8dBBwEiF3B2giTYu7ULr3ioIab6LQiM85lmVEuyhBqBRZjAO4ebSoNOXahsuFuhlLjguVAh88Za8ayw6CGinZMbLzlnz0pBPZj/tETJLVaq91YXCU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1dddad37712so33685885ad.3 for ; Sun, 17 Mar 2024 05:19:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710677958; x=1711282758; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bZNkny8eb+NNkPm3I+3YtoQHhrQiN11SMwzwh4QpAew=; b=IUREEbH4a1uxRKqNbgPfmKUbJ5Yejisk38qRQ7J3IdWPH4gmS5liVfQShvy6jMwKHU 9Dra1OnjcDmUruBsuJ/0L38FCh8U7VhoSoV//7oJwfS8GFXyRFH61OWX8l6z74W4pO3n AsqZQRG7ukRlWlL1ovoEOoLoV3CuB2BihF8ROLeAl78tdUeAvWb5KLQG9JFMhSRXPbnA CzsVI2Rwk/McHkwzuUmx+vcokkNXbIJkzux0MLlr2bXyOhpjqikYbgMHaufHbsxSZt1A Xy+Uzr4IfyTJR09BmSHBkPe1zDifEMxBxeWizX856zoF1vOk15nurzozexM4l/KK/k6e PG5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710677958; x=1711282758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bZNkny8eb+NNkPm3I+3YtoQHhrQiN11SMwzwh4QpAew=; b=Xc5bLScJM9fdb+Z4wQTLGbhYIS/suAwaw4oC+1SMgaO4r9G5zzzj33R0OV0jGWAzSU O9uDXZ1tTksIz8FrGLl1Au3sYRWbm4LjokwVgiN/YRALCuvlz6lmVKgxAO9Y/g3NT4B5 Elutog5L+GTsijcO/kRU6dltEeDcioomqkOpKEGbhigFcKGLTp5XKa/3mg1kpDCW3wek K1VxJKfAey3fYA5br8ksrk3mAtURvARPbDGnLrGID6uhuFEza8BkV53VStYpEuxYEYFO VH5yi+3B5jOUBa9+qo9+Hs9PQORGQMdJjDBA7UtpRudLZsQc57pwmiFp1LZxtpRj52r2 4TaA== X-Gm-Message-State: AOJu0YzWQKBMkTxeNl1o5R3eOaXIQiv2lU5/wEuSE7GWID1rAykEf9CO PenbHqDohpcPaQbkkuXvhWKL4SyKKU+DZ8/VT7fhTTcMHSHfa7uT X-Google-Smtp-Source: AGHT+IGgmytEOiFpegYRym0ugBrVM9W17PmWB1jChckrLck/3+iuvzmOyhsMjNIjJ55dkoT253semw== X-Received: by 2002:a17:902:ba8c:b0:1dd:7485:b4c9 with SMTP id k12-20020a170902ba8c00b001dd7485b4c9mr9908136pls.22.1710677958066; Sun, 17 Mar 2024 05:19:18 -0700 (PDT) Received: from gnu-cfl-3.localdomain ([172.58.89.72]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b001ddb505d50asm7186521pll.244.2024.03.17.05.19.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Mar 2024 05:19:16 -0700 (PDT) Received: from gnu-cfl-3.. (localhost [IPv6:::1]) by gnu-cfl-3.localdomain (Postfix) with ESMTP id 1963C740723; Sun, 17 Mar 2024 05:19:13 -0700 (PDT) From: "H.J. Lu" To: binutils@sourceware.org Cc: goldstein.w.n@gmail.com, sam@gentoo.org, amodra@gmail.com Subject: [PATCH v12 5/6] elf: Always keep symbol table and relocation info for eh_frame Date: Sun, 17 Mar 2024 05:19:11 -0700 Message-ID: <20240317121912.799372-6-hjl.tools@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240317121912.799372-1-hjl.tools@gmail.com> References: <20240317121912.799372-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-3019.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,GIT_PATCH_0,RCVD_IN_ABUSEAT,RCVD_IN_DNSWL_NONE,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: When --no-keep-memory is used, the symbol table and relocation info for eh_frame are freed after they are retrieved for each text section in the input object. If an input object has many text sections, the same data is retrieved and freed many times which can take a very long time. Update _bfd_elf_gc_mark to keep the symbol table and relocation info for eh_frame to avoid it. Data to link the 3.5GB clang executable in LLVM 17 debug build on Linux/x86-64 with 32GB RAM is: before after improvement user 86.31 86.44 -0.2% system 8.77 8.63 1.6% total 95.58 96.81 -1.3% maximum set(GB) 13.1 13.1 0% page faults 3024752 3028699 -1.3% and data to link the 275M cc1plus executable in GCC 14 stage 1 build is: user 5.49 5.46 -0.5% system 0.73 0.73 0% total 6.26 6.25 0.3% maximum set(MB) 964 964 0% page faults 235173 235796 -0.3% The memory usage impact is minimum and the link time of the Rust binary in https://sourceware.org/bugzilla/show_bug.cgi?id=31466 is reduced from 500+ seconds to 1.44 seconds, a 300x speedup. PR ld/31466 * elflink.c (init_reloc_cookie): Add a bool argument, keep_memory, for keeping memory. Always keep memory if keep_memory is true. (init_reloc_cookie_rels): Likewise (init_reloc_cookie_for_section): Add a bool argument for keeping memory and pass it to init_reloc_cookie and init_reloc_cookie_rels. (_bfd_elf_gc_mark_reloc): Pass false to _bfd_elf_gc_mark. (_bfd_elf_gc_mark): Pass true to init_reloc_cookie_for_section for the eh_frame section. Pass false to init_reloc_cookie_for_section for other sections. (_bfd_elf_gc_mark_extra_sections): Add Add a bool argument for keeping memory and pass it to _bfd_elf_gc_mark. (bfd_elf_parse_eh_frame_entries): Pass false to init_reloc_cookie and init_reloc_cookie_rels. (bfd_elf_gc_sections): Pass false to init_reloc_cookie_for_section and _bfd_elf_gc_mark. (bfd_elf_discard_info): Pass false to init_reloc_cookie_for_section and init_reloc_cookie. --- bfd/elflink.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/bfd/elflink.c b/bfd/elflink.c index dcbe6e1a18c..601d15e9cef 100644 --- a/bfd/elflink.c +++ b/bfd/elflink.c @@ -13621,7 +13621,8 @@ bfd_elf_final_link (bfd *abfd, struct bfd_link_info *info) static bool init_reloc_cookie (struct elf_reloc_cookie *cookie, - struct bfd_link_info *info, bfd *abfd) + struct bfd_link_info *info, bfd *abfd, + bool keep_memory) { Elf_Internal_Shdr *symtab_hdr; const struct elf_backend_data *bed; @@ -13659,7 +13660,7 @@ init_reloc_cookie (struct elf_reloc_cookie *cookie, info->callbacks->einfo (_("%P%X: can not read symbols: %E\n")); return false; } - if (_bfd_elf_link_keep_memory (info) ) + if (keep_memory || _bfd_elf_link_keep_memory (info)) { symtab_hdr->contents = (bfd_byte *) cookie->locsyms; info->cache_size += (cookie->locsymcount @@ -13687,7 +13688,7 @@ fini_reloc_cookie (struct elf_reloc_cookie *cookie, bfd *abfd) static bool init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, bfd *abfd, - asection *sec) + asection *sec, bool keep_memory) { if (sec->reloc_count == 0) { @@ -13698,7 +13699,7 @@ init_reloc_cookie_rels (struct elf_reloc_cookie *cookie, { cookie->rels = _bfd_elf_link_info_read_relocs (abfd, info, sec, NULL, NULL, - _bfd_elf_link_keep_memory (info)); + keep_memory || _bfd_elf_link_keep_memory (info)); if (cookie->rels == NULL) return false; cookie->rel = cookie->rels; @@ -13724,11 +13725,12 @@ fini_reloc_cookie_rels (struct elf_reloc_cookie *cookie, static bool init_reloc_cookie_for_section (struct elf_reloc_cookie *cookie, struct bfd_link_info *info, - asection *sec) + asection *sec, bool keep_memory) { - if (!init_reloc_cookie (cookie, info, sec->owner)) + if (!init_reloc_cookie (cookie, info, sec->owner, keep_memory)) goto error1; - if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec)) + if (!init_reloc_cookie_rels (cookie, info, sec->owner, sec, + keep_memory)) goto error2; return true; @@ -13939,7 +13941,7 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, sec)) + if (!init_reloc_cookie_for_section (&cookie, info, sec, false)) ret = false; else { @@ -13957,7 +13959,14 @@ _bfd_elf_gc_mark (struct bfd_link_info *info, { struct elf_reloc_cookie cookie; - if (!init_reloc_cookie_for_section (&cookie, info, eh_frame)) + /* NB: When --no-keep-memory is used, the symbol table and + relocation info for eh_frame are freed after they are retrieved + for each text section in the input object. If an input object + has many text sections, the same data is retrieved and freed + many times which can take a very long time. Always keep the + symbol table and relocation info for eh_frame to avoid it. */ + if (!init_reloc_cookie_for_section (&cookie, info, eh_frame, + true)) ret = false; else { @@ -14397,13 +14406,14 @@ bfd_elf_parse_eh_frame_entries (bfd *abfd ATTRIBUTE_UNUSED, if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; - if (!init_reloc_cookie (&cookie, info, ibfd)) + if (!init_reloc_cookie (&cookie, info, ibfd, false)) return false; for (sec = ibfd->sections; sec; sec = sec->next) { if (startswith (bfd_section_name (sec), ".eh_frame_entry") - && init_reloc_cookie_rels (&cookie, info, ibfd, sec)) + && init_reloc_cookie_rels (&cookie, info, ibfd, sec, + false)) { _bfd_elf_parse_eh_frame_entry (info, sec, &cookie); fini_reloc_cookie_rels (&cookie, sec); @@ -14448,7 +14458,8 @@ bfd_elf_gc_sections (bfd *abfd, struct bfd_link_info *info) if (sec == NULL || sec->sec_info_type == SEC_INFO_TYPE_JUST_SYMS) continue; sec = bfd_get_section_by_name (sub, ".eh_frame"); - while (sec && init_reloc_cookie_for_section (&cookie, info, sec)) + while (sec && init_reloc_cookie_for_section (&cookie, info, sec, + false)) { _bfd_elf_parse_eh_frame (sub, info, sec, &cookie); if (elf_section_data (sec)->sec_info @@ -14960,7 +14971,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_discard_section_stabs (abfd, i, @@ -14991,7 +15002,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; _bfd_elf_parse_eh_frame (abfd, info, i, &cookie); @@ -15056,7 +15067,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bfd_get_flavour (abfd) != bfd_target_elf_flavour) continue; - if (!init_reloc_cookie_for_section (&cookie, info, i)) + if (!init_reloc_cookie_for_section (&cookie, info, i, false)) return -1; if (_bfd_elf_parse_sframe (abfd, info, i, &cookie)) @@ -15092,7 +15103,7 @@ bfd_elf_discard_info (bfd *output_bfd, struct bfd_link_info *info) if (bed->elf_backend_discard_info != NULL) { - if (!init_reloc_cookie (&cookie, info, abfd)) + if (!init_reloc_cookie (&cookie, info, abfd, false)) return -1; if ((*bed->elf_backend_discard_info) (abfd, &cookie, info)) -- 2.44.0