From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x144.google.com (mail-il1-x144.google.com [IPv6:2607:f8b0:4864:20::144]) by sourceware.org (Postfix) with ESMTPS id 7D9CC3861838 for ; Tue, 8 Sep 2020 12:43:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 7D9CC3861838 Received: by mail-il1-x144.google.com with SMTP id h11so15177549ilj.11 for ; Tue, 08 Sep 2020 05:43:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GvNF7Cut5uS9CpfGaqxcjdt1AkpRRdOtdb3DANGhs3o=; b=GF/NTEy4Wd6qO/dcjSJarALraOWxr5TfZtTlokzr5Ok6aAB8pXrs8BYAn052GgG+Fh tQHAv6C9BbzZhnfI+VMdfCVn65XqHF/LQZtH/E5xcAAP2I7lGOdSqpfnrynFXtTitN8s jGNKQ6el1zOzd/HpcyHgx0t1T8r95l1jP2NMZjYsRukf0Gs3wpfkb2eima/4oKR8NncC 2S5BjLz/jhyxTxmkIcAUkeYBzMwObATUqMBODJCOht6KSaghkSiXPRr0wc6Xuw8eZNzz gOzDSfKHi838oifH1KSPnZUinB3JFVgxK5g1wlqTwdh7czPkPdoPWfAExfua3zj88WFv LbNg== X-Gm-Message-State: AOAM530pzKjLrLdAdwcXD6kmGEudZ26CCbXP8MBfqBJTQfSRyDlSPINr Ig+La5FyZ170Ba71NMGCu1nWlCWdoBfCOrL9L7p/RzAEdHM= X-Google-Smtp-Source: ABdhPJyCqGMviSRL+djI1cHEZups7CSIkFfOOfoeRaZszcMYvjCIXNUNuBx8YqrO+ELPV7BmgDDc6QCw/3QT21paPu8= X-Received: by 2002:a05:6e02:10ce:: with SMTP id s14mr24561576ilj.13.1599569024747; Tue, 08 Sep 2020 05:43:44 -0700 (PDT) MIME-Version: 1.0 References: <20200902130522.GI15695@bubble.grove.modra.org> <20200902142930.GJ15695@bubble.grove.modra.org> <20200903013137.GM15695@bubble.grove.modra.org> <20200903060752.GO15695@bubble.grove.modra.org> <20200904072528.GS15695@bubble.grove.modra.org> <20200908054251.GX15695@bubble.grove.modra.org> In-Reply-To: <20200908054251.GX15695@bubble.grove.modra.org> From: "H.J. Lu" Date: Tue, 8 Sep 2020 05:43:08 -0700 Message-ID: Subject: Re: [PATCH] elf: Don't load archive element after dynamic definition To: Alan Modra Cc: =?UTF-8?Q?Martin_Li=C5=A1ka?= , Binutils Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2020 12:43:47 -0000 On Mon, Sep 7, 2020 at 10:42 PM Alan Modra wrote: > > On Fri, Sep 04, 2020 at 04:06:36AM -0700, H.J. Lu wrote: > > On Fri, Sep 4, 2020 at 12:25 AM Alan Modra wrote: > > > > > > On Thu, Sep 03, 2020 at 04:34:23AM -0700, H.J. Lu wrote: > > > > On Wed, Sep 2, 2020 at 11:07 PM Alan Modra wrote: > > > > > > > > > > On Wed, Sep 02, 2020 at 07:16:14PM -0700, H.J. Lu wrote: > > > > > > On Wed, Sep 2, 2020 at 6:31 PM Alan Modra wrote: > > > > > > > > > > > > > > On Wed, Sep 02, 2020 at 07:35:58AM -0700, H.J. Lu wrote: > > > > > > > > On Wed, Sep 2, 2020 at 7:29 AM Alan Modra wrote: > > > > > > > > > > > > > > > > > > On Wed, Sep 02, 2020 at 06:22:08AM -0700, H.J. Lu wrote: > > > > > > > > > > > It's reasonably obvious that we need to load archive elements when > > > > > > > > > > > they define IR referenced symbols, because the archive element might > > > > > > > > > > > be an LTO object. What's not so obvious is whether loading of shared > > > > > > > > > > > libraries should follow the same rule. I think they should, in order > > > > > > > > > > > for LTO to get symbol resolution correct in corner cases. Basically > > > > > > > > > > > LTO needs to know what shared libraries are loaded before > > > > > > > > > > > recompilation. See commit a896df97b952 log comments. > > > > > > > > > > > > > > > > > > > > There is dynamic_def for this purpose. > > > > > > > > > > > > > > > > > > Your patch doesn't make changes to ld/plugin.c to inform LTO of the > > > > > > > > > availability of these symbols. And if you did, then how does the > > > > > > > > > linker work out whether or not the LTO recompilation depended on those > > > > > > > > > symbols? If it did change LTO recompilation then you had better > > > > > > > > > ensure the library really is loaded. By the time you work all of that > > > > > > > > > out, if it is even possible, your patch will likely be very > > > > > > > > > complicated indeed. > > > > > > > > > > > > > > > > A testcase? > > > > > > > > > > > > > > What don't you understand from my emails in this thread? I suggest > > > > > > > you look at what happened with the testcase in PR26314, in regard to > > > > > > > my comment > > > > > > > The lto recompilation didn't see symbol references from libbfd.so and > > > > > > > variables like _xexit_cleanup are made local in the recompiled > > > > > > > objects. Oops, two copies of them. > > > > > > > > > > > > A testcase? > > > > > > > > > > You kindly provided it yourself. > > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=26314#c4 > > > > > > > > > > It takes only a small amount of digging to see the _xexit_cleanup > > > > > problem. > > > > > > > > This particular problem came from: > > > > > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96385 > > > > > > > > where GCC generated incorrect output and I do have a mitigation > > > > patch. > > > > > > Yes, you gave that as a testcase for unwanted dynamic symbols. I > > > suspected there was more than just that going on, so didn't approve > > > the entirety of your patch. While my initial guess at a ranlib issue > > > was wrong, on spending time properly analysing the testcase I found > > > that we had a problem with symbol info we hand off to LTO. > > > Specifically, we need to tell LTO about symbols in all shared > > > libraries loaded. That means we can't load extra shared libraries > > > after LTO recompilation, at least, not those that affect the set of > > > symbols that LTO cares about, the IR symbols. > > > > > > Committed. > > > > > > bfd/ > > > PR 15146 > > > PR 26314 > > > PR 26530 > > > * elflink.c (elf_link_add_object_symbols): Do set def_regular > > > and ref_regular for IR symbols. Don't clear dynsym, allowing > > > IR symbols to load --as-needed shared libraries, but prevent > > > IR symbols from becoming dynamic. > > > ld/ > > > * testsuite/ld-plugin/lto.exp: Don't run pr15146 tests. > > > * testsuite/ld-plugin/pr15146.d: Delete. > > > * testsuite/ld-plugin/pr15146a.c: Delete. > > > * testsuite/ld-plugin/pr15146b.c: Delete. > > > * testsuite/ld-plugin/pr15146c.c: Delete. > > > * testsuite/ld-plugin/pr15146d.c: Delete. > > > > Are we loading any non-IR objects after LTO symbol resolution? > > If yes, why is shared object disallowed? > > > > You make changes for PR without any testcases added. How do > > you know PR is really fixed? How do you make sure that it stays fixed? > > It's easy to make a testcase showing that loading an archive after an > --as-needed shared library is wrong, like the one you posted at > https://sourceware.org/pipermail/binutils/2020-August/112996.html > where functions in the archive differ from those in the shared > library. That's OK to commit, BTW. I would have liked something > closer to the situation in pr26314, where the archive functions are > identical to those in the shared library. So far I've only managed > that by cheating using protected visibility. > > I also spent quite a lot of time looking for a testcase that showed a > real problem with duplicated static variables like that of > _xexit_cleanup I saw, and failed. Thing is, a duplicated static isn't > a problem if it is never used. I suspect that is the case with > libbfd.so and libiberty.a compiled with -flto. So I don't have a > small testcase to justify removing the special treatment of IR symbol > references with respect to as-needed libraries, but I still think that > is the right thing to do. > I have an updated patch: https://sourceware.org/pipermail/binutils/2020-September/113227.html which changes the IR resolution from LDPR_UNDEF to LDPR_RESOLVED_DYN if the symbol is defined in a DT_NEEDED shared object in the first pass. -- H.J.