From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by sourceware.org (Postfix) with ESMTPS id 141DC3857C66 for ; Thu, 10 Sep 2020 03:30:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 141DC3857C66 Received: by mail-pl1-x641.google.com with SMTP id d16so216155pll.13 for ; Wed, 09 Sep 2020 20:30:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=WDFV0hFdQxT6nvPM1daKBRaZ8IQtKBYjNFPiJhqalyE=; b=jpKFBGj7LBDpRNPxIKV/Ci1SYBTHZhSTTjf/xNXV8lUYbLkwyL512AYdGXPjAjKRMA HuwWWDLEGZqE+MUi2FR1sWzEX9VyWbhzNHiMAdNLdvR3tNGJJZLDyZIosutwuhtDap/d Fd+Qib2LWJZfPbPHZ7G3MiLeKdE5H1U4hc6nvYXoVRsFew0/IOvj6zFfQqnqGTtFrqle U96YpNxd19R3U6HEnz1WoNNgGroQyMLZ9JUtFf1JXoNxWICKQpDkf3znwFsjInBDK7v2 4u+zvLmX5ogpIlDX/S7kD8p7acR5hvhJStn/MWUiiRohPGbHh8KrVGGy+omWGFjfy2cA llyw== X-Gm-Message-State: AOAM533adonDzozZUe3xXJd+OG3wL3cbZhYZ5o29eDuljmxHTud+kt7f qUm6heLF3qOnN0I7ZmwB3z0= X-Google-Smtp-Source: ABdhPJzCIctFbN12QX49Rv7oQQwCJV3xeL5YsOJvDvl9j81M0k5vXA/4ZMOuh4mPWIMhA278V1ZqMg== X-Received: by 2002:a17:90a:e60d:: with SMTP id j13mr3460065pjy.61.1599708606082; Wed, 09 Sep 2020 20:30:06 -0700 (PDT) Received: from bubble.grove.modra.org ([2406:3400:51d:8cc0:113:eed0:d378:41c6]) by smtp.gmail.com with ESMTPSA id v69sm3466109pgd.61.2020.09.09.20.30.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Sep 2020 20:30:05 -0700 (PDT) Received: by bubble.grove.modra.org (Postfix, from userid 1000) id 4EF5086AEF; Thu, 10 Sep 2020 13:00:01 +0930 (ACST) Date: Thu, 10 Sep 2020 13:00:01 +0930 From: Alan Modra To: "H.J. Lu" Cc: binutils@sourceware.org Subject: Re: [PATCH] ld: Add more tests for --as-needed Message-ID: <20200910033001.GI15695@bubble.grove.modra.org> References: <20200909161640.335379-1-hjl.tools@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200909161640.335379-1-hjl.tools@gmail.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Sep 2020 03:30:08 -0000 On Wed, Sep 09, 2020 at 09:16:40AM -0700, H.J. Lu via Binutils wrote: > commit fb7331ae2b551247a33e8da5387ebc910c4a7308 > Author: Alan Modra > Date: Wed Sep 9 12:16:23 2020 +0930 > > lto-18 test > > added tests to show the reason to use IR symbols when deciding an > --as-needed library should be loaded. Exactly. The testcase compiles and links fine without LTO, and compiles and links with LTO if references from IR symbols are used to decide whether a library is needed. Reverting 1e3b96fd6cf0 results in FAIL: lto-18 (1) FAIL: lto-18 (2) You kept asking for a testcase in the thread where I committed 1e3b96fd6cf0, and now you have one. I think the testcase is valid, but like many testcases, is nasty. That testcase is not the only reason why I think we should be using IR symbol refs to decide that an as-needed library is needed. Here are my reasons: 1) --as-needed is supposed to load a shared library if there is a non-weak undefined symbol remaining from previously loaded objects, defined by the library. By previously loaded objects we mean those loaded on the command line or in scripts prior to the library, similar to the logic used by the linker to extract objects from archives. Archive searches must extract objects to satisfy IR references, because the object in the archive might be an LTO object. Since --as-needed was intended to be somewhat modelled on the way ld extracts objects from archives, it makes sense to do the same for --as-needed libraries. 2) Trying to only emit DT_NEEDED for --as-needed libraries based on the final LTO set of symbols is complicated, and has led to a number of bug reports in the past. Those bug reports have meant that we keep some symbol state from an as-needed library that isn't marked as needed at that point in linking, to feed to the LTO recompilation. While doing that for references generally won't break anything if the as-needed library is also found to be not needed after LTO, I worry that doing the same for symbol definitions will lead to breakage. If a symbol being defined might affect LTO output, but the LTO output not have a reference to the symbol in question, then that could result in an as-needed library not being marked as needed on the second pass through libraries after LTO. The result would be that ld tells LTO a symbol is defined but it ends up undefined. 3) Further patches complicating the linker like the one you posted at https://sourceware.org/pipermail/binutils/2020-September/113227.html are likely to be necessary, if we go back to IR symbols not marking as-needed libraries as needed. > But the requirement of the > --as-needed library doesn't come from the IR inputs since the IR symbol > reference of the --as-needed library is removed by LTO. The requirement > comes from another --as-needed library which is referenced by the LTO > output as shown by PR ld/26590. PR26590 merely shows --as-needed working as designed. > Not use IR symbols when deciding an > --as-needed library should be loaded exposes the same issue with input > --as-needed libraries as non-LTO inputs. The current ld behavior is > desirable since it sures that both lazy and non-lazy bindings work the > same way. > > Add more tests for --as-needed and add tmpdir/liblto-18b.so DT_NEEDED > to liblto-18c.so. No, you may not commit the lto.exp patch. Shared libraries are not required to have DT_NEEDED entries. Imagine for moment that liblto-18b.so is a library from author B that has some undefined functions that normally would be provided in the executable. Author B doesn't even know about liblto-18c.so, thus no DT_NEEDED for it. Author C has a similar liblto-18c.so library with some undefined functions normally provided in the executables. Author A writing the app wants to use both liblto-18b.so and liblto-18c.so and realizes that they satisfy each other's undefined symbols. So he doesn't need to write those functions in the executable. He gets everything working without LTO, then turns on LTO and it fails mysteriously. The other tests are fine to commit. -- Alan Modra Australia Development Lab, IBM