From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 116221 invoked by alias); 19 Apr 2016 21:03:26 -0000 Mailing-List: contact binutils-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: binutils-owner@sourceware.org Received: (qmail 116191 invoked by uid 89); 19 Apr 2016 21:03:25 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.2 required=5.0 tests=AWL,BAYES_50,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=ham version=3.3.2 spammy=invested, frustrating, establishment, calm X-HELO: mail-qg0-f53.google.com Received: from mail-qg0-f53.google.com (HELO mail-qg0-f53.google.com) (209.85.192.53) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Tue, 19 Apr 2016 21:03:15 +0000 Received: by mail-qg0-f53.google.com with SMTP id v14so17538548qge.0 for ; Tue, 19 Apr 2016 14:03:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=4c9Z7gqp/2WelF243aYGzQX3YFEmVmZGeIZHWNF+wmM=; b=QWeLQwyS3PmUIcwtz7XJ5EPbb5zuI6aRRoYr+j2Ia6UCHgTxGXEx09vFJ+qOtF0Tif ilCJC2k6er0WoggzqO+Z5iVDw/tp8fuJlTi88FwANwrXdChFwJ9QS3kiR1KNiiipzIY1 JlHoHL9RoT/NoNhpFVJZ33y2aPFt3JLaKkRNZA+qOnRzJBDBcf08QY56xdA1qiqYblgy DqSfRHpj0VD1O8snrH7QbUmvzn7fbpMEK8qBTl04c6LWiZ0PZ9KLYvAwny1ysMcpmWbZ Bv4AOfHAIPh1hKLVHCpAzgSdA8c86RqNPtuYXw6Jynt9ohGV/in7nW7MKvjIRDpASOud Caog== X-Gm-Message-State: AOPr4FVb3whDcUvG6Id6VMpNtCncm7MmRlLOEM8h9Yno9uTfvx2gEGWoWLmhj8Yz5r3lxsY6ME3RuKBHRUq7Cg== MIME-Version: 1.0 X-Received: by 10.140.82.69 with SMTP id g63mr6419699qgd.106.1461099793114; Tue, 19 Apr 2016 14:03:13 -0700 (PDT) Received: by 10.55.60.3 with HTTP; Tue, 19 Apr 2016 14:03:12 -0700 (PDT) In-Reply-To: <20160419201711.GS21636@brightrain.aerifal.cx> References: <20160330143421.GM15812@bubble.grove.modra.org> <571161D0.10601@redhat.com> <20160418144911.GG15088@bubble.grove.modra.org> <20160419050805.GI15088@bubble.grove.modra.org> <20160419201711.GS21636@brightrain.aerifal.cx> Date: Tue, 19 Apr 2016 21:03:00 -0000 Message-ID: Subject: Re: Preventing preemption of 'protected' symbols in GNU ld 2.26 [aka should we revert the fix for 65248] From: Cary Coutant To: Rich Felker Cc: "H.J. Lu" , Michael Matz , Richard Biener , Alan Modra , Jeff Law , Joe Groff , Binutils , GCC , Jim Dehnert Content-Type: text/plain; charset=UTF-8 X-IsSubscribed: yes X-SW-Source: 2016-04/txt/msg00317.txt.bz2 > As one of the strong advocates for the fix that was made to make > protected visibility work correctly with data symbols, I'd like to > explain why it was the right decision and why it matters. This whole > process is really frustrating to me -- having invested a lot of effort > into getting something important fixed, only to have people come > trying to break it again -- but I'm going to try to be calm and not to > snap at anybody. Ironically, you've just described my feelings almost exactly, only I come here finding that someone already broke it, and I'm trying to get it fixed again. With all due respect, I think you're misinterpreting what the visibility feature was intended for, and you're projecting your own needs onto everyone else that uses this feature. I can state with first-hand knowledge that the intent was so that compilers could optimize access to the protected data symbols based on an assumption that they are "relatively nearby". Here's a quote from the gABI proposal (the latest revision that I could find, dated April 16, 1999), which was submitted by Jim Dehnert, then at SGI: "Optimization Note: "The visibility semantics of these attributes allow various optimizations. While care must be taken to maintain position-independence and proper GOT usage for references to and definitions of symbols which might be preempted by or referenced from other components, these restrictions all allow references from the same component to make stricter assumptions about the definitions. References to protected symbols (and hence to hidden or internal symbols) may be optimized by using absolute or PC-relative addresses in executable files or by assuming addresses to be relatively nearby. Internal functions (as defined in the MIPS ABI) do not normally require gp establishment code even in psABIs requiring callee establishment/restore of gp, because they will always be entered from the same component with the correct gp already in place from the caller." Unfortunately, this optimization note didn't make it into the gABI, probably because the editor felt it was unnecessary, and because it contained some MIPS-specific details. Nevertheless, it clearly shows the intent. > From a programming standpoint, the semantics of protected visibility > need to be free of arch-specific implementation details. Otherwise > programmers can't use it without hard-coding arch-specific details, > which for practical purposes, means good software can't use it at all. It's unfortunate that copy relocations intrude on your stated goal, but I'd prefer to fix that problem without breaking (and I do mean "break") the original intent behind protected visibility. > My original motivation for wanting protected visibility to "just work" > was to be able to use: > > #pragma GCC visibility push(protected) > > around the inclusion of a library's public headers when including them > from the implementation files, This is far from being the only usage > case, and I'll expand on more important usage cases below, but it is > an important one because it allows you to eliminate all GOT/PLT cost > of intra-library function calls without any fine-grained maintenance > of which declarations to apply visibility too (and without any > GNUC-specific clutter in the header files themselves). > > I understand that some people want protected visibility to avoid the > GOT for data symbols too for the sake of performance, but for my usage > case, the fact that the semantics were wrong for data symbols meant > that my configure check for "does protected visibility work" would > return "no", and the whole optimization would get turned off. Yes, some people do want protected visibility to avoid the GOT for data symbols, and it makes a significant difference in many cases. For those people, the changes I'm objecting to cause a performance regression. > Anyway, let's move past optimization, because it's a distraction. I disagree. Optimization isn't a distraction -- it's the whole motivation for the feature. > After all, with the old (broken) behavior of protected data, one > _could_ work around the above problem and still get the performance > benefits for functions without breaking data by explicitly declaring > all data with default visibility. In fact, this is how I solve the > problem in musl libc, where there are only a small number of data > symbols that should be externally accessible, and maintaining a list > of them is managable: > > http://git.musl-libc.org/cgit/musl/tree/src/internal/vis.h?id=d1b29c2a54588401494c1a3ac7103c1e91c61fa1 > > This is done for the sake of compatibility with a wide range of > toolchains including ones with the old/broken behavior for protected > data. s/broken/correct/ :-) Given that, now you can have efficient access to the symbols that aren't externally accessible, and the symbols that are accessible are marked correctly with default visibility, right? So what's the problem? Why would you want to give up the efficient direct access to the symbols that can remain protected? It looks to me like you have a solution, and it's compatible with the intent behind the feature. > The actual documented purpose of protected visibility is to prevent > other definitions of a symbol from taking precedence over the one in > the library itself. Um, I'd say that's the documented *meaning* -- the *purpose* is to enable compiler optimizations. > For example, suppose you have the following > situation: mainapp depends on libA which defines symbol foo with > normal visibility, and libA depends on libB, which also defines foo, > but intentionally with protected visibility so that libB always uses > its own definition, not the one from libA. There is no reasonable way > to obtain the desired semantics here without the current/correct > behavior for protected data. Any other approaches I'm aware of would > either allow libB to bind to the wrong definition of foo, or would > prevent another main app which links libB (but not libA) from being > able to use the symbol foo from libB. If libA and libB both export symbol foo (protected or not), you're playing games with the linker. This is not good programming practice. It was never our intent to enable developers to play linker games. At any rate, I fail to see how you get unexpected semantics either way. With the old (correct to me) behavior, the compiler and linker can bind the references from within libB to its own foo, and the dynamic loader never has to get involved. With the new behavior, the compiler and linker must leave the GOT-indirect accesses to foo, and rely on the dynamic loader to properly resolve the reference (and introducing quite a bit of additional complexity in the process). In addition, libB probably ought to be making foo hidden, not protected. > On the other hand, there are plenty of other ways to get the > old/broken behavior if desired. The easiest is to simply use hidden > visibility when you don't want the symbol to be accessible outside the > library. If you _do_ want it to be visible to and usable by other > shared libraries, just not main-programs, this can be achieved using a > hidden alias for a default-visibility symbol: > > int foo; > extern int fast_foo __attribute__((__alias__("foo"), > __visibility__("hidden"))); "Just not main-programs." If you're willing to ignore main programs, the copy relocations don't matter either, and what was the issue again? A hidden alias makes the problem even worse, because now the linker can't even diagnose the error when it makes a copy relocation -- you'll simply get one set of code using the copy, and another set of code using the original (equally broken with old and new behavior). > I expressed that here explicitly for the sake of clarity but of course > in practice people use things like the glibc macro-maze to do this > kind of binding to hidden aliases. But why go through that, when protected visibility does (used to do) exactly the right thing? -cary