* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
2016-01-01 0:00 ` Maciej W. Rozycki
@ 2016-01-01 0:00 ` Szabolcs Nagy
0 siblings, 0 replies; 6+ messages in thread
From: Szabolcs Nagy @ 2016-01-01 0:00 UTC (permalink / raw)
To: Maciej W. Rozycki
Cc: Szabolcs Nagy, H.J. Lu, gnu-gabi, Ramana Radhakrishnan, Jeff Law,
Cary Coutant, Joe Groff, Alan Modra, Binutils, nd, Rich Felker
* Maciej W. Rozycki <macro@imgtec.com> [2016-04-16 00:59:17 +0100]:
> On Fri, 15 Apr 2016, Szabolcs Nagy wrote:
>
> > > Copy relocation and protected visibility are fundamentally incompatible.
> > > On on hand, copy relocation is the part of the psABI and is used to
> > > access global data defined in a shared object from the executable. It
> > > moves the definition of global data, which is defined in a share object,
> > > to the executable at run-time. On the other hand, protected visibility
> > > indicates that a symbol is defined locally in the shared object at
> > > run-time. Both can't be true at the same time. The current solution
> >
> > protected visibility indicates
> >
> > "that references within the defining module bind to
> > the definition in that module. That is, the declared
> > entity cannot be overridden by another module."
> >
> > here "definition in that module" does not mean addresses
> > at runtime, but the c language level definition.
> > (c semantics can only indicate abstract machine behaviour,
> > not relocation types and in memory layout at runtime).
>
> Where did you get this definition from and how do you infer this is a "c
> language level" rather than a binary level definition? The SVR4/ELF gABI
> says[1]:
>
> "A symbol defined in the current component is protected if it is visible
> in other components but not preemptable, meaning that any reference to
> such a symbol from within the defining component must be resolved to the
> definition in that component, even if there is a definition in another
> component that would preempt by the default rules."
>
> and:
>
> "The presence of the STV_PROTECTED flag on a symbol in a given load module
> does not affect the symbol resolution rules for references to that symbol
> from outside the containing load module."
>
> so it clearly indicates that it is dynamic load modules (i.e. either the
> main executable or any of its referred DSOs) that are considered here, not
> C source-level semantics. Besides the ELF ABI is programming language
> agnostic, using the C language as a reference only, so it cannot really
> consider source-level semantics.
>
sorry, i just looked at the c extension as documented
by gcc, but the elf gabi is the right source. however
it seems to verify my interpretation.
> > so there is no conflict between copy relocation and
> > protected visibility: the definition is not overridden,
> > it's an implementation detail that the address happens
> > to be in the data section of the main executable instead
> > of the defining module.
>
> Both the reference above and established practice assume a protected
> symbol must have its address within the module that defines it. See also:
copy relocation is not another definition for the symbol
(it's an implementation detail), and symbol resolution
rules for external references are not affected, so both
constraints you cited above hold now.
> <https://sourceware.org/ml/binutils/2016-03/msg00368.html> for my earlier
> comment.
ok, so it was intended to be an optimization (for function
symbols), but that breaks correctness for object symbols
in the presence of copy relocations.. so the optimization
is not valid: the symbol must be visible to other modules
which only works for the non-pic main module via copy relocs.
> References:
>
> [1] "System V Application Binary Interface - DRAFT - 10 June 2013",
> The Santa Cruz Operation, Inc., "Symbol Table",
> <http://www.sco.com/developers/gabi/latest/ch4.symtab.html>
>
> Maciej
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
2016-01-01 0:00 Preventing preemption of 'protected' symbols in GNU ld 2.26 H.J. Lu
@ 2016-01-01 0:00 ` Szabolcs Nagy
2016-01-01 0:00 ` Maciej W. Rozycki
2016-01-01 0:00 ` Jeff Law
1 sibling, 1 reply; 6+ messages in thread
From: Szabolcs Nagy @ 2016-01-01 0:00 UTC (permalink / raw)
To: H.J. Lu, gnu-gabi
Cc: Ramana Radhakrishnan, Jeff Law, Cary Coutant, Joe Groff,
Alan Modra, Binutils, nd, Rich Felker
On 15/04/16 17:16, H.J. Lu wrote:
> On Fri, Apr 15, 2016 at 9:09 AM, Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>> On 31/03/16 14:26, Ramana Radhakrishnan wrote:
>>> On Thu, Mar 31, 2016 at 1:52 AM, Jeff Law <law@redhat.com> wrote:
>>>> On 03/30/2016 06:40 PM, Cary Coutant wrote:
>>>>>>
>>>>>> It would help me immensely on the GCC side if things if you and Alan
>>>>>> could
>>>>>> easily summarize correct behavior and the impact if we were to just
>>>>>> revert
>>>>>> HJ's change. A testcase would be amazingly helpful too.
>>>>>
>>>>>
>>>>> It looks like it's not just the one change. There's this patch:
>>>>>
>>>>> https://gcc.gnu.org/ml/gcc-patches/2015-07/msg01871.html
>>>>>
>>>>> which took the idea that protected can still be pre-empted by a COPY
>>>>> relocation and extended it to three more targets that use COPY
>>>>> relocations.
>>>>>
>>>>> I wonder how many other patches have been based on the same
>>>>> misunderstanding?
>>
>> (sorry i missed this thread)
>>
>> this was not a misunderstanding.
>>
>> that patch is necessary for correctness (odr) in
>> the presence of copy relocations as described in
>> https://gcc.gnu.org/ml/gcc-patches/2015-09/msg02365.html
>> and
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55012
>>
>> this was a long standing code gen bug in gcc and was
>> about time to fix it (it was also broken in glibc's
>> dynamic linker, but e.g. not in musl libc).
>>
>> (i don't see what is the issue with using the copy in
>> the main executable from a shared library, performance
>> is not a correctness issue, nor how it is possible to
>> avoid the copy relocs.)
>>
>
> Here is my understanding:
>
> Copy relocation and protected visibility are fundamentally incompatible.
> On on hand, copy relocation is the part of the psABI and is used to
> access global data defined in a shared object from the executable. It
> moves the definition of global data, which is defined in a share object,
> to the executable at run-time. On the other hand, protected visibility
> indicates that a symbol is defined locally in the shared object at
> run-time. Both can't be true at the same time. The current solution
protected visibility indicates
"that references within the defining module bind to
the definition in that module. That is, the declared
entity cannot be overridden by another module."
here "definition in that module" does not mean addresses
at runtime, but the c language level definition.
(c semantics can only indicate abstract machine behaviour,
not relocation types and in memory layout at runtime).
so there is no conflict between copy relocation and
protected visibility: the definition is not overridden,
it's an implementation detail that the address happens
to be in the data section of the main executable instead
of the defining module.
i think your fixes in gcc, etc are ok.
(i understand that ppl want to optimize this but that
should be a separate discussion once there is consensus
about the correct semantics).
> is to make protected symbol more or less like normal symbol, which
> prevents optimizing local access to protected symbol within the shared
> object.
>
> I propose to add GNU_PROPERTY_NO_COPY_ON_PROTECTED:
>
> https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
>
> GNU_PROPERTY_NO_COPY_ON_PROTECTED This indicates that there
> should be no copy relocations against protected data symbols. If a relocat-
> able object contains this property, linker should treat protected data symbol
> as defined locally at run-time and copy this property to the output share
> object. Linker should add this property to the output share object if any pro-
> tected symbol is expected to be defined locally at run-time. Run-time loader
> should disallow copy relocations against protected data symbols defined in
> share objects with GNU_PROPERTY_NO_COPY_ON_PROTECTED prop-
> erty. Its PR_DATASZ should be 0.
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
@ 2016-01-01 0:00 H.J. Lu
2016-01-01 0:00 ` Szabolcs Nagy
2016-01-01 0:00 ` Jeff Law
0 siblings, 2 replies; 6+ messages in thread
From: H.J. Lu @ 2016-01-01 0:00 UTC (permalink / raw)
To: Szabolcs Nagy, gnu-gabi
Cc: Ramana Radhakrishnan, Jeff Law, Cary Coutant, Joe Groff,
Alan Modra, Binutils, nd
On Fri, Apr 15, 2016 at 9:09 AM, Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
> On 31/03/16 14:26, Ramana Radhakrishnan wrote:
>> On Thu, Mar 31, 2016 at 1:52 AM, Jeff Law <law@redhat.com> wrote:
>>> On 03/30/2016 06:40 PM, Cary Coutant wrote:
>>>>>
>>>>> It would help me immensely on the GCC side if things if you and Alan
>>>>> could
>>>>> easily summarize correct behavior and the impact if we were to just
>>>>> revert
>>>>> HJ's change. A testcase would be amazingly helpful too.
>>>>
>>>>
>>>> It looks like it's not just the one change. There's this patch:
>>>>
>>>> https://gcc.gnu.org/ml/gcc-patches/2015-07/msg01871.html
>>>>
>>>> which took the idea that protected can still be pre-empted by a COPY
>>>> relocation and extended it to three more targets that use COPY
>>>> relocations.
>>>>
>>>> I wonder how many other patches have been based on the same
>>>> misunderstanding?
>
> (sorry i missed this thread)
>
> this was not a misunderstanding.
>
> that patch is necessary for correctness (odr) in
> the presence of copy relocations as described in
> https://gcc.gnu.org/ml/gcc-patches/2015-09/msg02365.html
> and
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55012
>
> this was a long standing code gen bug in gcc and was
> about time to fix it (it was also broken in glibc's
> dynamic linker, but e.g. not in musl libc).
>
> (i don't see what is the issue with using the copy in
> the main executable from a shared library, performance
> is not a correctness issue, nor how it is possible to
> avoid the copy relocs.)
>
Here is my understanding:
Copy relocation and protected visibility are fundamentally incompatible.
On on hand, copy relocation is the part of the psABI and is used to
access global data defined in a shared object from the executable. It
moves the definition of global data, which is defined in a share object,
to the executable at run-time. On the other hand, protected visibility
indicates that a symbol is defined locally in the shared object at
run-time. Both can't be true at the same time. The current solution
is to make protected symbol more or less like normal symbol, which
prevents optimizing local access to protected symbol within the shared
object.
I propose to add GNU_PROPERTY_NO_COPY_ON_PROTECTED:
https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
GNU_PROPERTY_NO_COPY_ON_PROTECTED This indicates that there
should be no copy relocations against protected data symbols. If a relocat-
able object contains this property, linker should treat protected data symbol
as defined locally at run-time and copy this property to the output share
object. Linker should add this property to the output share object if any pro-
tected symbol is expected to be defined locally at run-time. Run-time loader
should disallow copy relocations against protected data symbols defined in
share objects with GNU_PROPERTY_NO_COPY_ON_PROTECTED prop-
erty. Its PR_DATASZ should be 0.
--
H.J.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
2016-01-01 0:00 ` Jeff Law
@ 2016-01-01 0:00 ` H.J. Lu
0 siblings, 0 replies; 6+ messages in thread
From: H.J. Lu @ 2016-01-01 0:00 UTC (permalink / raw)
To: Jeff Law, GNU C Library
Cc: Szabolcs Nagy, gnu-gabi, Ramana Radhakrishnan, Cary Coutant,
Joe Groff, Alan Modra, Binutils, nd
On Fri, Apr 15, 2016 at 9:36 AM, Jeff Law <law@redhat.com> wrote:
> On 04/15/2016 10:16 AM, H.J. Lu wrote:
>>
>> On Fri, Apr 15, 2016 at 9:09 AM, Szabolcs Nagy <szabolcs.nagy@arm.com>
>> wrote:
>>>
>>> On 31/03/16 14:26, Ramana Radhakrishnan wrote:
>>>>
>>>> On Thu, Mar 31, 2016 at 1:52 AM, Jeff Law <law@redhat.com> wrote:
>>>>>
>>>>> On 03/30/2016 06:40 PM, Cary Coutant wrote:
>>>>>>>
>>>>>>>
>>>>>>> It would help me immensely on the GCC side if things if you and Alan
>>>>>>> could
>>>>>>> easily summarize correct behavior and the impact if we were to just
>>>>>>> revert
>>>>>>> HJ's change. A testcase would be amazingly helpful too.
>>>>>>
>>>>>>
>>>>>>
>>>>>> It looks like it's not just the one change. There's this patch:
>>>>>>
>>>>>> https://gcc.gnu.org/ml/gcc-patches/2015-07/msg01871.html
>>>>>>
>>>>>> which took the idea that protected can still be pre-empted by a COPY
>>>>>> relocation and extended it to three more targets that use COPY
>>>>>> relocations.
>>>>>>
>>>>>> I wonder how many other patches have been based on the same
>>>>>> misunderstanding?
>>>
>>>
>>> (sorry i missed this thread)
>>>
>>> this was not a misunderstanding.
>>>
>>> that patch is necessary for correctness (odr) in
>>> the presence of copy relocations as described in
>>> https://gcc.gnu.org/ml/gcc-patches/2015-09/msg02365.html
>>> and
>>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55012
>>>
>>> this was a long standing code gen bug in gcc and was
>>> about time to fix it (it was also broken in glibc's
>>> dynamic linker, but e.g. not in musl libc).
>>>
>>> (i don't see what is the issue with using the copy in
>>> the main executable from a shared library, performance
>>> is not a correctness issue, nor how it is possible to
>>> avoid the copy relocs.)
>>>
>>
>> Here is my understanding:
>>
>> Copy relocation and protected visibility are fundamentally incompatible.
>> On on hand, copy relocation is the part of the psABI and is used to
>> access global data defined in a shared object from the executable. It
>> moves the definition of global data, which is defined in a share object,
>> to the executable at run-time. On the other hand, protected visibility
>> indicates that a symbol is defined locally in the shared object at
>> run-time. Both can't be true at the same time. The current solution
>> is to make protected symbol more or less like normal symbol, which
>> prevents optimizing local access to protected symbol within the shared
>> object.
>>
>> I propose to add GNU_PROPERTY_NO_COPY_ON_PROTECTED:
>>
>> https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
>>
>> GNU_PROPERTY_NO_COPY_ON_PROTECTED This indicates that there
>> should be no copy relocations against protected data symbols. If a
>> relocat-
>> able object contains this property, linker should treat protected data
>> symbol
>> as defined locally at run-time and copy this property to the output share
>> object. Linker should add this property to the output share object if any
>> pro-
>> tected symbol is expected to be defined locally at run-time. Run-time
>> loader
>> should disallow copy relocations against protected data symbols defined in
>> share objects with GNU_PROPERTY_NO_COPY_ON_PROTECTED prop-
>> erty. Its PR_DATASZ should be 0.
>
> I'd strongly suggest discussing directly with Carlos, Cary and Alan. My
> worry here is this just adding another layer of stuff to deal with a
> fundamentally broken concept -- protected visibility.
>
Adding glibc.
Protected symbol has been a tricky issue for glibc and binutils
from day 1. We have special treatment for pointers of protected
functions in both ld and ld.so. Protected symbol is a useful
feature. It just doesn't work with copy relocation. My proposal
will make it work for psABIs with copy relocation by disallowing
copy relocation on protected symbol. Linker change is minimum
and it will bypass extra symbol lookups in ld.so.
--
H.J.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
2016-01-01 0:00 Preventing preemption of 'protected' symbols in GNU ld 2.26 H.J. Lu
2016-01-01 0:00 ` Szabolcs Nagy
@ 2016-01-01 0:00 ` Jeff Law
2016-01-01 0:00 ` H.J. Lu
1 sibling, 1 reply; 6+ messages in thread
From: Jeff Law @ 2016-01-01 0:00 UTC (permalink / raw)
To: H.J. Lu, Szabolcs Nagy, gnu-gabi
Cc: Ramana Radhakrishnan, Cary Coutant, Joe Groff, Alan Modra, Binutils, nd
On 04/15/2016 10:16 AM, H.J. Lu wrote:
> On Fri, Apr 15, 2016 at 9:09 AM, Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>> On 31/03/16 14:26, Ramana Radhakrishnan wrote:
>>> On Thu, Mar 31, 2016 at 1:52 AM, Jeff Law <law@redhat.com> wrote:
>>>> On 03/30/2016 06:40 PM, Cary Coutant wrote:
>>>>>>
>>>>>> It would help me immensely on the GCC side if things if you and Alan
>>>>>> could
>>>>>> easily summarize correct behavior and the impact if we were to just
>>>>>> revert
>>>>>> HJ's change. A testcase would be amazingly helpful too.
>>>>>
>>>>>
>>>>> It looks like it's not just the one change. There's this patch:
>>>>>
>>>>> https://gcc.gnu.org/ml/gcc-patches/2015-07/msg01871.html
>>>>>
>>>>> which took the idea that protected can still be pre-empted by a COPY
>>>>> relocation and extended it to three more targets that use COPY
>>>>> relocations.
>>>>>
>>>>> I wonder how many other patches have been based on the same
>>>>> misunderstanding?
>>
>> (sorry i missed this thread)
>>
>> this was not a misunderstanding.
>>
>> that patch is necessary for correctness (odr) in
>> the presence of copy relocations as described in
>> https://gcc.gnu.org/ml/gcc-patches/2015-09/msg02365.html
>> and
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55012
>>
>> this was a long standing code gen bug in gcc and was
>> about time to fix it (it was also broken in glibc's
>> dynamic linker, but e.g. not in musl libc).
>>
>> (i don't see what is the issue with using the copy in
>> the main executable from a shared library, performance
>> is not a correctness issue, nor how it is possible to
>> avoid the copy relocs.)
>>
>
> Here is my understanding:
>
> Copy relocation and protected visibility are fundamentally incompatible.
> On on hand, copy relocation is the part of the psABI and is used to
> access global data defined in a shared object from the executable. It
> moves the definition of global data, which is defined in a share object,
> to the executable at run-time. On the other hand, protected visibility
> indicates that a symbol is defined locally in the shared object at
> run-time. Both can't be true at the same time. The current solution
> is to make protected symbol more or less like normal symbol, which
> prevents optimizing local access to protected symbol within the shared
> object.
>
> I propose to add GNU_PROPERTY_NO_COPY_ON_PROTECTED:
>
> https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI
>
> GNU_PROPERTY_NO_COPY_ON_PROTECTED This indicates that there
> should be no copy relocations against protected data symbols. If a relocat-
> able object contains this property, linker should treat protected data symbol
> as defined locally at run-time and copy this property to the output share
> object. Linker should add this property to the output share object if any pro-
> tected symbol is expected to be defined locally at run-time. Run-time loader
> should disallow copy relocations against protected data symbols defined in
> share objects with GNU_PROPERTY_NO_COPY_ON_PROTECTED prop-
> erty. Its PR_DATASZ should be 0.
I'd strongly suggest discussing directly with Carlos, Cary and Alan. My
worry here is this just adding another layer of stuff to deal with a
fundamentally broken concept -- protected visibility.
Jeff
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Preventing preemption of 'protected' symbols in GNU ld 2.26
2016-01-01 0:00 ` Szabolcs Nagy
@ 2016-01-01 0:00 ` Maciej W. Rozycki
2016-01-01 0:00 ` Szabolcs Nagy
0 siblings, 1 reply; 6+ messages in thread
From: Maciej W. Rozycki @ 2016-01-01 0:00 UTC (permalink / raw)
To: Szabolcs Nagy
Cc: H.J. Lu, gnu-gabi, Ramana Radhakrishnan, Jeff Law, Cary Coutant,
Joe Groff, Alan Modra, Binutils, nd, Rich Felker
On Fri, 15 Apr 2016, Szabolcs Nagy wrote:
> > Copy relocation and protected visibility are fundamentally incompatible.
> > On on hand, copy relocation is the part of the psABI and is used to
> > access global data defined in a shared object from the executable. It
> > moves the definition of global data, which is defined in a share object,
> > to the executable at run-time. On the other hand, protected visibility
> > indicates that a symbol is defined locally in the shared object at
> > run-time. Both can't be true at the same time. The current solution
>
> protected visibility indicates
>
> "that references within the defining module bind to
> the definition in that module. That is, the declared
> entity cannot be overridden by another module."
>
> here "definition in that module" does not mean addresses
> at runtime, but the c language level definition.
> (c semantics can only indicate abstract machine behaviour,
> not relocation types and in memory layout at runtime).
Where did you get this definition from and how do you infer this is a "c
language level" rather than a binary level definition? The SVR4/ELF gABI
says[1]:
"A symbol defined in the current component is protected if it is visible
in other components but not preemptable, meaning that any reference to
such a symbol from within the defining component must be resolved to the
definition in that component, even if there is a definition in another
component that would preempt by the default rules."
and:
"The presence of the STV_PROTECTED flag on a symbol in a given load module
does not affect the symbol resolution rules for references to that symbol
from outside the containing load module."
so it clearly indicates that it is dynamic load modules (i.e. either the
main executable or any of its referred DSOs) that are considered here, not
C source-level semantics. Besides the ELF ABI is programming language
agnostic, using the C language as a reference only, so it cannot really
consider source-level semantics.
> so there is no conflict between copy relocation and
> protected visibility: the definition is not overridden,
> it's an implementation detail that the address happens
> to be in the data section of the main executable instead
> of the defining module.
Both the reference above and established practice assume a protected
symbol must have its address within the module that defines it. See also:
<https://sourceware.org/ml/binutils/2016-03/msg00368.html> for my earlier
comment.
References:
[1] "System V Application Binary Interface - DRAFT - 10 June 2013",
The Santa Cruz Operation, Inc., "Symbol Table",
<http://www.sco.com/developers/gabi/latest/ch4.symtab.html>
Maciej
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-04-16 1:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-01 0:00 Preventing preemption of 'protected' symbols in GNU ld 2.26 H.J. Lu
2016-01-01 0:00 ` Szabolcs Nagy
2016-01-01 0:00 ` Maciej W. Rozycki
2016-01-01 0:00 ` Szabolcs Nagy
2016-01-01 0:00 ` Jeff Law
2016-01-01 0:00 ` H.J. Lu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).