Re: [PATCH 2/2] elf: Add GNU_PROPERTY_1_NEEDED check

Re: [PATCH 2/2] elf: Add GNU_PROPERTY_1_NEEDED check

[H.J. Lu]

On Mon, Jun 21, 2021 at 10:46 PM Fangrui Song <i@maskray.me> wrote:
>
> On 2021-06-21, H.J. Lu wrote:
> >On Mon, Jun 21, 2021 at 9:16 PM Alan Modra <amodra@gmail.com> wrote:
> >>
> >> On Mon, Jun 21, 2021 at 07:12:02PM -0700, H.J. Lu wrote:
> >> > On Mon, Jun 21, 2021 at 5:06 PM Alan Modra <amodra@gmail.com> wrote:
> >> > >
> >> > > On Mon, Jun 21, 2021 at 03:34:38PM -0700, Fangrui Song wrote:
> >> > > > clang -fno-pic -fno-direct-access-extern-data  works with clang>=12.0.0 today.
> >> > >
> >> > > -fno-direct-access-extern-data or variations on that also seem good to
> >> > > me.  -fpic-extern would also work.  I liked -fprotected-abi because
> >> > > it shows the intent of correcting abi issues related to protected
> >> > > visibility.  (Yes, it affects code for all undefined symbols because
> >> > > the compiler clearly isn't seeing the entire program if there are
> >> > > undefined symbols.)
> >> >
> >> > I need an option which can be turned on and off.   How about
> >> > -fextern-access=direct and -fextern-access=indirect?  It will cover
> >> > both data and function?
>
> -fno-direct-access-external-data and -fdirect-access-external-data can turn on/off the bit.
>
> clang -fno-pic -fno-direct-access-external-data  works for x86-64 and aarch64.
>
> We can add a -fno-direct-access-external

Since both clang and GCC will add a new option for both data and function
symbols, can we have an agreement on the new option name?  I am listing
options here:

1. -fdirect-access-external/-fno-direct-access-external
2. -fdirect-extern-access/-fno-direct-exern-access
3. -fdirect-external-access/-fno-direct-exernal-access
4. -fextern-access=direct/-fextern-access=indirect
5. -fexternal-access=direct/-fexternal-access=indirect

My order of preferences are 4, 5, 2, 3, 1.

> >> Yes, FWIW that option name for gcc also looks good to me.
> >
> >I will change the gcc option to
> >
> >-fextern-access=direct
> >-fextern-access=indirect
> >
> >and change GNU_PROPERTY_1_NEEDED_SINGLE_GLOBAL_DEFINITION
> >to GNU_PROPERTY_1_NEEDED_INDIRECT_EXTERN_ACCESS
>
> Note that this will be a glibc + GNU ld specific thing.
>
> gold and ld.lld error for copy relocations on protected data symbols by default.

At run-time, there will be a mixture of components built with different tools
over time.  A marker will help glibc to avoid potential run-time failures due
to binary incompatibility.

> >> Now as to the need for a corresponding linker option, I'm of the
> >> opinion that it is ideal for the linker to be able to cope without
> >> needing special options.  Can you show me a set of object files (or
> >> just describe them) where ld cannot deduce from relocations and
> >> dynamic symbols what dynbss copies, plt stubs, and dynamic relocations
> >> are needed?  I'm fairly sure I manage to do that for powerpc.
> >>
> >> Note that I'm not against a new option to force the linker to go
> >> against what it would do based on input object files (perhaps
> >
> >I'd like to turn it on in linker without any compiler changes, especially
> >when building shared libraries, kind of a subset of -Bsymbolic.
> >
> >> reporting errors), but don't think we should have a new option without
> >> some effort being made to see whether we really need it.
> >
> >Here is a glibc patch to use both linker options on some testcases:
> >
> >https://sourceware.org/pipermail/libc-alpha/2021-June/127770.html
> >
> >> > > The main thing that struck me about -fsingle-global-definition is that
> >> > > the option doesn't do what it says.  You can still have multiple
> >> > > global definitions of a given symbol, one in the executable and one in
> >> > > each of the shared libraries making up the complete program.  Which of
> >> > > course is no different to code without -fsingle-global-definition.
> >> >
> >> >
> >> > --
> >> > H.J.
> >>
> >> --
> >> Alan Modra
> >> Australia Development Lab, IBM
> >
> >
> >
> >--
> >H.J.



-- 
H.J.

Re: [PATCH 2/2] elf: Add GNU_PROPERTY_1_NEEDED check

[Fangrui Song]

On 2021-06-22, H.J. Lu wrote:
>On Mon, Jun 21, 2021 at 10:46 PM Fangrui Song <i@maskray.me> wrote:
>>
>> On 2021-06-21, H.J. Lu wrote:
>> >On Mon, Jun 21, 2021 at 9:16 PM Alan Modra <amodra@gmail.com> wrote:
>> >>
>> >> On Mon, Jun 21, 2021 at 07:12:02PM -0700, H.J. Lu wrote:
>> >> > On Mon, Jun 21, 2021 at 5:06 PM Alan Modra <amodra@gmail.com> wrote:
>> >> > >
>> >> > > On Mon, Jun 21, 2021 at 03:34:38PM -0700, Fangrui Song wrote:
>> >> > > > clang -fno-pic -fno-direct-access-extern-data  works with clang>=12.0.0 today.
>> >> > >
>> >> > > -fno-direct-access-extern-data or variations on that also seem good to
>> >> > > me.  -fpic-extern would also work.  I liked -fprotected-abi because
>> >> > > it shows the intent of correcting abi issues related to protected
>> >> > > visibility.  (Yes, it affects code for all undefined symbols because
>> >> > > the compiler clearly isn't seeing the entire program if there are
>> >> > > undefined symbols.)
>> >> >
>> >> > I need an option which can be turned on and off.   How about
>> >> > -fextern-access=direct and -fextern-access=indirect?  It will cover
>> >> > both data and function?
>>
>> -fno-direct-access-external-data and -fdirect-access-external-data can turn on/off the bit.
>>
>> clang -fno-pic -fno-direct-access-external-data  works for x86-64 and aarch64.
>>
>> We can add a -fno-direct-access-external
>
>Since both clang and GCC will add a new option for both data and function
>symbols, can we have an agreement on the new option name?  I am listing
>options here:
>
>1. -fdirect-access-external/-fno-direct-access-external
>2. -fdirect-extern-access/-fno-direct-exern-access
>3. -fdirect-external-access/-fno-direct-exernal-access
>4. -fextern-access=direct/-fextern-access=indirect
>5. -fexternal-access=direct/-fexternal-access=indirect
>
>My order of preferences are 4, 5, 2, 3, 1.

Preferring "extern" to "external" looks fine to me. (`extern` is the C/C++ construct anyway and this option describes what to do with default visibility non-definition `extern int xxx`/`extern void foo()`).

-fextern-access=direct/-fextern-access=indirect and -fdirect-extern-access/-fno-direct-exern-access

look good to me.

I am happy to add aliases to clang if consensus is moving toward  -fextern-access=indirect or -fno-direct-extern-access.

>> >> Yes, FWIW that option name for gcc also looks good to me.
>> >
>> >I will change the gcc option to
>> >
>> >-fextern-access=direct
>> >-fextern-access=indirect
>> >
>> >and change GNU_PROPERTY_1_NEEDED_SINGLE_GLOBAL_DEFINITION
>> >to GNU_PROPERTY_1_NEEDED_INDIRECT_EXTERN_ACCESS
>>
>> Note that this will be a glibc + GNU ld specific thing.
>>
>> gold and ld.lld error for copy relocations on protected data symbols by default.
>
>At run-time, there will be a mixture of components built with different tools
>over time.  A marker will help glibc to avoid potential run-time failures due
>to binary incompatibility.

glibc can perform the check without a GNU PROPERTY.


For a st_value!=0 && st_shndx==0 symbol lookup during relocation
processing, if the definition is found protected in a shared object,
ld.so can report an error and make a suggestion like
"consider recompiling the executable with -fno-direct-extern-access or
-fpie/-fpic"


I echo Michael's question in another thread
https://sourceware.org/pipermail/binutils/2021-June/117137.html

"IOW: which scenario do you want to not error on when you want to make the error conditional?"

For such rare cases, we can use a LD_* environment variable.

>> >> Now as to the need for a corresponding linker option, I'm of the
>> >> opinion that it is ideal for the linker to be able to cope without
>> >> needing special options.  Can you show me a set of object files (or
>> >> just describe them) where ld cannot deduce from relocations and
>> >> dynamic symbols what dynbss copies, plt stubs, and dynamic relocations
>> >> are needed?  I'm fairly sure I manage to do that for powerpc.
>> >>
>> >> Note that I'm not against a new option to force the linker to go
>> >> against what it would do based on input object files (perhaps
>> >
>> >I'd like to turn it on in linker without any compiler changes, especially
>> >when building shared libraries, kind of a subset of -Bsymbolic.
>> >
>> >> reporting errors), but don't think we should have a new option without
>> >> some effort being made to see whether we really need it.
>> >
>> >Here is a glibc patch to use both linker options on some testcases:
>> >
>> >https://sourceware.org/pipermail/libc-alpha/2021-June/127770.html
>> >
>> >> > > The main thing that struck me about -fsingle-global-definition is that
>> >> > > the option doesn't do what it says.  You can still have multiple
>> >> > > global definitions of a given symbol, one in the executable and one in
>> >> > > each of the shared libraries making up the complete program.  Which of
>> >> > > course is no different to code without -fsingle-global-definition.
>> >> >
>> >> >
>> >> > --
>> >> > H.J.
>> >>
>> >> --
>> >> Alan Modra
>> >> Australia Development Lab, IBM
>> >
>> >
>> >
>> >--
>> >H.J.
>
>
>
>-- 
>H.J.

Re: [PATCH 2/2] elf: Add GNU_PROPERTY_1_NEEDED check

[H.J. Lu]

On Tue, Jun 22, 2021 at 11:15 AM Fangrui Song <i@maskray.me> wrote:
>
> On 2021-06-22, H.J. Lu wrote:
> >On Mon, Jun 21, 2021 at 10:46 PM Fangrui Song <i@maskray.me> wrote:
> >>
> >> On 2021-06-21, H.J. Lu wrote:
> >> >On Mon, Jun 21, 2021 at 9:16 PM Alan Modra <amodra@gmail.com> wrote:
> >> >>
> >> >> On Mon, Jun 21, 2021 at 07:12:02PM -0700, H.J. Lu wrote:
> >> >> > On Mon, Jun 21, 2021 at 5:06 PM Alan Modra <amodra@gmail.com> wrote:
> >> >> > >
> >> >> > > On Mon, Jun 21, 2021 at 03:34:38PM -0700, Fangrui Song wrote:
> >> >> > > > clang -fno-pic -fno-direct-access-extern-data  works with clang>=12.0.0 today.
> >> >> > >
> >> >> > > -fno-direct-access-extern-data or variations on that also seem good to
> >> >> > > me.  -fpic-extern would also work.  I liked -fprotected-abi because
> >> >> > > it shows the intent of correcting abi issues related to protected
> >> >> > > visibility.  (Yes, it affects code for all undefined symbols because
> >> >> > > the compiler clearly isn't seeing the entire program if there are
> >> >> > > undefined symbols.)
> >> >> >
> >> >> > I need an option which can be turned on and off.   How about
> >> >> > -fextern-access=direct and -fextern-access=indirect?  It will cover
> >> >> > both data and function?
> >>
> >> -fno-direct-access-external-data and -fdirect-access-external-data can turn on/off the bit.
> >>
> >> clang -fno-pic -fno-direct-access-external-data  works for x86-64 and aarch64.
> >>
> >> We can add a -fno-direct-access-external
> >
> >Since both clang and GCC will add a new option for both data and function
> >symbols, can we have an agreement on the new option name?  I am listing
> >options here:
> >
> >1. -fdirect-access-external/-fno-direct-access-external
> >2. -fdirect-extern-access/-fno-direct-exern-access
> >3. -fdirect-external-access/-fno-direct-exernal-access
> >4. -fextern-access=direct/-fextern-access=indirect
> >5. -fexternal-access=direct/-fexternal-access=indirect
> >
> >My order of preferences are 4, 5, 2, 3, 1.
>
> Preferring "extern" to "external" looks fine to me. (`extern` is the C/C++ construct anyway and this option describes what to do with default visibility non-definition `extern int xxx`/`extern void foo()`).
>
> -fextern-access=direct/-fextern-access=indirect and -fdirect-extern-access/-fno-direct-exern-access
>
> look good to me.

Let's go with -fdirect-extern-access/-fno-direct-exern-access

> I am happy to add aliases to clang if consensus is moving toward  -fextern-access=indirect or -fno-direct-extern-access.
>
> >> >> Yes, FWIW that option name for gcc also looks good to me.
> >> >
> >> >I will change the gcc option to
> >> >
> >> >-fextern-access=direct
> >> >-fextern-access=indirect
> >> >
> >> >and change GNU_PROPERTY_1_NEEDED_SINGLE_GLOBAL_DEFINITION
> >> >to GNU_PROPERTY_1_NEEDED_INDIRECT_EXTERN_ACCESS
> >>
> >> Note that this will be a glibc + GNU ld specific thing.
> >>
> >> gold and ld.lld error for copy relocations on protected data symbols by default.
> >
> >At run-time, there will be a mixture of components built with different tools
> >over time.  A marker will help glibc to avoid potential run-time failures due
> >to binary incompatibility.
>
> glibc can perform the check without a GNU PROPERTY.
>
>
> For a st_value!=0 && st_shndx==0 symbol lookup during relocation
> processing, if the definition is found protected in a shared object,
> ld.so can report an error and make a suggestion like
> "consider recompiling the executable with -fno-direct-extern-access or
> -fpie/-fpic"

There is no run-time issue when the shared object isn't built with
-fno-direct-extern-access.   That is what the marker is used for.

>
> I echo Michael's question in another thread
> https://sourceware.org/pipermail/binutils/2021-June/117137.html
>
> "IOW: which scenario do you want to not error on when you want to make the error conditional?"
>
> For such rare cases, we can use a LD_* environment variable.
>
> >> >> Now as to the need for a corresponding linker option, I'm of the
> >> >> opinion that it is ideal for the linker to be able to cope without
> >> >> needing special options.  Can you show me a set of object files (or
> >> >> just describe them) where ld cannot deduce from relocations and
> >> >> dynamic symbols what dynbss copies, plt stubs, and dynamic relocations
> >> >> are needed?  I'm fairly sure I manage to do that for powerpc.
> >> >>
> >> >> Note that I'm not against a new option to force the linker to go
> >> >> against what it would do based on input object files (perhaps
> >> >
> >> >I'd like to turn it on in linker without any compiler changes, especially
> >> >when building shared libraries, kind of a subset of -Bsymbolic.
> >> >
> >> >> reporting errors), but don't think we should have a new option without
> >> >> some effort being made to see whether we really need it.
> >> >
> >> >Here is a glibc patch to use both linker options on some testcases:
> >> >
> >> >https://sourceware.org/pipermail/libc-alpha/2021-June/127770.html
> >> >
> >> >> > > The main thing that struck me about -fsingle-global-definition is that
> >> >> > > the option doesn't do what it says.  You can still have multiple
> >> >> > > global definitions of a given symbol, one in the executable and one in
> >> >> > > each of the shared libraries making up the complete program.  Which of
> >> >> > > course is no different to code without -fsingle-global-definition.
> >> >> >
> >> >> >
> >> >> > --
> >> >> > H.J.
> >> >>
> >> >> --
> >> >> Alan Modra
> >> >> Australia Development Lab, IBM
> >> >
> >> >
> >> >
> >> >--
> >> >H.J.
> >
> >
> >
> >--
> >H.J.



-- 
H.J.