Re: Specify how undefined weak symbol should be resolved in executable

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Tue, 23 Feb 2016, H.J. Lu wrote:

> Not only we need to change defined weak symbol behavior, we also need to 
> change undefined non-weak symbol behavior when taking its address.

Why?  An undefined non-weak symbol reference leads to a linker error, 
done.  Why change that?


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 9:45 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Mon, Feb 22, 2016 at 8:25 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> Currently when building executable, undefined weak symbol is resolved
>> differently, depending on if the relocatable input is compiled with PIC or
>> not:
>>
>> hjl@gnu-tools-1 weak-1]$ cat x.c
>> extern void __attribute__ ((weak)) fun (void);
>> int main (void)
>> {
>>   if (&fun != 0)
>>     fun ();
>>   return 0;
>> }
>> [hjl@gnu-tools-1 weak-1]$ make
>> gcc -B./ -O2 -o x x.c
>> gcc -B./ -O2 -o y x.c -fPIC
>> readelf -r x
>>
>> Relocation section '.rela.dyn' at offset 0x328 contains 1 entries:
>>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 000000600870  000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>>
>> Relocation section '.rela.plt' at offset 0x340 contains 1 entries:
>>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 000000600890  000100000007 R_X86_64_JUMP_SLO 0000000000000000
>> __libc_start_main@GLIBC_2.2.5 + 0
>>
>> There is no dynamic relocation against fun, whose value is
>> resolved to zero at link-time.
>>
>> readelf -r y
>>
>> Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
>>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 0000006008a0  000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
>> 0000006008a8  000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>>
>> Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
>>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
>> 0000006008c8  000200000007 R_X86_64_JUMP_SLO 0000000000000000
>> __libc_start_main@GLIBC_2.2.5 + 0
>> [hjl@gnu-tools-1 weak-1]$
>>
>> There is dynamic relocation against fun, whose value is resolved
>> at runt-time. We should make it consistent, regardless if input
>> relocatable file is compiled with PIC or not.
>>
>> I raised the issue at gABI:
>>
>> https://groups.google.com/d/msg/generic-abi/Ckq19PfLxyk/eCbXPofkAQAJ
>>
>> But if gABI change is incompatible with other OSes, we need to
>> specify it in GNU ABI.
>
> We should extend GNU ABI to:
>
> When creating executable, if dynamic relocation is available at
> run-time, the link
> editor should generate dynamic relocations against unresolved weak symbols so
> that their values will be resolved at run-time.
>

I opened:

https://sourceware.org/bugzilla/show_bug.cgi?id=19704

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 8:40 PM, Alan Modra <amodra@gmail.com> wrote:
> On Mon, Feb 22, 2016 at 02:43:55PM -0800, H.J. Lu wrote:
>> we create executable with dynamic R_X86_64_32 relocation against
>> fun .text section, which leads to DT_TEXTREL.  Is this what we want?
>
> I think that avoiding DT_TEXTREL is historically why we haven't
> supported weak symbols well in non-PIC executables.  Another closely
> related issue is the old hack written into most ABIs that the address
> of a function referenced in an executable and defined in a shared
> library is the address of that function's PLT code in the executable.
>
> Note that this affects both defined and undefined weak functions in
> non-PIC executables.  Currently, if we find a definition for "fun" at
> link time in a shared library, "&fun" is the PLT entry in the
> executable and "if (&fun)" effectively becomes "if (1)".  So if you
> link the executable with a shared library containing a definition of
> "fun" but at run-time load a newer shared library *not* defining "fun",
> your program will segfault.
>
> Therefore, if the aim is to make PIC and non-PIC executables behave
> the same you need to change defined weak symbol behaviour too.  Any
> non-PIC executable that takes the address of a weak function in code
> or rodata would become DT_TEXTREL.
>

Not only we need to change defined weak symbol behavior, we
also need to change undefined non-weak symbol behavior when
taking its address.  We need to generate PIC-like code to take
address of a symbol in non-PIC text section.  Is this what we
want?


-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 8:40 PM, Alan Modra <amodra@gmail.com> wrote:
>
> However, that might be a bad idea.  Lots of C++ code uses weak symbols
> for functions defined in header files, and other objects with vague
> linkage.  These result in weak definitions in eg. libstdc++.so.  I'm
> not sure how many executables take the address of such functions and
> thus might become DT_TEXTREL.

Most, if not all, of programs will have DT_TEXTREL on x86 if undefined
weak symbols require dynamic relocations.


-- 
H.J.
---
[hjl@gnu-4 ~]$ objdump -dwr  /usr/lib/gcc/x86_64-redhat-linux/5.3.1/crtbegin.o
/usr/lib/gcc/x86_64-redhat-linux/5.3.1/crtbegin.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <deregister_tm_clones>:
   0: b8 00 00 00 00       mov    $0x0,%eax 1: R_X86_64_32 __TMC_END__+0x7
   5: 55                   push   %rbp
   6: 48 2d 00 00 00 00     sub    $0x0,%rax 8: R_X86_64_32S .tm_clone_table
   c: 48 83 f8 0e           cmp    $0xe,%rax
  10: 48 89 e5             mov    %rsp,%rbp
  13: 76 1b                 jbe    30 <deregister_tm_clones+0x30>
  15: b8 00 00 00 00       mov    $0x0,%eax 16: R_X86_64_32
_ITM_deregisterTMCloneTable
  1a: 48 85 c0             test   %rax,%rax
  1d: 74 11                 je     30 <deregister_tm_clones+0x30>
  1f: 5d                   pop    %rbp
  20: bf 00 00 00 00       mov    $0x0,%edi 21: R_X86_64_32 .tm_clone_table
  25: ff e0                 jmpq   *%rax
  27: 66 0f 1f 84 00 00 00 00 00 nopw   0x0(%rax,%rax,1)
  30: 5d                   pop    %rbp
  31: c3                   retq
  32: 0f 1f 40 00           nopl   0x0(%rax)
  36: 66 2e 0f 1f 84 00 00 00 00 00 nopw   %cs:0x0(%rax,%rax,1)

0000000000000040 <register_tm_clones>:
  40: be 00 00 00 00       mov    $0x0,%esi 41: R_X86_64_32 __TMC_END__
  45: 55                   push   %rbp
  46: 48 81 ee 00 00 00 00 sub    $0x0,%rsi 49: R_X86_64_32S .tm_clone_table
  4d: 48 c1 fe 03           sar    $0x3,%rsi
  51: 48 89 e5             mov    %rsp,%rbp
  54: 48 89 f0             mov    %rsi,%rax
  57: 48 c1 e8 3f           shr    $0x3f,%rax
  5b: 48 01 c6             add    %rax,%rsi
  5e: 48 d1 fe             sar    %rsi
  61: 74 15                 je     78 <register_tm_clones+0x38>
  63: b8 00 00 00 00       mov    $0x0,%eax 64: R_X86_64_32
_ITM_registerTMCloneTable
  68: 48 85 c0             test   %rax,%rax
  6b: 74 0b                 je     78 <register_tm_clones+0x38>
  6d: 5d                   pop    %rbp
  6e: bf 00 00 00 00       mov    $0x0,%edi 6f: R_X86_64_32 .tm_clone_table
  73: ff e0                 jmpq   *%rax
  75: 0f 1f 00             nopl   (%rax)
  78: 5d                   pop    %rbp
  79: c3                   retq
  7a: 66 0f 1f 44 00 00     nopw   0x0(%rax,%rax,1)

0000000000000080 <__do_global_dtors_aux>:
  80: 80 3d 00 00 00 00 00 cmpb   $0x0,0x0(%rip)        # 87
<__do_global_dtors_aux+0x7> 82: R_X86_64_PC32 .bss-0x5
  87: 75 11                 jne    9a <__do_global_dtors_aux+0x1a>
  89: 55                   push   %rbp
  8a: 48 89 e5             mov    %rsp,%rbp
  8d: e8 6e ff ff ff       callq  0 <deregister_tm_clones>
  92: 5d                   pop    %rbp
  93: c6 05 00 00 00 00 01 movb   $0x1,0x0(%rip)        # 9a
<__do_global_dtors_aux+0x1a> 95: R_X86_64_PC32 .bss-0x5
  9a: f3 c3                 repz retq
  9c: 0f 1f 40 00           nopl   0x0(%rax)

00000000000000a0 <frame_dummy>:
  a0: bf 00 00 00 00       mov    $0x0,%edi a1: R_X86_64_32 .jcr
  a5: 48 83 3f 00           cmpq   $0x0,(%rdi)
  a9: 75 05                 jne    b0 <frame_dummy+0x10>
  ab: eb 93                 jmp    40 <register_tm_clones>
  ad: 0f 1f 00             nopl   (%rax)
  b0: b8 00 00 00 00       mov    $0x0,%eax b1: R_X86_64_32 _Jv_RegisterClasses
  b5: 48 85 c0             test   %rax,%rax
  b8: 74 f1                 je     ab <frame_dummy+0xb>
  ba: 55                   push   %rbp
  bb: 48 89 e5             mov    %rsp,%rbp
  be: ff d0                 callq  *%rax
  c0: 5d                   pop    %rbp
  c1: e9 7a ff ff ff       jmpq   40 <register_tm_clones>
[hjl@gnu-4 ~]$ readelf -sW
/usr/lib/gcc/x86_64-redhat-linux/5.3.1/crtbegin.o  | grep WEAK
    23: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_deregisterTMCloneTable
    24: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_registerTMCloneTable
    26: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
[hjl@gnu-4 ~]$

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Tue, 23 Feb 2016, H.J. Lu wrote:

> On Mon, Feb 22, 2016 at 8:40 PM, Alan Modra <amodra@gmail.com> wrote:
> >
> > However, that might be a bad idea.  Lots of C++ code uses weak symbols
> > for functions defined in header files, and other objects with vague
> > linkage.  These result in weak definitions in eg. libstdc++.so.  I'm
> > not sure how many executables take the address of such functions and
> > thus might become DT_TEXTREL.
> 
> Most, if not all, of programs will have DT_TEXTREL on x86 if undefined
> weak symbols require dynamic relocations.

Hmm, that's less than ideal of course.  Well, if the goal is to make PIC 
and non-PIC the same, we could also go the opposite way: make PIC behave 
like non-PIC, i.e. resolve weak symbols always at link editing time.  That 
of course would remove features (can't change libs at runtime anymore, if 
they change in definedness of such a symbol).

Or, a third variant: change the compiler to emit PIC like code for taking 
addresses of weak symbols also in generally non-PIC code, so we could 
avoid TEXTREL.

I think the ideal solution would be that last one, change link editor now 
to behave like with PIC code, and eventually fix the compiler to not have 
to generate TEXTREL.

Note that the existence of DT_TEXTREL itself isn't that bad: only those 
pages that actually contain relocations will be unshared, so for the 
example of crtbegin.o it's only one page per process.  In addition 
crtbegin could of course always be PIC code, avoiding the issue.

I've looked at a normal c++ program (zypper) and the only weak undef 
symbols are those from crtbegin.  There are many other weak symbols, but 
they are defined in the executable itself (it's mostly template 
instantiations), so pose no problem.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 8:25 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> Currently when building executable, undefined weak symbol is resolved
> differently, depending on if the relocatable input is compiled with PIC or
> not:
>
> hjl@gnu-tools-1 weak-1]$ cat x.c
> extern void __attribute__ ((weak)) fun (void);
> int main (void)
> {
>   if (&fun != 0)
>     fun ();
>   return 0;
> }
> [hjl@gnu-tools-1 weak-1]$ make
> gcc -B./ -O2 -o x x.c
> gcc -B./ -O2 -o y x.c -fPIC
> readelf -r x
>
> Relocation section '.rela.dyn' at offset 0x328 contains 1 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000000600870  000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>
> Relocation section '.rela.plt' at offset 0x340 contains 1 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 000000600890  000100000007 R_X86_64_JUMP_SLO 0000000000000000
> __libc_start_main@GLIBC_2.2.5 + 0
>
> There is no dynamic relocation against fun, whose value is
> resolved to zero at link-time.
>
> readelf -r y
>
> Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 0000006008a0  000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
> 0000006008a8  000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>
> Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
>   Offset          Info           Type           Sym. Value    Sym. Name + Addend
> 0000006008c8  000200000007 R_X86_64_JUMP_SLO 0000000000000000
> __libc_start_main@GLIBC_2.2.5 + 0
> [hjl@gnu-tools-1 weak-1]$
>
> There is dynamic relocation against fun, whose value is resolved
> at runt-time. We should make it consistent, regardless if input
> relocatable file is compiled with PIC or not.
>
> I raised the issue at gABI:
>
> https://groups.google.com/d/msg/generic-abi/Ckq19PfLxyk/eCbXPofkAQAJ
>
> But if gABI change is incompatible with other OSes, we need to
> specify it in GNU ABI.

We should extend GNU ABI to:

When creating executable, if dynamic relocation is available at
run-time, the link
editor should generate dynamic relocations against unresolved weak symbols so
that their values will be resolved at run-time.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 9:10 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Tue, Feb 23, 2016 at 9:01 AM, Michael Matz <matz@suse.de> wrote:
>> Hi,
>>
>> On Tue, 23 Feb 2016, H.J. Lu wrote:
>>
>>> >> Not only we need to change defined weak symbol behavior, we also need
>>> >> to change undefined non-weak symbol behavior when taking its address.
>>> >
>>> > Why?  An undefined non-weak symbol reference leads to a linker error,
>>> > done.  Why change that?
>>>
>>> Make it "external non-weak symbol", which is a symbol defined in a
>>> shared object.
>>
>> What would you like to change in behaviour for them (and why must it be
>> done at the same time as changing something for weak symbols)?  A
>> reference to the address of such a function symbol from an executable
>> resolves to the plt slot currently (which in turn is exported, so that
>> address references from other modules resolve to the same).  A reference
>> to a non-function symbol is loaded from the got.  In all cases must this
>> symbol be defined somewhere (otherwise linker error), so I don't see what
>> we need to change, nor why that would force DT_TEXTREL (or the compiler to
>> emit PIC like code).
>>
>
> At run-time, there is no difference between weak defined and non-weak
> defined symbols.  If we change defined weak symbol behavior, we also
> need to change defined non-weak symbol behavior.
>

I will check this patch:

https://sourceware.org/ml/binutils/2016-02/msg00339.html

into binutils master branch to make behaviors of static executable,
dynamic executable and position independent executable consistent with
references to undefined weak and defined symbols, regardless if the
relocatable input is compiled with PIC or not.

I created users/hjl/pr19704 branch to treat undefined weak symbol in
the executable as dynamic, requiring dynamic relocation, on x86 via
a command line switch:

-z dynamic-undefined-weak   Treat undefined weak symbol as dynamic

We need to fully investigate its implications before moving forward
with it.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Wed, 24 Feb 2016, H.J. Lu wrote:

> > Agreed. Anything that brings back weak symbols into the dynamic loader 
> > is going to be highly suspect to me, and require a lot of explaining.
> 
> I was saying as far as ld was concerned, weak defined and non-weak 
> defined dynamic symbols would be treated equally at run-time.  Do you 
> agree with me?

I at least don't.  The difference is that a defined weak symbol (at link 
edit time) might become undefined at runtime.  A defined non-weak symbol 
can't.  So they have to be handled differently.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 2:47 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Mon, Feb 22, 2016 at 2:43 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> On Mon, Feb 22, 2016 at 10:28 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>>> On Mon, Feb 22, 2016 at 10:17 AM, Michael Matz <matz@suse.de> wrote:
>>>> Hi,
>>>>
>>>> On Mon, 22 Feb 2016, H.J. Lu wrote:
>>>>
>>>>> > Typo?  What is the meaning of "dynamic relocation available at
>>>>> > runtime". Are you talking about a relocation entry or the relocation
>>>>> > process?
>>>>>
>>>>> Dynamic relocation isn't available with static linking.
>>>>
>>>> So you _are_ talking about the process.  I think it's customary to call
>>>> such executables "dynamic", or perhaps non-static, isn't it?  (Why I'm
>>>> confused with your wording: a dynamic executable which happens to have no
>>>> dynamic reloc entries, has "no dynamic relocation available at runtime",
>>>> and so we couldn't add one :))
>>>>
>>>>> > How about "When creating a dynamic executable the link editor should
>>>>> > ..."?
>>>>
>>>> So, I still can understand my version better (possibly with
>>>> s/dynamic/non-static/) :)
>>>>
>>>
>>> Let's go with:
>>>
>>> When creating dynamic executable, the link editor should generate dynamic
>>> relocations against unresolved weak symbols so that their values will be
>>> resolved at run-time.
>>>
>>
>> Second thought.  To generate dynamic relocation for R_X86_64_32
>> against fun in:
>>
>> [hjl@gnu-6 pr19704]$ objdump -dwr foo.o
>>
>> foo.o:     file format elf64-x86-64
>>
>>
>> Disassembly of section .text.startup:
>>
>> 0000000000000000 <main>:
>>    0: b8 00 00 00 00       mov    $0x0,%eax 1: R_X86_64_32 fun
>>    5: 48 85 c0             test   %rax,%rax
>>    8: 74 10                 je     1a <main+0x1a>
>>    a: 48 83 ec 08           sub    $0x8,%rsp
>>    e: e8 00 00 00 00       callq  13 <main+0x13> f: R_X86_64_PC32 fun-0x4
>>   13: 31 c0                 xor    %eax,%eax
>>   15: 48 83 c4 08           add    $0x8,%rsp
>>   19: c3                   retq
>>   1a: 31 c0                 xor    %eax,%eax
>>   1c: c3                   retq
>> [hjl@gnu-6 pr19704]$
>>
>> we create executable with dynamic R_X86_64_32 relocation against
>> fun .text section, which leads to DT_TEXTREL.  Is this what we want?
>>
>
> I believe the intention of undefined weak symbol is it should be resolved
> at link-time when creating executable, dynamic or static.

There is discrepancy between PIC input and non-PIC input on x86.
There may be a discrepancy between x86, where PIC is off by default,
and the target, where PIC is always on.  If we can't get an agreement
among Linux targets, we should address it in x86 psABI.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 2:43 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Mon, Feb 22, 2016 at 10:28 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> On Mon, Feb 22, 2016 at 10:17 AM, Michael Matz <matz@suse.de> wrote:
>>> Hi,
>>>
>>> On Mon, 22 Feb 2016, H.J. Lu wrote:
>>>
>>>> > Typo?  What is the meaning of "dynamic relocation available at
>>>> > runtime". Are you talking about a relocation entry or the relocation
>>>> > process?
>>>>
>>>> Dynamic relocation isn't available with static linking.
>>>
>>> So you _are_ talking about the process.  I think it's customary to call
>>> such executables "dynamic", or perhaps non-static, isn't it?  (Why I'm
>>> confused with your wording: a dynamic executable which happens to have no
>>> dynamic reloc entries, has "no dynamic relocation available at runtime",
>>> and so we couldn't add one :))
>>>
>>>> > How about "When creating a dynamic executable the link editor should
>>>> > ..."?
>>>
>>> So, I still can understand my version better (possibly with
>>> s/dynamic/non-static/) :)
>>>
>>
>> Let's go with:
>>
>> When creating dynamic executable, the link editor should generate dynamic
>> relocations against unresolved weak symbols so that their values will be
>> resolved at run-time.
>>
>
> Second thought.  To generate dynamic relocation for R_X86_64_32
> against fun in:
>
> [hjl@gnu-6 pr19704]$ objdump -dwr foo.o
>
> foo.o:     file format elf64-x86-64
>
>
> Disassembly of section .text.startup:
>
> 0000000000000000 <main>:
>    0: b8 00 00 00 00       mov    $0x0,%eax 1: R_X86_64_32 fun
>    5: 48 85 c0             test   %rax,%rax
>    8: 74 10                 je     1a <main+0x1a>
>    a: 48 83 ec 08           sub    $0x8,%rsp
>    e: e8 00 00 00 00       callq  13 <main+0x13> f: R_X86_64_PC32 fun-0x4
>   13: 31 c0                 xor    %eax,%eax
>   15: 48 83 c4 08           add    $0x8,%rsp
>   19: c3                   retq
>   1a: 31 c0                 xor    %eax,%eax
>   1c: c3                   retq
> [hjl@gnu-6 pr19704]$
>
> we create executable with dynamic R_X86_64_32 relocation against
> fun .text section, which leads to DT_TEXTREL.  Is this what we want?
>

I believe the intention of undefined weak symbol is it should be resolved
at link-time when creating executable, dynamic or static.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Wed, Feb 24, 2016 at 7:30 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Wed, 24 Feb 2016, H.J. Lu wrote:
>
>> > Agreed. Anything that brings back weak symbols into the dynamic loader
>> > is going to be highly suspect to me, and require a lot of explaining.
>>
>> I was saying as far as ld was concerned, weak defined and non-weak
>> defined dynamic symbols would be treated equally at run-time.  Do you
>> agree with me?
>
> I at least don't.  The difference is that a defined weak symbol (at link
> edit time) might become undefined at runtime.  A defined non-weak symbol
> can't.  So they have to be handled differently.
>

How?

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 9:01 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Tue, 23 Feb 2016, H.J. Lu wrote:
>
>> >> Not only we need to change defined weak symbol behavior, we also need
>> >> to change undefined non-weak symbol behavior when taking its address.
>> >
>> > Why?  An undefined non-weak symbol reference leads to a linker error,
>> > done.  Why change that?
>>
>> Make it "external non-weak symbol", which is a symbol defined in a
>> shared object.
>
> What would you like to change in behaviour for them (and why must it be
> done at the same time as changing something for weak symbols)?  A
> reference to the address of such a function symbol from an executable
> resolves to the plt slot currently (which in turn is exported, so that
> address references from other modules resolve to the same).  A reference
> to a non-function symbol is loaded from the got.  In all cases must this
> symbol be defined somewhere (otherwise linker error), so I don't see what
> we need to change, nor why that would force DT_TEXTREL (or the compiler to
> emit PIC like code).
>

At run-time, there is no difference between weak defined and non-weak
defined symbols.  If we change defined weak symbol behavior, we also
need to change defined non-weak symbol behavior.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 10:11 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Mon, 22 Feb 2016, H.J. Lu wrote:
>
>> We should extend GNU ABI to:
>>
>> When creating executable, if dynamic relocation is available at
>> run-time,
>
> Typo?  What is the meaning of "dynamic relocation available at runtime".
> Are you talking about a relocation entry or the relocation process?

Dynamic relocation isn't available with static linking.

> How about "When creating a dynamic executable the link editor should ..."?
>
>> the link editor should generate dynamic relocations against
>> unresolved weak symbols so that their values will be resolved at
>> run-time.
>
>
> Ciao,
> Michael.



-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 8:49 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Tue, 23 Feb 2016, H.J. Lu wrote:
>
>> Not only we need to change defined weak symbol behavior, we also need to
>> change undefined non-weak symbol behavior when taking its address.
>
> Why?  An undefined non-weak symbol reference leads to a linker error,
> done.  Why change that?
>

Make it "external non-weak symbol", which is a symbol defined in
a shared object.



-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Alan Modra]

On Mon, Feb 22, 2016 at 02:43:55PM -0800, H.J. Lu wrote:
> we create executable with dynamic R_X86_64_32 relocation against
> fun .text section, which leads to DT_TEXTREL.  Is this what we want?

I think that avoiding DT_TEXTREL is historically why we haven't
supported weak symbols well in non-PIC executables.  Another closely
related issue is the old hack written into most ABIs that the address
of a function referenced in an executable and defined in a shared
library is the address of that function's PLT code in the executable.

Note that this affects both defined and undefined weak functions in
non-PIC executables.  Currently, if we find a definition for "fun" at
link time in a shared library, "&fun" is the PLT entry in the
executable and "if (&fun)" effectively becomes "if (1)".  So if you
link the executable with a shared library containing a definition of
"fun" but at run-time load a newer shared library *not* defining "fun",
your program will segfault.

Therefore, if the aim is to make PIC and non-PIC executables behave
the same you need to change defined weak symbol behaviour too.  Any
non-PIC executable that takes the address of a weak function in code
or rodata would become DT_TEXTREL.

However, that might be a bad idea.  Lots of C++ code uses weak symbols
for functions defined in header files, and other objects with vague
linkage.  These result in weak definitions in eg. libstdc++.so.  I'm
not sure how many executables take the address of such functions and
thus might become DT_TEXTREL.

-- 
Alan Modra
Australia Development Lab, IBM

Re: Specify how undefined weak symbol should be resolved in executable

[Szabolcs Nagy]

* Michael Matz <matz@suse.de> [2016-02-23 16:16:15 +0100]:
> On Tue, 23 Feb 2016, H.J. Lu wrote:
> > I checked cc1plus and found:
> > 
> > [hjl@gnu-6 5.3.1]$ readelf -sW cc1plus| grep WEAK | grep UND
> 
> Careful, that includes .symtab, not only .dynsym.  These are the .dynsyms:
> 
> >     33: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_signal@GLIBC_2.3.2 (3)
> >   2924: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_broadcast@GLIBC_2.3.2 (3)
> >   4861: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_create
> >   6330: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_getspecific
> >   7205: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_mutex_unlock@GLIBC_2.2.5 (2)
> >   7719: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_delete
> >   9118: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __pthread_key_create
> >  11985: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
> >  12269: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_once
> >  15201: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_mutex_lock@GLIBC_2.2.5 (2)
> >  15794: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_wait@GLIBC_2.3.2 (3)
> >  18312: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTable
> >  19108: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_setspecific
> >  19649: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
> >  19871: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
> >  20107: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_destroy@GLIBC_2.3.2 (3)
> 
> After here it's .symtab:
> 
> >  18570: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_signal@@GLIBC_2.3.2
> > 
> > Do you know how many of them lead to  DT_TEXTREL?
> 
> I think all those not going via the PLT, hence looking at the relocs for 
> those symbols involved, ignoring JUMP_SLOT relocs will give you that.  For 
> my cc1plus it's only this:
> 
> pthread_cancel
> 
> (of those that are undef-weak), and only one reference.
> 

libgcc/gthr-posix.h checks 'multi-threadedness' by looking
at a weakref (if it's zero then single-threaded execution
is assumed).

on glibc the weakref is __pthread_key_create nowadays,
but on other systems or older libgcc it's pthread_cancel.

so the address of this weakref is taken, the other pthread
weakrefs are called only if this weakref was non-zero.

i'm not sure how the proposed changes affect this use-case,
but this way of deciding single-threadedness is broken and
a bug that gcc should eventually fix.  (e.g. a static linked
multi-threaded c++ application may not have a pthread_cancel
symbol defined and thus libstdc++ may not use locks where
necessary or crash when pthread_cancel is defined but other
weakrefs are not).

Re: Specify how undefined weak symbol should be resolved in executable

[Alan Modra]

On Tue, Feb 23, 2016 at 05:18:16PM -0800, H.J. Lu wrote:
> On Tue, Feb 23, 2016 at 5:04 PM, Alan Modra <amodra@gmail.com> wrote:
> > On Tue, Feb 23, 2016 at 09:10:51AM -0800, H.J. Lu wrote:
> >> At run-time, there is no difference between weak defined and non-weak
> >> defined symbols.
> >
> > This is not true, and even if it was..
> >
> 
> Please do
> 
> # git grep dl_dynamic_weak
> 
> in glibc to see it for yourself.

That's exactly the code that shows ld.so can treat weak and strong
symbols differently!  (And yes, I know the default setting of
dl_dynamic_weak.)

Besides, the main point of my comment was that it is simply faulty
reasoning to claim that the linker needs to change because of some
ld.so behaviour.

-- 
Alan Modra
Australia Development Lab, IBM

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Wed, 24 Feb 2016, H.J. Lu wrote:

> > To not do that (i.e. resolve them always at link edit time) breaks 
> > various current uses of weak symbols.  We could do that of course, but 
> > I don't think that would be useful to users.
> 
> I suggest you raise this issue at
> 
> https://groups.google.com/forum/#!forum/generic-abi

You want to change existing behaviour, not me.  The Solaris and HP-UX guys 
already stated what they do, and are unlikely to want to change behaviour.  
I don't think we can change the gABI to be more explicit about weak 
symbols (not the least because Solaris and HP-UX already differ in the 
details), so I think we're left with discussing it here.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Tue, 23 Feb 2016, H.J. Lu wrote:

> >> Not only we need to change defined weak symbol behavior, we also need 
> >> to change undefined non-weak symbol behavior when taking its address.
> >
> > Why?  An undefined non-weak symbol reference leads to a linker error, 
> > done.  Why change that?
> 
> Make it "external non-weak symbol", which is a symbol defined in a 
> shared object.

What would you like to change in behaviour for them (and why must it be 
done at the same time as changing something for weak symbols)?  A 
reference to the address of such a function symbol from an executable 
resolves to the plt slot currently (which in turn is exported, so that 
address references from other modules resolve to the same).  A reference 
to a non-function symbol is loaded from the got.  In all cases must this 
symbol be defined somewhere (otherwise linker error), so I don't see what 
we need to change, nor why that would force DT_TEXTREL (or the compiler to 
emit PIC like code).


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Tue, 23 Feb 2016, Szabolcs Nagy wrote:

> libgcc/gthr-posix.h checks 'multi-threadedness' by looking
> at a weakref (if it's zero then single-threaded execution
> is assumed).
> 
> on glibc the weakref is __pthread_key_create nowadays,
> but on other systems or older libgcc it's pthread_cancel.

Yep, that's the one.

> so the address of this weakref is taken, the other pthread weakrefs are 
> called only if this weakref was non-zero.
> 
> i'm not sure how the proposed changes affect this use-case,

Some of the variants will make this use-case work also for non-PIC
executables, at the expense of one more unsharable .text page on some 
architectures (the one where this adress-checking takes place).

> but this way of deciding single-threadedness is broken and a bug that 
> gcc should eventually fix.  (e.g. a static linked multi-threaded c++ 
> application may not have a pthread_cancel symbol defined and thus 
> libstdc++ may not use locks where necessary or crash when pthread_cancel 
> is defined but other weakrefs are not).

Yep, that's one problem of our current behaviour with addresses of weak 
symbols on non-PIC.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 10:17 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Mon, 22 Feb 2016, H.J. Lu wrote:
>
>> > Typo?  What is the meaning of "dynamic relocation available at
>> > runtime". Are you talking about a relocation entry or the relocation
>> > process?
>>
>> Dynamic relocation isn't available with static linking.
>
> So you _are_ talking about the process.  I think it's customary to call
> such executables "dynamic", or perhaps non-static, isn't it?  (Why I'm
> confused with your wording: a dynamic executable which happens to have no
> dynamic reloc entries, has "no dynamic relocation available at runtime",
> and so we couldn't add one :))
>
>> > How about "When creating a dynamic executable the link editor should
>> > ..."?
>
> So, I still can understand my version better (possibly with
> s/dynamic/non-static/) :)
>

Let's go with:

When creating dynamic executable, the link editor should generate dynamic
relocations against unresolved weak symbols so that their values will be
resolved at run-time.


-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Mon, 22 Feb 2016, H.J. Lu wrote:

> We should extend GNU ABI to:
> 
> When creating executable, if dynamic relocation is available at 
> run-time,

Typo?  What is the meaning of "dynamic relocation available at runtime".  
Are you talking about a relocation entry or the relocation process?

How about "When creating a dynamic executable the link editor should ..."?

> the link editor should generate dynamic relocations against 
> unresolved weak symbols so that their values will be resolved at 
> run-time.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Mon, 22 Feb 2016, H.J. Lu wrote:

> > Typo?  What is the meaning of "dynamic relocation available at 
> > runtime". Are you talking about a relocation entry or the relocation 
> > process?
> 
> Dynamic relocation isn't available with static linking.

So you _are_ talking about the process.  I think it's customary to call 
such executables "dynamic", or perhaps non-static, isn't it?  (Why I'm 
confused with your wording: a dynamic executable which happens to have no 
dynamic reloc entries, has "no dynamic relocation available at runtime", 
and so we couldn't add one :))

> > How about "When creating a dynamic executable the link editor should 
> > ..."?

So, I still can understand my version better (possibly with 
s/dynamic/non-static/) :)

> >> the link editor should generate dynamic relocations against 
> >> unresolved weak symbols so that their values will be resolved at 
> >> run-time.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Mon, Feb 22, 2016 at 10:28 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Mon, Feb 22, 2016 at 10:17 AM, Michael Matz <matz@suse.de> wrote:
>> Hi,
>>
>> On Mon, 22 Feb 2016, H.J. Lu wrote:
>>
>>> > Typo?  What is the meaning of "dynamic relocation available at
>>> > runtime". Are you talking about a relocation entry or the relocation
>>> > process?
>>>
>>> Dynamic relocation isn't available with static linking.
>>
>> So you _are_ talking about the process.  I think it's customary to call
>> such executables "dynamic", or perhaps non-static, isn't it?  (Why I'm
>> confused with your wording: a dynamic executable which happens to have no
>> dynamic reloc entries, has "no dynamic relocation available at runtime",
>> and so we couldn't add one :))
>>
>>> > How about "When creating a dynamic executable the link editor should
>>> > ..."?
>>
>> So, I still can understand my version better (possibly with
>> s/dynamic/non-static/) :)
>>
>
> Let's go with:
>
> When creating dynamic executable, the link editor should generate dynamic
> relocations against unresolved weak symbols so that their values will be
> resolved at run-time.
>

Second thought.  To generate dynamic relocation for R_X86_64_32
against fun in:

[hjl@gnu-6 pr19704]$ objdump -dwr foo.o

foo.o:     file format elf64-x86-64


Disassembly of section .text.startup:

0000000000000000 <main>:
   0: b8 00 00 00 00       mov    $0x0,%eax 1: R_X86_64_32 fun
   5: 48 85 c0             test   %rax,%rax
   8: 74 10                 je     1a <main+0x1a>
   a: 48 83 ec 08           sub    $0x8,%rsp
   e: e8 00 00 00 00       callq  13 <main+0x13> f: R_X86_64_PC32 fun-0x4
  13: 31 c0                 xor    %eax,%eax
  15: 48 83 c4 08           add    $0x8,%rsp
  19: c3                   retq
  1a: 31 c0                 xor    %eax,%eax
  1c: c3                   retq
[hjl@gnu-6 pr19704]$

we create executable with dynamic R_X86_64_32 relocation against
fun .text section, which leads to DT_TEXTREL.  Is this what we want?

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Alan Modra]

On Tue, Feb 23, 2016 at 09:10:51AM -0800, H.J. Lu wrote:
> At run-time, there is no difference between weak defined and non-weak
> defined symbols.

This is not true, and even if it was..

>  If we change defined weak symbol behavior, we also
> need to change defined non-weak symbol behavior.

..it does not follow that run-time loader treatment of weak vs. strong
symbols should determine linker behaviour regarding the same.

-- 
Alan Modra
Australia Development Lab, IBM

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Wed, Feb 24, 2016 at 9:08 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Wed, 24 Feb 2016, H.J. Lu wrote:
>
>> >> I was saying as far as ld was concerned, weak defined and non-weak
>> >> defined dynamic symbols would be treated equally at run-time.  Do you
>> >> agree with me?
>> >
>> > I at least don't.  The difference is that a defined weak symbol (at
>> > link edit time) might become undefined at runtime.  A defined non-weak
>> > symbol can't.  So they have to be handled differently.
>> >
>>
>> How?
>
> Like right now (mostly)?  Resolution of weak symbols needs to be deferred
> to the dynamic linker, in particular they mustn't be resolved (to either
> zero or an address) at link edit time.  There's one case where weak
> symbols can be resolved early: if at link edit time the executable itself
> provides a definition of 'foo' then weak references can be directly
> resolved to that one (because at runtime of that exectuable there's no
> possibility to not have this symbol defined).
>
> To not do that (i.e. resolve them always at link edit time) breaks various
> current uses of weak symbols.  We could do that of course, but I don't
> think that would be useful to users.

I suggest you raise this issue at

https://groups.google.com/forum/#!forum/generic-abi

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Carlos O'Donell]

On 02/23/2016 08:56 PM, Alan Modra wrote:
> On Tue, Feb 23, 2016 at 05:18:16PM -0800, H.J. Lu wrote:
>> On Tue, Feb 23, 2016 at 5:04 PM, Alan Modra <amodra@gmail.com> wrote:
>>> On Tue, Feb 23, 2016 at 09:10:51AM -0800, H.J. Lu wrote:
>>>> At run-time, there is no difference between weak defined and non-weak
>>>> defined symbols.
>>>
>>> This is not true, and even if it was..
>>>
>>
>> Please do
>>
>> # git grep dl_dynamic_weak
>>
>> in glibc to see it for yourself.
> 
> That's exactly the code that shows ld.so can treat weak and strong
> symbols differently!  (And yes, I know the default setting of
> dl_dynamic_weak.)
> 
> Besides, the main point of my comment was that it is simply faulty
> reasoning to claim that the linker needs to change because of some
> ld.so behaviour.

Agreed. Anything that brings back weak symbols into the dynamic loader
is going to be highly suspect to me, and require a lot of explaining.

Cheers,
Carlos.
 

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Wed, 24 Feb 2016, H.J. Lu wrote:

> >> I was saying as far as ld was concerned, weak defined and non-weak 
> >> defined dynamic symbols would be treated equally at run-time.  Do you 
> >> agree with me?
> >
> > I at least don't.  The difference is that a defined weak symbol (at 
> > link edit time) might become undefined at runtime.  A defined non-weak 
> > symbol can't.  So they have to be handled differently.
> >
> 
> How?

Like right now (mostly)?  Resolution of weak symbols needs to be deferred 
to the dynamic linker, in particular they mustn't be resolved (to either 
zero or an address) at link edit time.  There's one case where weak 
symbols can be resolved early: if at link edit time the executable itself 
provides a definition of 'foo' then weak references can be directly 
resolved to that one (because at runtime of that exectuable there's no 
possibility to not have this symbol defined).

To not do that (i.e. resolve them always at link edit time) breaks various 
current uses of weak symbols.  We could do that of course, but I don't 
think that would be useful to users.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Wed, Feb 24, 2016 at 9:54 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Wed, 24 Feb 2016, H.J. Lu wrote:
>
>> > To not do that (i.e. resolve them always at link edit time) breaks
>> > various current uses of weak symbols.  We could do that of course, but
>> > I don't think that would be useful to users.
>>
>> I suggest you raise this issue at
>>
>> https://groups.google.com/forum/#!forum/generic-abi
>
> You want to change existing behaviour, not me.  The Solaris and HP-UX guys
> already stated what they do, and are unlikely to want to change behaviour.
> I don't think we can change the gABI to be more explicit about weak
> symbols (not the least because Solaris and HP-UX already differ in the
> details), so I think we're left with discussing it here.
>

There is a difference between yours and mine.  I wanted to
clarify reference to undefined weak symbols in executable.
You want to change reference to defined weak symbols in
executable.  It is undefined weak vs defined weak.


-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 5:55 AM, Michael Matz <matz@suse.de> wrote:
> Hi,
>
> On Tue, 23 Feb 2016, H.J. Lu wrote:
>
>> On Mon, Feb 22, 2016 at 8:40 PM, Alan Modra <amodra@gmail.com> wrote:
>> >
>> > However, that might be a bad idea.  Lots of C++ code uses weak symbols
>> > for functions defined in header files, and other objects with vague
>> > linkage.  These result in weak definitions in eg. libstdc++.so.  I'm
>> > not sure how many executables take the address of such functions and
>> > thus might become DT_TEXTREL.
>>
>> Most, if not all, of programs will have DT_TEXTREL on x86 if undefined
>> weak symbols require dynamic relocations.
>
> Hmm, that's less than ideal of course.  Well, if the goal is to make PIC
> and non-PIC the same, we could also go the opposite way: make PIC behave
> like non-PIC, i.e. resolve weak symbols always at link editing time.  That
> of course would remove features (can't change libs at runtime anymore, if
> they change in definedness of such a symbol).

This "feature" never worked on x86 for non-PIC input.

> Or, a third variant: change the compiler to emit PIC like code for taking
> addresses of weak symbols also in generally non-PIC code, so we could
> avoid TEXTREL.
>
> I think the ideal solution would be that last one, change link editor now
> to behave like with PIC code, and eventually fix the compiler to not have
> to generate TEXTREL.

If we change ld now, all of a sudden, x86 binaris are marked with
DT_TEXTREL.  What we can do is to add a new linker command
option, -z dynamic-undef-weak, and GCC passes it to ld only when
compiler is changed to generate PIC-like code for address of all
undefined symbol in text section.

> Note that the existence of DT_TEXTREL itself isn't that bad: only those
> pages that actually contain relocations will be unshared, so for the
> example of crtbegin.o it's only one page per process.  In addition
> crtbegin could of course always be PIC code, avoiding the issue.
>
> I've looked at a normal c++ program (zypper) and the only weak undef
> symbols are those from crtbegin.  There are many other weak symbols, but
> they are defined in the executable itself (it's mostly template
> instantiations), so pose no problem.
>
>

I checked cc1plus and found:

[hjl@gnu-6 5.3.1]$ readelf -sW cc1plus| grep WEAK | grep UND
    33: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_signal@GLIBC_2.3.2 (3)
  2924: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_broadcast@GLIBC_2.3.2 (3)
  4861: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_create
  6330: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_getspecific
  7205: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_mutex_unlock@GLIBC_2.2.5 (2)
  7719: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_delete
  9118: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __pthread_key_create
 11985: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
 12269: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_once
 15201: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_mutex_lock@GLIBC_2.2.5 (2)
 15794: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_wait@GLIBC_2.3.2 (3)
 18312: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_deregisterTMCloneTable
 19108: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_setspecific
 19649: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
 19871: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_registerTMCloneTable
 20107: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_destroy@GLIBC_2.3.2 (3)
 18570: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_signal@@GLIBC_2.3.2
 21461: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_broadcast@@GLIBC_2.3.2
 23398: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_create
 24867: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_getspecific
 25742: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_mutex_unlock@@GLIBC_2.2.5
 26256: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_delete
 27655: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __pthread_key_create
 30521: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
 30805: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_once
 33737: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_mutex_lock@@GLIBC_2.2.5
 34330: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_wait@@GLIBC_2.3.2
 36848: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_deregisterTMCloneTable
 37644: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_setspecific
 38185: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
 38407: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND
_ITM_registerTMCloneTable
 38643: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND
pthread_cond_destroy@@GLIBC_2.3.2
[hjl@gnu-6 5.3.1]$

Do you know how many of them lead to  DT_TEXTREL?

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 6:04 PM, Carlos O'Donell <carlos@redhat.com> wrote:
> On 02/23/2016 08:56 PM, Alan Modra wrote:
>> On Tue, Feb 23, 2016 at 05:18:16PM -0800, H.J. Lu wrote:
>>> On Tue, Feb 23, 2016 at 5:04 PM, Alan Modra <amodra@gmail.com> wrote:
>>>> On Tue, Feb 23, 2016 at 09:10:51AM -0800, H.J. Lu wrote:
>>>>> At run-time, there is no difference between weak defined and non-weak
>>>>> defined symbols.
>>>>
>>>> This is not true, and even if it was..
>>>>
>>>
>>> Please do
>>>
>>> # git grep dl_dynamic_weak
>>>
>>> in glibc to see it for yourself.
>>
>> That's exactly the code that shows ld.so can treat weak and strong
>> symbols differently!  (And yes, I know the default setting of
>> dl_dynamic_weak.)
>>
>> Besides, the main point of my comment was that it is simply faulty
>> reasoning to claim that the linker needs to change because of some
>> ld.so behaviour.
>
> Agreed. Anything that brings back weak symbols into the dynamic loader
> is going to be highly suspect to me, and require a lot of explaining.
>

I was saying as far as ld was concerned, weak defined and non-weak
defined dynamic symbols would be treated equally at run-time.  Do you
agree with me?


-- 
H.J.

Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

Currently when building executable, undefined weak symbol is resolved
differently, depending on if the relocatable input is compiled with PIC or
not:

hjl@gnu-tools-1 weak-1]$ cat x.c
extern void __attribute__ ((weak)) fun (void);
int main (void)
{
  if (&fun != 0)
    fun ();
  return 0;
}
[hjl@gnu-tools-1 weak-1]$ make
gcc -B./ -O2 -o x x.c
gcc -B./ -O2 -o y x.c -fPIC
readelf -r x

Relocation section '.rela.dyn' at offset 0x328 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000600870  000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x340 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000600890  000100000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0

There is no dynamic relocation against fun, whose value is
resolved to zero at link-time.

readelf -r y

Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000006008a0  000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
0000006008a8  000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000006008c8  000200000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0
[hjl@gnu-tools-1 weak-1]$

There is dynamic relocation against fun, whose value is resolved
at runt-time. We should make it consistent, regardless if input
relocatable file is compiled with PIC or not.

I raised the issue at gABI:

https://groups.google.com/d/msg/generic-abi/Ckq19PfLxyk/eCbXPofkAQAJ

But if gABI change is incompatible with other OSes, we need to
specify it in GNU ABI.

-- 
H.J.

Re: Specify how undefined weak symbol should be resolved in executable

[Michael Matz]

Hi,

On Tue, 23 Feb 2016, H.J. Lu wrote:

> > Hmm, that's less than ideal of course.  Well, if the goal is to make 
> > PIC and non-PIC the same, we could also go the opposite way: make PIC 
> > behave like non-PIC, i.e. resolve weak symbols always at link editing 
> > time.  That of course would remove features (can't change libs at 
> > runtime anymore, if they change in definedness of such a symbol).
> 
> This "feature" never worked on x86 for non-PIC input.

True (and that's exactly the difference between PIC and non-PIC).

> If we change ld now, all of a sudden, x86 binaris are marked with 
> DT_TEXTREL.  What we can do is to add a new linker command option, -z 
> dynamic-undef-weak, and GCC passes it to ld only when compiler is 
> changed to generate PIC-like code for address of all undefined symbol in 
> text section.

Hmmnyeah, possible.

> > Note that the existence of DT_TEXTREL itself isn't that bad: only those
> > pages that actually contain relocations will be unshared, so for the
> > example of crtbegin.o it's only one page per process.  In addition
> > crtbegin could of course always be PIC code, avoiding the issue.
> >
> > I've looked at a normal c++ program (zypper) and the only weak undef
> > symbols are those from crtbegin.  There are many other weak symbols, but
> > they are defined in the executable itself (it's mostly template
> > instantiations), so pose no problem.
> >
> >
> 
> I checked cc1plus and found:
> 
> [hjl@gnu-6 5.3.1]$ readelf -sW cc1plus| grep WEAK | grep UND

Careful, that includes .symtab, not only .dynsym.  These are the .dynsyms:

>     33: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_signal@GLIBC_2.3.2 (3)
>   2924: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_broadcast@GLIBC_2.3.2 (3)
>   4861: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_create
>   6330: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_getspecific
>   7205: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_mutex_unlock@GLIBC_2.2.5 (2)
>   7719: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_key_delete
>   9118: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __pthread_key_create
>  11985: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
>  12269: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_once
>  15201: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_mutex_lock@GLIBC_2.2.5 (2)
>  15794: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_wait@GLIBC_2.3.2 (3)
>  18312: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTable
>  19108: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND pthread_setspecific
>  19649: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
>  19871: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
>  20107: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_destroy@GLIBC_2.3.2 (3)

After here it's .symtab:

>  18570: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND pthread_cond_signal@@GLIBC_2.3.2
> 
> Do you know how many of them lead to  DT_TEXTREL?

I think all those not going via the PLT, hence looking at the relocs for 
those symbols involved, ignoring JUMP_SLOT relocs will give you that.  For 
my cc1plus it's only this:

pthread_cancel

(of those that are undef-weak), and only one reference.


Ciao,
Michael.

Re: Specify how undefined weak symbol should be resolved in executable

[H.J. Lu]

On Tue, Feb 23, 2016 at 5:04 PM, Alan Modra <amodra@gmail.com> wrote:
> On Tue, Feb 23, 2016 at 09:10:51AM -0800, H.J. Lu wrote:
>> At run-time, there is no difference between weak defined and non-weak
>> defined symbols.
>
> This is not true, and even if it was..
>

Please do

# git grep dl_dynamic_weak

in glibc to see it for yourself.

-- 
H.J.