Avoid privatization of TLS variables

Avoid privatization of TLS variables

[Jan Hubicka]

Hi,
libreoffice fails to build with TLS because of "Cannot load any more object
with static TLS". Iant pointed out to me the difference that the initial exec
TLS model is also used by static TLS variables.

This patch prevents turning TLS variables into static that lets me to finish
libreoffice build.

Can the conditional be strenghtened somewhat? decl_default_tls_model has:

  if (!flag_shlib)
    {
      if (is_local)
        kind = TLS_MODEL_LOCAL_EXEC;
      else
        kind = TLS_MODEL_INITIAL_EXEC;
    }

  /* Local dynamic is inefficient when we're not combining the
     parts of the address.  */
  else if (optimize && is_local)
    kind = TLS_MODEL_LOCAL_DYNAMIC;
  else
    kind = TLS_MODEL_GLOBAL_DYNAMIC;

perhaps we should have fake TLS_MODEL_EXEC and TLS_MODE_DYNAMIC modes
that get more specified later once the visibility is finalized instead of
deciding it at compile time?

Bootstrapped/regtested x86_64-linux, comitted for now until we get better
solution (if it exists).

Honza

Index: ChangeLog
===================================================================
--- ChangeLog	(revision 215416)
+++ ChangeLog	(working copy)
@@ -1,5 +1,10 @@
 2014-09-19  Jan Hubicka  <hubicka@ucw.cz>
 
+	* ipa-visibility.c (varpool_node::externally_visible_p): Do not
+	privatize dynamic TLS variables.
+
+2014-09-19  Jan Hubicka  <hubicka@ucw.cz>
+
 	* diagnostic.c (warning_n): New function.
 	* diagnostic-core.h (warning_n): Declare.
 	* ipa-devirt.c (ipa_devirt): Handle singulars correctly;
Index: ipa-visibility.c
===================================================================
--- ipa-visibility.c	(revision 215415)
+++ ipa-visibility.c	(working copy)
@@ -277,6 +277,13 @@ varpool_node::externally_visible_p (void
   if (used_from_object_file_p ())
     return true;
 
+  /* Bringing TLS variables local may cause dynamic linker failures
+     on limits of static TLS vars.  */
+  if (DECL_THREAD_LOCAL_P (decl)
+      && (DECL_TLS_MODEL (decl) != TLS_MODEL_EMULATED
+	  && DECL_TLS_MODEL (decl) != TLS_MODEL_INITIAL_EXEC))
+    return true;
+
   if (DECL_HARD_REGISTER (decl))
     return true;
   if (DECL_PRESERVE_P (decl))

Re: Avoid privatization of TLS variables

[Markus Trippelsdorf]

On 2014.09.20 at 05:16 +0200, Jan Hubicka wrote:
> Hi,
> libreoffice fails to build with TLS because of "Cannot load any more object
> with static TLS". Iant pointed out to me the difference that the initial exec
> TLS model is also used by static TLS variables.
> 
> This patch prevents turning TLS variables into static that lets me to finish
> libreoffice build.

This patch triggers an assert in gold:

~ % echo "int main () {}" | g++ -flto -fprofile-generate -x c++ -
ld: internal error in relocate_tls, at/gold/x86_64.cc:3829
collect2: error: ld returned 1 exit status

ld.bfd works fine.

-- 
Markus

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> On 2014.09.20 at 05:16 +0200, Jan Hubicka wrote:
> > Hi,
> > libreoffice fails to build with TLS because of "Cannot load any more object
> > with static TLS". Iant pointed out to me the difference that the initial exec
> > TLS model is also used by static TLS variables.
> > 
> > This patch prevents turning TLS variables into static that lets me to finish
> > libreoffice build.
> 
> This patch triggers an assert in gold:
> 
> ~ % echo "int main () {}" | g++ -flto -fprofile-generate -x c++ -
> ld: internal error in relocate_tls, at/gold/x86_64.cc:3829
> collect2: error: ld returned 1 exit status

I saw this error, too, just not with such a simple testcase :)
Ian, I think this is bug in gold. What does the assert mean?

Honza
> 
> ld.bfd works fine.
> 
> -- 
> Markus

Re: Avoid privatization of TLS variables

[Markus Trippelsdorf]

On 2014.09.24 at 20:18 +0200, Jan Hubicka wrote:
> > On 2014.09.20 at 05:16 +0200, Jan Hubicka wrote:
> > > Hi,
> > > libreoffice fails to build with TLS because of "Cannot load any more object
> > > with static TLS". Iant pointed out to me the difference that the initial exec
> > > TLS model is also used by static TLS variables.
> > > 
> > > This patch prevents turning TLS variables into static that lets me to finish
> > > libreoffice build.
> > 
> > This patch triggers an assert in gold:
> > 
> > ~ % echo "int main () {}" | g++ -flto -fprofile-generate -x c++ -
> > ld: internal error in relocate_tls, at/gold/x86_64.cc:3829
> > collect2: error: ld returned 1 exit status
> 
> I saw this error, too, just not with such a simple testcase :)
> Ian, I think this is bug in gold. What does the assert mean?

I've opened: https://sourceware.org/bugzilla/show_bug.cgi?id=17432

-- 
Markus

Re: Avoid privatization of TLS variables

[Ian Lance Taylor]

On Wed, Sep 24, 2014 at 11:18 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> On 2014.09.20 at 05:16 +0200, Jan Hubicka wrote:
>> > Hi,
>> > libreoffice fails to build with TLS because of "Cannot load any more object
>> > with static TLS". Iant pointed out to me the difference that the initial exec
>> > TLS model is also used by static TLS variables.
>> >
>> > This patch prevents turning TLS variables into static that lets me to finish
>> > libreoffice build.
>>
>> This patch triggers an assert in gold:
>>
>> ~ % echo "int main () {}" | g++ -flto -fprofile-generate -x c++ -
>> ld: internal error in relocate_tls, at/gold/x86_64.cc:3829
>> collect2: error: ld returned 1 exit status
>
> I saw this error, too, just not with such a simple testcase :)
> Ian, I think this is bug in gold. What does the assert mean?

I'm not sure because my line numbers don't match.  If it is this
assert:

    case elfcpp::R_X86_64_TPOFF32:          // Local-exec
      if (tls_segment == NULL)
	{
	  gold_assert(parameters->errors()->error_count() > 0
		      || issue_undefined_symbol_error(gsym));
	  return;
	}
      value -= tls_segment->memsz();
      Relocate_functions<size, false>::rela32(view, value, addend);
      break;

then the problem is that there is a TPOFF32 relocation but none of the
input files has a section with the SHF_TLS flag set.  This should not
be possible--the TPOFF32 relocation implies a TLS variable, but the
absence of any SHF_TLS sections implies that there are no TLS
variables.

Why would there be a TPOFF32 relocation in the simple test case from
Markus?

If that is not the assertion, let me know what is.

Ian

Re: Avoid privatization of TLS variables

[Jan Hubicka]

[-- Attachment #1: Type: text/plain, Size: 4670 bytes --]

> 
> I'm not sure because my line numbers don't match.  If it is this
> assert:
> 
>     case elfcpp::R_X86_64_TPOFF32:          // Local-exec
>       if (tls_segment == NULL)
> 	{
> 	  gold_assert(parameters->errors()->error_count() > 0
> 		      || issue_undefined_symbol_error(gsym));
> 	  return;
> 	}
>       value -= tls_segment->memsz();
>       Relocate_functions<size, false>::rela32(view, value, addend);
>       break;

I looked it up and it was the place I got ICE, too.
> 
> then the problem is that there is a TPOFF32 relocation but none of the
> input files has a section with the SHF_TLS flag set.  This should not
> be possible--the TPOFF32 relocation implies a TLS variable, but the
> absence of any SHF_TLS sections implies that there are no TLS
> variables.
I get:
a.o:     file format elf64-x86-64


Disassembly of section .text:

0000000000000000 <main>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   64 48 8b 04 25 00 00    mov    %fs:0x0,%rax
   b:   00 00 
                        9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
   d:   64 48 8b 14 25 00 00    mov    %fs:0x0,%rdx
  14:   00 00 
                        12: R_X86_64_TPOFF32    __gcov_indirect_call_callee_ltopriv
  16:   48 89 d1                mov    %rdx,%rcx
  19:   ba 00 00 00 00          mov    $0x0,%edx
                        1a: R_X86_64_32 main
  1e:   be eb 72 70 06          mov    $0x67072eb,%esi
  23:   48 89 c7                mov    %rax,%rdi
  26:   e8 00 00 00 00          callq  2b <main+0x2b>
                        27: R_X86_64_PC32       __gcov_indirect_call_profiler+0xfffffffffffffffc
  2b:   64 48 c7 04 25 00 00    movq   $0x0,%fs:0x0
  32:   00 00 00 00 00 00 
                        30: R_X86_64_TPOFF32    __gcov_indirect_call_callee_ltopriv
  38:   bf 00 00 00 00          mov    $0x0,%edi
                        39: R_X86_64_32 .bss+0x8
  3d:   e8 00 00 00 00          callq  42 <main+0x42>
                        3e: R_X86_64_PC32       __gcov_time_profiler+0xfffffffffffffffc
  42:   ba 00 00 00 00          mov    $0x0,%edx
  47:   48 8b 05 00 00 00 00    mov    0x0(%rip),%rax        # 4e <main+0x4e>
                        4a: R_X86_64_PC32       .bss+0xfffffffffffffffc
  4e:   48 83 c0 01             add    $0x1,%rax
  52:   48 89 05 00 00 00 00    mov    %rax,0x0(%rip)        # 59 <main+0x59>
                        55: R_X86_64_PC32       .bss+0xfffffffffffffffc
  59:   89 d0                   mov    %edx,%eax
  5b:   5d                      pop    %rbp
  5c:   c3                      retq   

000000000000005d <_GLOBAL__sub_I_65535_0_main>:
  5d:   55                      push   %rbp
  5e:   48 89 e5                mov    %rsp,%rbp
  61:   bf 00 00 00 00          mov    $0x0,%edi
                        62: R_X86_64_32 .data+0x40
  66:   e8 00 00 00 00          callq  6b <_GLOBAL__sub_I_65535_0_main+0xe>
                        67: R_X86_64_PC32       __gcov_init+0xfffffffffffffffc
  6b:   5d                      pop    %rbp
  6c:   c3                      retq   


and in asm:
        .file   "ccstYwxO.ltrans0.o"
        .text
        .globl  main
        .type   main, @function
main:
.LFB0: 
        .cfi_startproc
        pushq   %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset 6, -16
        movq    %rsp, %rbp
        .cfi_def_cfa_register 6
        movq    %fs:__gcov_indirect_call_counters_ltopriv@tpoff, %rax
        movq    %fs:__gcov_indirect_call_callee_ltopriv@tpoff, %rdx
        movq    %rdx, %rcx
        movl    $main, %edx
        movl    $108032747, %esi
        movq    %rax, %rdi
        call    __gcov_indirect_call_profiler
        movq    $0, %fs:__gcov_indirect_call_callee_ltopriv@tpoff
        movl    $__gcov8.main, %edi
        call    __gcov_time_profiler
        movl    $0, %edx
        movq    __gcov0.main(%rip), %rax
        addq    $1, %rax
        movq    %rax, __gcov0.main(%rip)
        movl    %edx, %eax
        popq    %rbp
        .cfi_def_cfa 7, 8
        ret
        .cfi_endproc
.LFE0: 
        .size   main, .-main
        .hidden __gcov_indirect_call_callee_ltopriv
        .tls_common     __gcov_indirect_call_callee_ltopriv,8,8
        .hidden __gcov_indirect_call_counters_ltopriv
        .tls_common     __gcov_indirect_call_counters_ltopriv,8,8
        .local  __gcov0.main
        .comm   __gcov0.main,8,8
        .local  __gcov8.main
        .comm   __gcov8.main,8,8
        .type   _GLOBAL__sub_I_65535_0_main, @function
...


Honza
> 
> Why would there be a TPOFF32 relocation in the simple test case from
> Markus?
> 
> If that is not the assertion, let me know what is.
> 
> Ian

[-- Attachment #2: a.o --]
[-- Type: application/x-object, Size: 3064 bytes --]

Re: Avoid privatization of TLS variables

[Ian Lance Taylor]

On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>
>    b:   00 00
>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv

Look at the .o file where __gcov_indirect_call_counters_ltopriv is
defined.  That .o file must have the symbol marked as STT_TLS and it
must be defined in a section with the SHF_TLS flag.  If that is not
true, then that is your problem.

Ian

Re: Avoid privatization of TLS variables

[H.J. Lu]

On Thu, Sep 25, 2014 at 8:24 AM, Ian Lance Taylor <iant@google.com> wrote:
> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>>
>>    b:   00 00
>>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
>
> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
> defined.  That .o file must have the symbol marked as STT_TLS and it
> must be defined in a section with the SHF_TLS flag.  If that is not
> true, then that is your problem.

SHF_TLS isn't required.

    16: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
__gcov_indirect_call_counters_ltopriv
    17: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
__gcov_indirect_call_callee_ltopriv

are also sufficient.

-- 
H.J.

Re: Avoid privatization of TLS variables

[Ian Lance Taylor]

On Thu, Sep 25, 2014 at 8:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Thu, Sep 25, 2014 at 8:24 AM, Ian Lance Taylor <iant@google.com> wrote:
>> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>>>
>>>    b:   00 00
>>>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
>>
>> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
>> defined.  That .o file must have the symbol marked as STT_TLS and it
>> must be defined in a section with the SHF_TLS flag.  If that is not
>> true, then that is your problem.
>
> SHF_TLS isn't required.
>
>     16: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> __gcov_indirect_call_counters_ltopriv
>     17: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> __gcov_indirect_call_callee_ltopriv
>
> are also sufficient.

I can create a .o file with a hidden common symbol, but I can't
recreate the problem.  When I try, gold creates a TLS section and TLS
segment itself.

How exactly is gold being invoked?

Ian

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> On Thu, Sep 25, 2014 at 8:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> > On Thu, Sep 25, 2014 at 8:24 AM, Ian Lance Taylor <iant@google.com> wrote:
> >> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
> >>>
> >>>    b:   00 00
> >>>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
> >>
> >> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
> >> defined.  That .o file must have the symbol marked as STT_TLS and it
> >> must be defined in a section with the SHF_TLS flag.  If that is not
> >> true, then that is your problem.
> >
> > SHF_TLS isn't required.
> >
> >     16: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> > __gcov_indirect_call_counters_ltopriv
> >     17: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> > __gcov_indirect_call_callee_ltopriv
> >
> > are also sufficient.
> 
> I can create a .o file with a hidden common symbol, but I can't
> recreate the problem.  When I try, gold creates a TLS section and TLS
> segment itself.
> 
> How exactly is gold being invoked?

It seems to happen with LTO compilation only, just build mainline tree
and try the original testcase.

Honza
> 
> Ian

Re: Avoid privatization of TLS variables

[Cary Coutant]

The plugin API doesn't have a way to mark a symbol as TLS, but it
doesn't really matter since the linker simply overrides the
placeholder from the claimed file with the symbol provided in the
replacement. Unfortunately, I excluded common symbols from this logic
in gold, so the symbol isn't getting marked as TLS COMMON even when we
see the real ELF file. I just need to remind myself why the exclusion
is there and figure out how to fix it. I think all the information we
need is there, so no changes to the plugin API should be necessary.

-cary


On Thu, Sep 25, 2014 at 9:29 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> On Thu, Sep 25, 2014 at 8:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
>> > On Thu, Sep 25, 2014 at 8:24 AM, Ian Lance Taylor <iant@google.com> wrote:
>> >> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
>> >>>
>> >>>    b:   00 00
>> >>>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
>> >>
>> >> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
>> >> defined.  That .o file must have the symbol marked as STT_TLS and it
>> >> must be defined in a section with the SHF_TLS flag.  If that is not
>> >> true, then that is your problem.
>> >
>> > SHF_TLS isn't required.
>> >
>> >     16: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
>> > __gcov_indirect_call_counters_ltopriv
>> >     17: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
>> > __gcov_indirect_call_callee_ltopriv
>> >
>> > are also sufficient.
>>
>> I can create a .o file with a hidden common symbol, but I can't
>> recreate the problem.  When I try, gold creates a TLS section and TLS
>> segment itself.
>>
>> How exactly is gold being invoked?
>
> It seems to happen with LTO compilation only, just build mainline tree
> and try the original testcase.
>
> Honza
>>
>> Ian

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> The plugin API doesn't have a way to mark a symbol as TLS, but it
> doesn't really matter since the linker simply overrides the
> placeholder from the claimed file with the symbol provided in the
> replacement. Unfortunately, I excluded common symbols from this logic
> in gold, so the symbol isn't getting marked as TLS COMMON even when we
> see the real ELF file. I just need to remind myself why the exclusion
> is there and figure out how to fix it. I think all the information we
> need is there, so no changes to the plugin API should be necessary.

Thank you! Now when I have your attention, perhaps we could discuss the original
motivation of the change that exposed this bug.
I was building libreoffice with profile feedback and I run into a message

cannot load any more object with static TLS

that took me a while to track as I did not see where static TLS is comming out.
Ian pointed out to me that static variables with TLS storage also consume
static TLS even if they are in dynamic model.  This is why I disabled
localization.  Is there better way to handle this?

Note that the variable __gcov_indirect_call_counters_ltopriv was added to work around
https://sourceware.org/bugzilla/show_bug.cgi?id=14342
that seems to be fixed. I would like to drop the hack (that will also make
profiling to work with current golds again), but I think I would like to document
when the bug went away, becuase it is only bit over a year now.

Any idea when it was fixed?

Honza

> 
> -cary
> 
> 
> On Thu, Sep 25, 2014 at 9:29 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
> >> On Thu, Sep 25, 2014 at 8:37 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> >> > On Thu, Sep 25, 2014 at 8:24 AM, Ian Lance Taylor <iant@google.com> wrote:
> >> >> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
> >> >>>
> >> >>>    b:   00 00
> >> >>>                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
> >> >>
> >> >> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
> >> >> defined.  That .o file must have the symbol marked as STT_TLS and it
> >> >> must be defined in a section with the SHF_TLS flag.  If that is not
> >> >> true, then that is your problem.
> >> >
> >> > SHF_TLS isn't required.
> >> >
> >> >     16: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> >> > __gcov_indirect_call_counters_ltopriv
> >> >     17: 0000000000000008     8 TLS     GLOBAL HIDDEN   COM
> >> > __gcov_indirect_call_callee_ltopriv
> >> >
> >> > are also sufficient.
> >>
> >> I can create a .o file with a hidden common symbol, but I can't
> >> recreate the problem.  When I try, gold creates a TLS section and TLS
> >> segment itself.
> >>
> >> How exactly is gold being invoked?
> >
> > It seems to happen with LTO compilation only, just build mainline tree
> > and try the original testcase.
> >
> > Honza
> >>
> >> Ian

Re: Avoid privatization of TLS variables

[Cary Coutant]

> Thank you! Now when I have your attention, perhaps we could discuss the original
> motivation of the change that exposed this bug.
> I was building libreoffice with profile feedback and I run into a message
>
> cannot load any more object with static TLS
>
> that took me a while to track as I did not see where static TLS is comming out.
> Ian pointed out to me that static variables with TLS storage also consume
> static TLS even if they are in dynamic model.  This is why I disabled
> localization.  Is there better way to handle this?

As I understand it, if you compile with -fpic, you shouldn't see any
static TLS. The compiler should only use the Local Exec model for
static/hidden variables in non-PIC compiles.

> Note that the variable __gcov_indirect_call_counters_ltopriv was added to work around
> https://sourceware.org/bugzilla/show_bug.cgi?id=14342
> that seems to be fixed. I would like to drop the hack (that will also make
> profiling to work with current golds again), but I think I would like to document
> when the bug went away, becuase it is only bit over a year now.
>
> Any idea when it was fixed?

I think it was fixed with this patch:

   https://sourceware.org/ml/binutils/2013-06/msg00139.html

I guess I should close that bug.

-cary

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> > Thank you! Now when I have your attention, perhaps we could discuss the original
> > motivation of the change that exposed this bug.
> > I was building libreoffice with profile feedback and I run into a message
> >
> > cannot load any more object with static TLS
> >
> > that took me a while to track as I did not see where static TLS is comming out.
> > Ian pointed out to me that static variables with TLS storage also consume
> > static TLS even if they are in dynamic model.  This is why I disabled
> > localization.  Is there better way to handle this?
> 
> As I understand it, if you compile with -fpic, you shouldn't see any
> static TLS. The compiler should only use the Local Exec model for
> static/hidden variables in non-PIC compiles.

Yep, what seems to be happening is that I have TLS_MODEL_GLOBAL_DYNAMIC variable
and I bring it static (or hidden) and it will not have TLS_MODEL_LOCAL_DYNAMIC.
In that case it seems that static TLS slots are occupied.
> 
> > Note that the variable __gcov_indirect_call_counters_ltopriv was added to work around
> > https://sourceware.org/bugzilla/show_bug.cgi?id=14342
> > that seems to be fixed. I would like to drop the hack (that will also make
> > profiling to work with current golds again), but I think I would like to document
> > when the bug went away, becuase it is only bit over a year now.
> >
> > Any idea when it was fixed?
> 
> I think it was fixed with this patch:
> 
>    https://sourceware.org/ml/binutils/2013-06/msg00139.html
> 
> I guess I should close that bug.

What is the earlierst binutils release fixing the bug?   I will add it into
changes.html for 5.0.

Honza
> 
> -cary

Re: Avoid privatization of TLS variables

[Markus Trippelsdorf]

On 2014.09.26 at 04:50 +0200, Jan Hubicka wrote:
> What is the earlierst binutils release fixing the bug?   I will add it into
> changes.html for 5.0.

binutils-2.24

-- 
Markus

Re: Avoid privatization of TLS variables

[Alan Modra]

On Fri, Sep 26, 2014 at 04:17:14AM +0200, Jan Hubicka wrote:
> I was building libreoffice with profile feedback and I run into a message
> 
> cannot load any more object with static TLS
> 
> that took me a while to track as I did not see where static TLS is comming out.
> Ian pointed out to me that static variables with TLS storage also consume
> static TLS even if they are in dynamic model.  This is why I disabled
> localization.  Is there better way to handle this?

Fix a glibc bug?  It has been a while since I looked into glibc in
any depth regarding TLS (2011-03), but I believe the l_tls_modid test
here
	  if (! RTLD_SINGLE_THREAD_P && imap->l_tls_modid > DTV_SURPLUS)
	    _dl_signal_error (0, "dlopen", NULL, N_("\
cannot load any more object with static TLS"));

is wrong.  The test is saying "if we have loaded a certain number of
dynamic objects with TLS segments, refuse to dlopen any more
containing TLS if we are multi-threaded".

What it should be saying is "if we have loaded a certain number of
dynamic objects with TLS segments *after we went multi-threaded*,
refuse to open any more".  In particular, any dynamic objects with TLS
segments loaded at program startup should not be counted.  This is
because DTV_SURPLUS *extra* slots are allocated above those needed at
program startup.  At least, that's how I think it works.

-- 
Alan Modra
Australia Development Lab, IBM

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> On Fri, Sep 26, 2014 at 04:17:14AM +0200, Jan Hubicka wrote:
> > I was building libreoffice with profile feedback and I run into a message
> > 
> > cannot load any more object with static TLS
> > 
> > that took me a while to track as I did not see where static TLS is comming out.
> > Ian pointed out to me that static variables with TLS storage also consume
> > static TLS even if they are in dynamic model.  This is why I disabled
> > localization.  Is there better way to handle this?
> 
> Fix a glibc bug?  It has been a while since I looked into glibc in
> any depth regarding TLS (2011-03), but I believe the l_tls_modid test
> here
> 	  if (! RTLD_SINGLE_THREAD_P && imap->l_tls_modid > DTV_SURPLUS)
> 	    _dl_signal_error (0, "dlopen", NULL, N_("\
> cannot load any more object with static TLS"));
> 
> is wrong.  The test is saying "if we have loaded a certain number of
> dynamic objects with TLS segments, refuse to dlopen any more
> containing TLS if we are multi-threaded".
> 
> What it should be saying is "if we have loaded a certain number of
> dynamic objects with TLS segments *after we went multi-threaded*,
> refuse to open any more".  In particular, any dynamic objects with TLS
> segments loaded at program startup should not be counted.  This is
> because DTV_SURPLUS *extra* slots are allocated above those needed at
> program startup.  At least, that's how I think it works.

Yeah, this also looks like very good idea to do (and would solve several
practical issues with this limit that I saw while googling for it). 

Still if someone dlopens bazzilion of shared libraries built with profile
feedback and does so after going multithreaded, it should not hit the limit. So
I think we need GCC side solution too.

Honza
> 
> -- 
> Alan Modra
> Australia Development Lab, IBM

Re: Avoid privatization of TLS variables

[Jan Hubicka]

> On Wed, Sep 24, 2014 at 6:58 PM, Jan Hubicka <hubicka@ucw.cz> wrote:
> >
> >    b:   00 00
> >                         9: R_X86_64_TPOFF32     __gcov_indirect_call_counters_ltopriv
> 
> Look at the .o file where __gcov_indirect_call_counters_ltopriv is
> defined.  That .o file must have the symbol marked as STT_TLS and it
> must be defined in a section with the SHF_TLS flag.  If that is not
> true, then that is your problem.

The variable is common, so all the object files define it same way as the
.o file I attached.  Note that the .o file links just fine when executed
out of LTO machinery.

Honza
> 
> Ian

Re: Avoid privatization of TLS variables

[Andrew Haley]

I may be guilty of missing a crucial point here, but: why do we care
about having a small limit of static TLS variables?

We surely could allocate, say, a megabyte of static TLS for each
thread.  We already allocate 64M for the thread-local malloc arena,
after all.  It doesn't cost anything beyond a little address space.

What am I missing?

Andrew.

Re: Avoid privatization of TLS variables

[Andrew Haley]

On 27/09/14 08:56, Andrew Haley wrote:
> I may be guilty of missing a crucial point here, but: why do we care
> about having a small limit of static TLS variables?
> 
> We surely could allocate, say, a megabyte of static TLS for each
> thread.  We already allocate 64M for the thread-local malloc arena,

On 64-bit systems, I mean.

> after all.  It doesn't cost anything beyond a little address space.
> 
> What am I missing?
> 
> Andrew.
>