limitations of TLS using GCC's __thread keyword and Cygwin

limitations of TLS using GCC's __thread keyword and Cygwin

[Václav Zeman]

Hi.

I am am porting a library that can use the __thread keyword in its
internals to provide thread local storage. Now, with MSVC there is a
limitation on pre-Vista Windows (see [1]) that DLLs using
__declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
declared that way only on process startup. Vista and later Windows do
not seem to have the limitation. Since Cygwin officially still
supports at least Windows XP, I want to provide a library that works
there as well.

Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
are Cygwin DLLs using TLS declared using __thread keyword safe to be
loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
that way?

[1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx

-- 
VZ

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Ryan Johnson]

On 04/09/2012 8:58 AM, Václav Zeman wrote:
> Hi.
>
> I am am porting a library that can use the __thread keyword in its
> internals to provide thread local storage. Now, with MSVC there is a
> limitation on pre-Vista Windows (see [1]) that DLLs using
> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
> declared that way only on process startup. Vista and later Windows do
> not seem to have the limitation. Since Cygwin officially still
> supports at least Windows XP, I want to provide a library that works
> there as well.
>
> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
> are Cygwin DLLs using TLS declared using __thread keyword safe to be
> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
> that way?
>
> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>
I suspect it's not a problem, but if I were you I'd write a simple test 
program to see. Unfortunately, TLS in general seems broken on my machine 
when I tried it, but that might be due to my home-brew gcc being 
configured wrong or something.

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Václav Zeman]

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

On 09/04/2012 04:39 PM, Ryan Johnson wrote:
> On 04/09/2012 8:58 AM, Václav Zeman wrote:
>> Hi.
>>
>> I am am porting a library that can use the __thread keyword in its
>> internals to provide thread local storage. Now, with MSVC there is a
>> limitation on pre-Vista Windows (see [1]) that DLLs using
>> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
>> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
>> declared that way only on process startup. Vista and later Windows do
>> not seem to have the limitation. Since Cygwin officially still
>> supports at least Windows XP, I want to provide a library that works
>> there as well.
>>
>> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
>> are Cygwin DLLs using TLS declared using __thread keyword safe to be
>> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
>> that way?
>>
>> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>>
> I suspect it's not a problem, but if I were you I'd write a simple
> test program to see. Unfortunately, TLS in general seems broken on my
> machine when I tried it, but that might be due to my home-brew gcc
> being configured wrong or something.
I would have done that already but I do not have any Windows XP machine
to try this on.

-- 
VZ



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 291 bytes --]

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Christopher Faylor]

On Tue, Sep 04, 2012 at 10:50:09PM +0200, V??clav Zeman wrote:
>On 09/04/2012 04:39 PM, Ryan Johnson wrote:
>> On 04/09/2012 8:58 AM, V??clav Zeman wrote:
>>> Hi.
>>>
>>> I am am porting a library that can use the __thread keyword in its
>>> internals to provide thread local storage. Now, with MSVC there is a
>>> limitation on pre-Vista Windows (see [1]) that DLLs using
>>> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
>>> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
>>> declared that way only on process startup. Vista and later Windows do
>>> not seem to have the limitation. Since Cygwin officially still
>>> supports at least Windows XP, I want to provide a library that works
>>> there as well.
>>>
>>> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
>>> are Cygwin DLLs using TLS declared using __thread keyword safe to be
>>> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
>>> that way?
>>>
>>> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>>>
>> I suspect it's not a problem, but if I were you I'd write a simple
>> test program to see. Unfortunately, TLS in general seems broken on my
>> machine when I tried it, but that might be due to my home-brew gcc
>> being configured wrong or something.
>I would have done that already but I do not have any Windows XP machine
>to try this on.

I don't believe that __thread is implemented on Cygwin.

cgf

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Václav Zeman]

On 4 September 2012 23:51, Christopher Faylor wrote:
> On Tue, Sep 04, 2012 at 10:50:09PM +0200, V??clav Zeman wrote:
>>On 09/04/2012 04:39 PM, Ryan Johnson wrote:
>>> On 04/09/2012 8:58 AM, V??clav Zeman wrote:
>>>> Hi.
>>>>
>>>> I am am porting a library that can use the __thread keyword in its
>>>> internals to provide thread local storage. Now, with MSVC there is a
>>>> limitation on pre-Vista Windows (see [1]) that DLLs using
>>>> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
>>>> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
>>>> declared that way only on process startup. Vista and later Windows do
>>>> not seem to have the limitation. Since Cygwin officially still
>>>> supports at least Windows XP, I want to provide a library that works
>>>> there as well.
>>>>
>>>> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
>>>> are Cygwin DLLs using TLS declared using __thread keyword safe to be
>>>> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
>>>> that way?
>>>>
>>>> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>>>>
>>> I suspect it's not a problem, but if I were you I'd write a simple
>>> test program to see. Unfortunately, TLS in general seems broken on my
>>> machine when I tried it, but that might be due to my home-brew gcc
>>> being configured wrong or something.
>>I would have done that already but I do not have any Windows XP machine
>>to try this on.
>
> I don't believe that __thread is implemented on Cygwin.
Adjust your beliefs. :) It is implemented and it works correct as far
as I can tell. It implements TLS using calls to __emutls* routines.
What I am unclear about is whether the implemention works around the
limitation mentioned in the MSDN link or not.

-- 
VZ

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Ryan Johnson]

[-- Attachment #1: Type: text/plain, Size: 3332 bytes --]

On 05/09/2012 4:05 AM, Václav Zeman wrote:
> On 4 September 2012 23:51, Christopher Faylor wrote:
>> On Tue, Sep 04, 2012 at 10:50:09PM +0200, V??clav Zeman wrote:
>>> On 09/04/2012 04:39 PM, Ryan Johnson wrote:
>>>> On 04/09/2012 8:58 AM, V??clav Zeman wrote:
>>>>> Hi.
>>>>>
>>>>> I am am porting a library that can use the __thread keyword in its
>>>>> internals to provide thread local storage. Now, with MSVC there is a
>>>>> limitation on pre-Vista Windows (see [1]) that DLLs using
>>>>> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
>>>>> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
>>>>> declared that way only on process startup. Vista and later Windows do
>>>>> not seem to have the limitation. Since Cygwin officially still
>>>>> supports at least Windows XP, I want to provide a library that works
>>>>> there as well.
>>>>>
>>>>> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
>>>>> are Cygwin DLLs using TLS declared using __thread keyword safe to be
>>>>> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
>>>>> that way?
>>>>>
>>>>> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>>>>>
>>>> I suspect it's not a problem, but if I were you I'd write a simple
>>>> test program to see. Unfortunately, TLS in general seems broken on my
>>>> machine when I tried it, but that might be due to my home-brew gcc
>>>> being configured wrong or something.
>>> I would have done that already but I do not have any Windows XP machine
>>> to try this on.
>> I don't believe that __thread is implemented on Cygwin.
> Adjust your beliefs. :) It is implemented and it works correct as far
> as I can tell. It implements TLS using calls to __emutls* routines.
> What I am unclear about is whether the implemention works around the
> limitation mentioned in the MSDN link or not.
GCC collects all TLS "slots" into a giant struct and then associates one 
copy of that struct with each thread. On targets without ABI support for 
TLS, the association is made using a single posix thread-local key. See 
gcc sources, libgcc/emutls.c for gory details.

Because the giant-TLS-struct is associated with a single Windows TLS 
slot, there should be no particular limitations on its use compared with 
linux.

Note, however, that you can't directly access TLS of a dlopen'd object: 
dlsym looks for a "normal" variable, failing because it doesn't exist. 
However, code inside the dlopened object can access its TLS correctly 
(so you could write a wrapper to return the TLS pointer), and dlopened 
objects can also correctly access TLS declared in the main object if you 
link them properly [1].

[1] http://cygwin.com/faq.html#faq.programming.unix-gui

STC attached, compile with:

g++ -Wl,--export-all-symbols,--out-implib,liba.exe.a tls.cpp && g++ 
-shared ext-tls.cpp liba.exe.a -oext-tls.dll && ./a

Caveat: I'm not completely sure how the dlopen'd file creates its TLS 
block, because there's no way it can share the same one as the main app. 
If it needs to create a new Windows TLS slot at load time, you may hit 
problems under Windows XP; I got some strange behavior running the STC 
under XP compatability mode on my Win7 machine (though oddly, it was the 
main object that broke; the dlopen'd object worked correctly).

Ryan


[-- Attachment #2: tls.cpp --]
[-- Type: text/plain, Size: 1053 bytes --]

#include <pthread.h>
#include <cstdio>
#include <unistd.h>
#include <dlfcn.h>

__thread int my_int;

bool go = false;
extern "C" void* run(void* obj) {
    int *(*get_ext_tls)() = (int*(*)()) dlsym(obj, "get_ext_int");
    int *(*get_my_tls)() = (int*(*)()) dlsym(obj, "get_my_int");
    int *ext_int = (int*) dlsym(obj, "ext_int");
    int *my_tls = get_my_tls();
    int *ext_tls = get_ext_tls();
    while(not go)
        usleep(100*1000);

    printf("tid=%x, &my_int=%p, &my_tls=%p, &ext_int=%p, &ext_tls=%p\n",
           (int)pthread_self(), &my_int, my_tls, ext_int, ext_tls);
    return 0;
}
int main() {
    void* obj = dlopen("ext-tls", 0);
    if (not obj)
        fprintf(stderr, "Unable to open dll: %s\n", dlerror());
    
    pthread_t tids[10];
    for(unsigned i=0; i < sizeof(tids)/sizeof(*tids); i++)
        pthread_create(&tids[i], 0, &run, obj);

    fprintf(stderr, "pid: %d\n", getpid());
    sleep(1);
    go = true;
    run(obj);

    for(unsigned i=0; i < sizeof(tids)/sizeof(*tids); i++)
        pthread_join(tids[i], 0);
}

[-- Attachment #3: ext-tls.cpp --]
[-- Type: text/plain, Size: 169 bytes --]

extern "C" {
    __thread int ext_int;
    
    extern __thread int my_int;
    
    int* get_ext_int() { return &ext_int; }
    int* get_my_int() { return &my_int; }
}

[-- Attachment #4: Type: text/plain, Size: 218 bytes --]

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: limitations of TLS using GCC's __thread keyword and Cygwin

[Václav Zeman]

On 5 September 2012 15:40, Ryan Johnson wrote:
> On 05/09/2012 4:05 AM, Václav Zeman wrote:
>>
>> On 4 September 2012 23:51, Christopher Faylor wrote:
>>>
>>> On Tue, Sep 04, 2012 at 10:50:09PM +0200, V??clav Zeman wrote:
>>>>
>>>> On 09/04/2012 04:39 PM, Ryan Johnson wrote:
>>>>>
>>>>> On 04/09/2012 8:58 AM, V??clav Zeman wrote:
>>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> I am am porting a library that can use the __thread keyword in its
>>>>>> internals to provide thread local storage. Now, with MSVC there is a
>>>>>> limitation on pre-Vista Windows (see [1]) that DLLs using
>>>>>> __declspec(thread) (MSVC equivalent of GCC's __thread) cannot be
>>>>>> loaded using LoadLibrary() because pre-Vista Windows allocate the TLS
>>>>>> declared that way only on process startup. Vista and later Windows do
>>>>>> not seem to have the limitation. Since Cygwin officially still
>>>>>> supports at least Windows XP, I want to provide a library that works
>>>>>> there as well.
>>>>>>
>>>>>> Does Cygwin's GCC and its TLS emulation work around this problem? IOW,
>>>>>> are Cygwin DLLs using TLS declared using __thread keyword safe to be
>>>>>> loaded using LoadLibrary()/dlopen() or are they not safe to be loaded
>>>>>> that way?
>>>>>>
>>>>>> [1] http://msdn.microsoft.com/en-us/library/2s9wt68x.aspx
>>>>>>
>>>>> I suspect it's not a problem, but if I were you I'd write a simple
>>>>> test program to see. Unfortunately, TLS in general seems broken on my
>>>>> machine when I tried it, but that might be due to my home-brew gcc
>>>>> being configured wrong or something.
>>>>
>>>> I would have done that already but I do not have any Windows XP machine
>>>> to try this on.
>>>
>>> I don't believe that __thread is implemented on Cygwin.
>>
>> Adjust your beliefs. :) It is implemented and it works correct as far
>> as I can tell. It implements TLS using calls to __emutls* routines.
>> What I am unclear about is whether the implemention works around the
>> limitation mentioned in the MSDN link or not.
>
> GCC collects all TLS "slots" into a giant struct and then associates one
> copy of that struct with each thread. On targets without ABI support for
> TLS, the association is made using a single posix thread-local key. See gcc
> sources, libgcc/emutls.c for gory details.
>
> Because the giant-TLS-struct is associated with a single Windows TLS slot,
> there should be no particular limitations on its use compared with linux.
>
> Note, however, that you can't directly access TLS of a dlopen'd object:
> dlsym looks for a "normal" variable, failing because it doesn't exist.
> However, code inside the dlopened object can access its TLS correctly (so
> you could write a wrapper to return the TLS pointer), and dlopened objects
> can also correctly access TLS declared in the main object if you link them
> properly [1].
>
> [1] http://cygwin.com/faq.html#faq.programming.unix-gui
>
> STC attached, compile with:
>
> g++ -Wl,--export-all-symbols,--out-implib,liba.exe.a tls.cpp && g++ -shared
> ext-tls.cpp liba.exe.a -oext-tls.dll && ./a
>
> Caveat: I'm not completely sure how the dlopen'd file creates its TLS block,
> because there's no way it can share the same one as the main app. If it
> needs to create a new Windows TLS slot at load time, you may hit problems
> under Windows XP; I got some strange behavior running the STC under XP
> compatability mode on my Win7 machine (though oddly, it was the main object
> that broke; the dlopen'd object worked correctly).
Thank you for this testing. I guess that the safest course of action
for me is to rely only on own instance of pthread_key_t then.

-- 
VZ

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple