dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0

dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0

[Fangrui Song]

I was trying to clean up llvm-project/compiler-rt sanitizers GetTls
function https://reviews.llvm.org/D98926 and my change was reverted
due to an annoying Ubuntu 16.04 glibc 2.23 bug (seem so on both amd64
and ppc64le)
dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0.

Thread 1 "tls_race.cpp.tm" hit Breakpoint 6,
__sanitizer::CollectStaticTlsRanges (info=0x7fffffffe8d0, size=64,
data=0x7fffffffe9d0) at
/tmp/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:294
294       if (!info->dlpi_tls_data)
(gdb) p *info
$5 = {dlpi_addr = 0, dlpi_name = 0x7ffff7ffe6f8 "", dlpi_phdr =
0x400040, dlpi_phnum = 10, dlpi_adds = 10, dlpi_subs = 0,
dlpi_tls_modid = 1, dlpi_tls_data = 0x0}

This works fine on glibc 2.31.

# dlpi_tls_data != NULL when dlpi_tls_modid>0
$1 = {dlpi_addr = 0, dlpi_name = 0x7ffff7ffe720 "", dlpi_phdr =
0x400040, dlpi_phnum = 12, dlpi_adds = 10, dlpi_subs = 0,
dlpi_tls_modid = 1, dlpi_tls_data = 0x7ffff7a35740}

So do people know when the bug was fixed? From the file log the
info.dlpi_tls_data assignment code hasn't changed since 2011-09.

commit 74718d13e4638ccc5922c2197b9088ff5fc00251
Author: Ulrich Drepper <drepper@gmail.com>
Date:   Fri Sep 9 06:43:43 2011

    Fill in real information in __dl_iterate_phdr

Re: dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0

[Florian Weimer]

* Fangrui Song:

> I was trying to clean up llvm-project/compiler-rt sanitizers GetTls
> function https://reviews.llvm.org/D98926 and my change was reverted
> due to an annoying Ubuntu 16.04 glibc 2.23 bug (seem so on both amd64
> and ppc64le)
> dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0.
>
> Thread 1 "tls_race.cpp.tm" hit Breakpoint 6,
> __sanitizer::CollectStaticTlsRanges (info=0x7fffffffe8d0, size=64,
> data=0x7fffffffe9d0) at
> /tmp/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:294
> 294       if (!info->dlpi_tls_data)
> (gdb) p *info
> $5 = {dlpi_addr = 0, dlpi_name = 0x7ffff7ffe6f8 "", dlpi_phdr =
> 0x400040, dlpi_phnum = 10, dlpi_adds = 10, dlpi_subs = 0,
> dlpi_tls_modid = 1, dlpi_tls_data = 0x0}
>
> This works fine on glibc 2.31.

How have you verified this on glibc 2.31?  As far as I can tell, the
TLS block address assignment happens lazily, and dl_iterate_phdr does
not trigger it.  So it could be the result of different TLS access
patterns.

Re: dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0

[Fangrui Song]

On Sat, Apr 3, 2021 at 2:14 AM Florian Weimer <fw@deneb.enyo.de> wrote:
>
> * Fangrui Song:
>
> > I was trying to clean up llvm-project/compiler-rt sanitizers GetTls
> > function https://reviews.llvm.org/D98926 and my change was reverted
> > due to an annoying Ubuntu 16.04 glibc 2.23 bug (seem so on both amd64
> > and ppc64le)
> > dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0.
> >
> > Thread 1 "tls_race.cpp.tm" hit Breakpoint 6,
> > __sanitizer::CollectStaticTlsRanges (info=0x7fffffffe8d0, size=64,
> > data=0x7fffffffe9d0) at
> > /tmp/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:294
> > 294       if (!info->dlpi_tls_data)
> > (gdb) p *info
> > $5 = {dlpi_addr = 0, dlpi_name = 0x7ffff7ffe6f8 "", dlpi_phdr =
> > 0x400040, dlpi_phnum = 10, dlpi_adds = 10, dlpi_subs = 0,
> > dlpi_tls_modid = 1, dlpi_tls_data = 0x0}
> >
> > This works fine on glibc 2.31.
>
> How have you verified this on glibc 2.31?  As far as I can tell, the
> TLS block address assignment happens lazily, and dl_iterate_phdr does
> not trigger it.  So it could be the result of different TLS access
> patterns.

The fix may be https://sourceware.org/bugzilla/show_bug.cgi?id=19826

+         /* Set up the DTV entry.  The simplified __tls_get_addr that
+            some platforms use in static programs requires it.  */
+         dtv[map->l_tls_modid].pointer.val = dest;

With this change, static TLS blocks seem to have non-NULL dtv values.
Previously, dlpi_tls_data returned by dl_iterate_phdr may be NULL, if
the TLS data hasn't been touched by the program.

In sanitizer runtime, I will add another dl_iterate_phdr with the
callback doing __tls_get_addr({dlpi_tls_modid, 0}).
This can ensure static (and also dynamic?) TLS blocks have non-NULL dtv values.

Re: dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0

[Szabolcs Nagy]

The 04/04/2021 00:05, Fangrui Song wrote:
> On Sat, Apr 3, 2021 at 2:14 AM Florian Weimer <fw@deneb.enyo.de> wrote:
> > * Fangrui Song:
> > > I was trying to clean up llvm-project/compiler-rt sanitizers GetTls
> > > function https://reviews.llvm.org/D98926 and my change was reverted
> > > due to an annoying Ubuntu 16.04 glibc 2.23 bug (seem so on both amd64
> > > and ppc64le)
> > > dlpi_tls_data is incorrect NULL when dlpi_tls_modid>0.
> > >
> > > Thread 1 "tls_race.cpp.tm" hit Breakpoint 6,
> > > __sanitizer::CollectStaticTlsRanges (info=0x7fffffffe8d0, size=64,
> > > data=0x7fffffffe9d0) at
> > > /tmp/llvm/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cpp:294
> > > 294       if (!info->dlpi_tls_data)
> > > (gdb) p *info
> > > $5 = {dlpi_addr = 0, dlpi_name = 0x7ffff7ffe6f8 "", dlpi_phdr =
> > > 0x400040, dlpi_phnum = 10, dlpi_adds = 10, dlpi_subs = 0,
> > > dlpi_tls_modid = 1, dlpi_tls_data = 0x0}
> > >
> > > This works fine on glibc 2.31.
> >
> > How have you verified this on glibc 2.31?  As far as I can tell, the
> > TLS block address assignment happens lazily, and dl_iterate_phdr does
> > not trigger it.  So it could be the result of different TLS access
> > patterns.
> 
> The fix may be https://sourceware.org/bugzilla/show_bug.cgi?id=19826
> 
> +         /* Set up the DTV entry.  The simplified __tls_get_addr that
> +            some platforms use in static programs requires it.  */
> +         dtv[map->l_tls_modid].pointer.val = dest;
> 
> With this change, static TLS blocks seem to have non-NULL dtv values.
> Previously, dlpi_tls_data returned by dl_iterate_phdr may be NULL, if
> the TLS data hasn't been touched by the program.
> 
> In sanitizer runtime, I will add another dl_iterate_phdr with the
> callback doing __tls_get_addr({dlpi_tls_modid, 0}).
> This can ensure static (and also dynamic?) TLS blocks have non-NULL dtv values.

i think that works, (non-null dtv value in current thread), but
if you control the callback then you can just start it with

  info.dlpi_tls_data = __tls_get_addr({modid, 0})

no need for a separate dl_iterate_phdr.
(note that itanium calls __tls_get_addr with two args)

if you do separate dl_iterate_phdr calls then you have to
look out for modules concurrently dlopened in between
(those can still return 0).