Re: [PATCH] nptl: Move stack list variables into _rtld_global

[Simon Marchi <simon.marchi@polymtl.ca>]

On 2021-03-05 12:15 p.m., Florian Weimer wrote:> * Simon Marchi via Libc-alpha:
> 
>> On 2020-11-13 10:10 a.m., Florian Weimer wrote:
>>> Now __thread_gscope_wait (the function behind THREAD_GSCOPE_WAIT,
>>> formerly __wait_lookup_done) can be implemented directly in ld.so,
>>> eliminating the unprotected GL (dl_wait_lookup_done) function
>>> pointer.
>>
>> Hi Florian,
>>
>> Presumably starting with this commit (I don't really know how to build a
>> glibc and test against it), GDB fails to attach to a threaded process
>> because libthread_db fails to initialize.  See:
>>
>>   https://sourceware.org/bugzilla/show_bug.cgi?id=27526
>>
>> The difference in behavior as seen from GDB is that libthread_db now
>> asks to look up a symbol "_dl_stack_user" in module NULL.  GDB can't
>> find this symbol, which fails the initialization.
>>
>> Can you shed some light on this?  Is this request expected, and where is
>> GDB expected to find this symbol?
> 
> It is not expected.  This is the fallback path if _rtld_global cannot be
> located.  The actual failure is that __td_ta_rtld_global does not
> succeed.

[adding gdb-patches]

Ok, thanks for that tip.  Indeed I see that GDB returns PS_NOSYM for
_rtld_global.  If I now log what GDB returns:

    LOOKUP nptl_version in libpthread.so.0
    Found 0x7fdb02713037
    LOOKUP _rtld_global in ld-linux-x86-64.so.2
    Not found
    LOOKUP _dl_stack_user in (null)
    Not found

So this lookup of _rtld_global is new too.  And I think I see the
problem, it looks like an ordering issue: libthread_db is loaded when
GDB notices the program has libpthread loaded in it.  When attaching,
GDB walks the shared library list.  In that list, libpthread comes
before ld-linux.  So at the time we try to load libthread_db, GDB hasn't
yet noticed that the program has ld-linux loaded in it, hasn't ingested
its symbols, so doesn't find _rtld_global.

For comparison, glibc 2.31 (on Ubuntu 20.04) only requested symbols in
libpthread itself, so there wasn't this ordering issue:

    LOOKUP nptl_version in libpthread.so.0
    Found 0x7f38538e9037
    LOOKUP __stack_user in libpthread.so.0
    Found 0x7f38538f3350
    LOOKUP _thread_db_list_t_next in libpthread.so.0
    Found 0x7f38538e93b0
    LOOKUP _thread_db_const_thread_area in libpthread.so.0
    Found 0x7f38538e92b4
    LOOKUP _thread_db_sizeof_pthread in libpthread.so.0
    Found 0x7f38538e92cc
    LOOKUP _thread_db_pthread_specific in libpthread.so.0
    Found 0x7f38538e9400
    LOOKUP _thread_db_pthread_schedpolicy in libpthread.so.0
    Found 0x7f38538e9420
    LOOKUP _thread_db_pthread_schedparam_sched_priority in libpthread.so.0
    Found 0x7f38538e9410
    LOOKUP _thread_db_pthread_tid in libpthread.so.0
    Found 0x7f38538e9450
    LOOKUP _thread_db_pthread_cancelhandling in libpthread.so.0
    Found 0x7f38538e9430
    LOOKUP _thread_db_pthread_report_events in libpthread.so.0
    Found 0x7f38538e9460
    LOOKUP _thread_db_pthread_start_routine in libpthread.so.0
    Found 0x7f38538e9440
    LOOKUP _thread_db_pthread_eventbuf_eventmask_event_bits in libpthread.so.0
    Found 0x7f38538e93d0

If we have to deal with this, I guess that GDB should now do things in a
different order: go through the whole library list and load their
symbols.  And then if one of those libraries were libpthread, try to
initialize libthread_db.

Simon