GDB support for thread-local storage

GDB support for thread-local storage

[Jim Blandy]

I'd like to extend GDB to support thread-local variables.  Richard
Henderson, Ulrich Drepper and I have come up with some pieces of the
solutions; pretty much everything outside GDB has been settled, but
I'm still trying to figure out how GDB will pull the pieces together.

This post describes:
- the feature we're trying to support,
- the parts we've worked out so far, and
- the parts in GDB that I'm still trying to sort out.

Some implementations of C and C++ support a ``__thread'' storage
class, for variables that occupy distinct memory in distinct threads.
For example, the definition:

       __thread int foo;

declares an integer variable named ``foo'' which has a separate value
and address in each thread, much as a variable declared ``auto'' has a
separate value and address in each invocation of the function
containing its declaration.  Creating a new thread creates a new
instance of ``foo'', and when the thread exits, the storage for
``foo'' is freed.

Typically, a program includes an ``initialization image'' --- a block
of memory containing the initial values for any thread-local variables
it defines.  When the program creates a new thread, the run-time
system allocates a fresh block of memory for those thread-local
variables, and copies the initialization image into it to give the
variables their initialized values.

A dynamically loaded library may also define thread-local variables.
Some implementations delay allocating memory for such variables until
the thread actually refers to them for the first time.  This avoids
the overhead of allocating and initializing the library's thread-local
storage for all the threads present in a program when the library is
loaded, even though only a few threads might actually use the library.

Thread-local storage requires support in the ABI, and support in the
dynamic linker, if you want reasonable performance.  There's a
complete description of how it's done on the IA-32, IA-64, and SPARC
at http://people.redhat.com/drepper/tls.pdf.  This is based on
specifications already written for the IA-64 and SPARC; I think the
IA-32 implementation is Ulrich Drepper's work.

For GDB, the first question is: how should the debugging information
describe the location of a thread-local variable?  We generally answer
this sort of question by looking at how the code generated by the
compiler finds the variable, and then emitting debugging information
that matches that.

To allow the run-time system to allocate thread-local storage on
demand, the ABI in certain circumstances requires the compiler to emit
a call to a function, __tls_get_addr, to find the address of a
thread-local variable for the current thread and a particular module.
This function looks up the address in a table, allocates and
initializes the storage if necessary, and returns its address.

Unfortunately, Dwarf 2 location expressions cannot perform function
calls in the inferior.  We could extend it to do this, but inferior
function calls are rather complicated (look at the *_push_arguments
functions and hand_function_call in GDB); I don't think this is a good
idea.

Instead, I've suggested adding a new Dwarf 2 opcode: 

    12. DW_OP_push_tls_address

    The DW_OP_push_tls_address operation pushes the base address of the
    current thread's thread-local storage block.  If the expression occurs
    in the Dwarf information for a dynamically loaded library, then
    DW_OP_push_tls_address pushes the base address of that library's block
    for the current thread.  If the library's storage for the current
    thread has not yet been allocated, a Dwarf consumer may arrange for it
    to be allocated now, or report an error to the user.

When an implementation allocates thread-local storage on demand, this
makes it hard to describe the location of a thread-local variable
using ordinary Dwarf expressions: referencing the storage may entail
allocating memory, copying an initialization image into place,
registering it with the thread, and so on.  A dedicated operation like
DW_OP_push_tls_address leaves this complicated task to the debugger,
which is presumably already familiar with the program's ABI and thread
system, and can handle the request appropriately.

I've posted a note to the Dwarf mailing list, describing the
DW_OP_push_tls_address approach, and saying that we'll experiment with
this as a GNU extension to Dwarf and write back when we've actually
got something working.

For STABS, we can simply invent a new symbol type, whose value is the
offset within the thread-local storage block for the current thread
for the module containing the stab.  I haven't written up a real
proposal for STABS yet.

On Linux, Ulrich Drepper has added the following function to
libthread_db:

/* Get address of thread local variable.  */
extern td_err_e td_thr_tls_get_addr (const td_thrhandle_t *__th,
                                     struct link_map *__map, size_t __offset,
                                     void **__address);

This takes a thread handle, an entry from the dynamic linker's link
map, and an offset, and sets *__address to point to the base of that
thread and module's thread-local storage, plus the offset.  It returns
an error code if the space hasn't been allocated yet.

Note that this interface is not cross-debugging clean.  Actually, none
of the libthread_db interface is --- the underlying proc_service
interface uses the hosts' `paddr_t' type to represent addresses in the
running program, and the top-side interface uses `void *' to represent
the location of thread-local data, and in the function above.  More on
this later.


So now we get to the part which isn't really sorted out yet, in my
opinion.  There's a lot that needs to happen between the point where
GDB reads debugging information for a thread-local variable, and the
point where GDB can find a thread-local variable's value.

GDB already has an address class apparently intended for describing
thread-local data.  In symtab.h:

    enum address_class
      {
        ...

        /* Value is at a thread-specific location calculated by a
           target-specific method. */

        LOC_THREAD_LOCAL_STATIC,

        ...
      };

It would be more proper for that comment to say that the location is
calculated by an ABI-specific method.  There needs to be a new gdbarch
method:

  /* return a `struct value' for the object of type TYPE at OFFSET in
     the thread-local storage block for THREAD and MODULE.  If the
     thread-local storage block hasn't been allocated yet, raise an
     error.  */
  struct value *gdbarch_tls_get_value (struct thread_info *thread,
                                       struct objfile *module,
                                       LONGEST offset,
                                       struct type *type);

In order for this to reach the thread layer where we can call
libthread_db, there will need to be a new target stack method as well,
which the gdbarch method can invoke if it pleases.  Something similar
to the gdbarch method:

    struct value *(*to_tls_get_addr) (struct thread_info *thread,
                                      struct objfile *module,
                                      LONGEST offset,
                                      struct type *type);

The default implementation will raise an error, saying we don't know
how to find thread-local storage on this system.  The libthread_db
target will override that default with something that calls
td_thr_tls_get_addr.

If you're not convinced it should be a target method, consider this:
Remember that libthread_db isn't clean for cross-debugging.  It's a
target library.  So at the moment, there are cases where gdbserver
loads and uses libthread_db, not GDB itself.  In those cases, the
tls_get_addr request needs to be sent across the network connection to
gdbserver, td_thr_tls_get_addr needs to be invoked there, and the
answer needs to be sent back.  By making tls_get_addr a target method,
it's easy for the remote protocol layer to provide its own definition
of the method and send a packet across for the request.

Re: GDB support for thread-local storage

[Daniel Berlin]

On Wed, 19 Jun 2002, Jim Blandy wrote:

> 
> I'd like to extend GDB to support thread-local variables.  Richard
> Henderson, Ulrich Drepper and I have come up with some pieces of the
> solutions; pretty much everything outside GDB has been settled, but
> I'm still trying to figure out how GDB will pull the pieces together.
> 
> This post describes:
> - the feature we're trying to support,
> - the parts we've worked out so far, and
> - the parts in GDB that I'm still trying to sort out.
> 
> Some implementations of C and C++ support a ``__thread'' storage
> class, for variables that occupy distinct memory in distinct threads.
> For example, the definition:
> 
>        __thread int foo;
> 
> declares an integer variable named ``foo'' which has a separate value
> and address in each thread, much as a variable declared ``auto'' has a
> separate value and address in each invocation of the function
> containing its declaration.  Creating a new thread creates a new
> instance of ``foo'', and when the thread exits, the storage for
> ``foo'' is freed.
> 
> Typically, a program includes an ``initialization image'' --- a block
> of memory containing the initial values for any thread-local variables
> it defines.  When the program creates a new thread, the run-time
> system allocates a fresh block of memory for those thread-local
> variables, and copies the initialization image into it to give the
> variables their initialized values.
> 
> A dynamically loaded library may also define thread-local variables.
> Some implementations delay allocating memory for such variables until
> the thread actually refers to them for the first time.  This avoids
> the overhead of allocating and initializing the library's thread-local
> storage for all the threads present in a program when the library is
> loaded, even though only a few threads might actually use the library.
> 
> Thread-local storage requires support in the ABI, and support in the
> dynamic linker, if you want reasonable performance.  There's a
> complete description of how it's done on the IA-32, IA-64, and SPARC
> at http://people.redhat.com/drepper/tls.pdf.  This is based on
> specifications already written for the IA-64 and SPARC; I think the
> IA-32 implementation is Ulrich Drepper's work.
> 
> For GDB, the first question is: how should the debugging information
> describe the location of a thread-local variable?  We generally answer
> this sort of question by looking at how the code generated by the
> compiler finds the variable, and then emitting debugging information
> that matches that.
> 
> To allow the run-time system to allocate thread-local storage on
> demand, the ABI in certain circumstances requires the compiler to emit
> a call to a function, __tls_get_addr, to find the address of a
> thread-local variable for the current thread and a particular module.
> This function looks up the address in a table, allocates and
> initializes the storage if necessary, and returns its address.
> 
> Unfortunately, Dwarf 2 location expressions cannot perform function
> calls in the inferior. 

Errr, buzz.

See DW_OP_call_*  in dwarf3


It's not just turing complete anymore, one could probably write useful 
application extensions in dwarf3.
Scary.

--Dan

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Berlin <dberlin@dberlin.org> writes:
> > Unfortunately, Dwarf 2 location expressions cannot perform function
> > calls in the inferior. 
> 
> Errr, buzz.
> 
> See DW_OP_call_*  in dwarf3

(As I wrote that sentence, I was wondering whether I should clarify
this point.)

The function one may need to invoke to find thread-local storage,
__tls_get_addr, is an actual native code function, in the dynamic
linker.  The DW_OP_call_* operations allow a Dwarf expression to call
another Dwarf expression like a function.  But you can't use the
DW_OP_call_* operations to invoke machine-language functions in the
inferior.

> It's not just turing complete anymore, one could probably write useful 
> application extensions in dwarf3.
> Scary.

I thought so too, but then I noticed that there are no memory store
operations.  You can do stuff on the stack, but you can't modify
memory, or do any I/O.  Grave omissions, which should be rectified
immediately.

Re: GDB support for thread-local storage

[Daniel Berlin]

On 19 Jun 2002, Jim Blandy wrote:

> 
> Daniel Berlin <dberlin@dberlin.org> writes:
> > > Unfortunately, Dwarf 2 location expressions cannot perform function
> > > calls in the inferior. 
> > 
> > Errr, buzz.
> > 
> > See DW_OP_call_*  in dwarf3
> 
> (As I wrote that sentence, I was wondering whether I should clarify
> this point.)
> 
> The function one may need to invoke to find thread-local storage,
> __tls_get_addr, is an actual native code function, in the dynamic
> linker.  The DW_OP_call_* operations allow a Dwarf expression to call
> another Dwarf expression like a function.  But you can't use the
> DW_OP_call_* operations to invoke machine-language functions in the
> inferior.

You've assumed you need to.
If you can transform tls_get_addr into a dwarf expression, there you go.
You can deref memory, you just can't store into it.
Since the TLS address is just a location somewhere, computed without 
modifying anything, one should be able to transform it pretty easily.

Not that this is the best idea, but it's a possibility.
:)

 > 
> > It's not just turing complete anymore, one could probably write useful 
> > application extensions in dwarf3.
> > Scary.
> 
> I thought so too, but then I noticed that there are no memory store
> operations.  You can do stuff on the stack, but you can't modify
> memory, or do any I/O.  Grave omissions, which should be rectified
> immediately.

I'm on it.
It also needs a DW_OP_fix_bug
> 

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Berlin <dberlin@dberlin.org> writes:
> > The function one may need to invoke to find thread-local storage,
> > __tls_get_addr, is an actual native code function, in the dynamic
> > linker.  The DW_OP_call_* operations allow a Dwarf expression to call
> > another Dwarf expression like a function.  But you can't use the
> > DW_OP_call_* operations to invoke machine-language functions in the
> > inferior.
> 
> You've assumed you need to.
> If you can transform tls_get_addr into a dwarf expression, there you go.
> You can deref memory, you just can't store into it.
> Since the TLS address is just a location somewhere, computed without 
> modifying anything, one should be able to transform it pretty easily.
> 
> Not that this is the best idea, but it's a possibility.
> :)

We did try to find a way to just express this as a Dwarf expression,
before we resorted to a custom opcode and a libthread_db function.

One objection was that this would embed knowledge about the
implementation of TLS in the executable's debugging info.  The way to
find TLS can change depending on which dynamic linker and thread
library you happen to be (dynamically) linking against.  The process
for finding TLS really should be tightly coupled with the actual
dynamic linker and thread library in use, the way libthread_db is.

One could put the right Dwarf expression in the thread library or
dynamic linker's debugging info, and use DW_OP_call* to invoke it that
way --- that would get the coupling right.  But GDB doesn't apply
dynamic relocs to its debugging info (and it's hard to see exactly how
it could without a lot more help from the dynamic linker --- see Uli's
TLS paper).  This makes it hard to get the right operand for the
DW_OP_call4/8/whathaveyou.

There is also another problem I didn't mention.  You need to know the
module number in order to look up its TLS.  The dynamic linker doesn't
assign the module a number until it's loaded.  But perhaps the Dwarf
expression could actually get that from a GOT entry.

Re: GDB support for thread-local storage

[Andrew Cagney]

> I've posted a note to the Dwarf mailing list, describing the
> DW_OP_push_tls_address approach, and saying that we'll experiment with
> this as a GNU extension to Dwarf and write back when we've actually
> got something working.

Hmm, would you be able to post the prososal here?

> For STABS, we can simply invent a new symbol type, whose value is the
> offset within the thread-local storage block for the current thread
> for the module containing the stab.  I haven't written up a real
> proposal for STABS yet.
> 
> On Linux, Ulrich Drepper has added the following function to
> libthread_db:

Has solaris, or even MS, done anything in this area?  The 
LOC_THREAD_LOCAL_STATIC must have come from somewhere, dig dig, you may 
want to look at what HP/UX is getting up to.

> If you're not convinced it should be a target method, consider this:
> Remember that libthread_db isn't clean for cross-debugging.  It's a
> target library.  So at the moment, there are cases where gdbserver
> loads and uses libthread_db, not GDB itself.  In those cases, the
> tls_get_addr request needs to be sent across the network connection to
> gdbserver, td_thr_tls_get_addr needs to be invoked there, and the
> answer needs to be sent back.  By making tls_get_addr a target method,
> it's easy for the remote protocol layer to provide its own definition
> of the method and send a packet across for the request.

Similar to this, both SOFTWARE_SINGLESTEP and hardware breakpoints are 
ment to be implemented with support from both the target vector and the 
architecture vector.  By doing that, a sequence like:

	can target single step?
	  yes, step target
	else
	  use architecture to software singlestep target

can be implemented (in both cases it isn't so it can't, ulgh).

However, in the case of the above, is the architecture method needed? 
Given that th only thing implementing this will be the above GNU/Linux 
thread-db library, and GDB's linux thread code will know to call that 
directly.

BTW, what happens if the target doesn't have execution (i.e. a corefile).

Andrew

(Use thread_local_static rather than thr_tls in function names)

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Thu, Jun 20, 2002 at 09:35:25PM -0400, Andrew Cagney wrote:
> However, in the case of the above, is the architecture method needed? 
> Given that th only thing implementing this will be the above GNU/Linux 
> thread-db library, and GDB's linux thread code will know to call that 
> directly.
> 
> BTW, what happens if the target doesn't have execution (i.e. a corefile).

We fall down, just like we do debugging thread_db capable corefiles,
I'd imagine.  Thread_db does not like read-only targets very much.

> Andrew
> 
> (Use thread_local_static rather than thr_tls in function names)

_storage, I think?

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Andrew Cagney]

>> BTW, what happens if the target doesn't have execution (i.e. a corefile).
>> 

> 
> We fall down, just like we do debugging thread_db capable corefiles,
> I'd imagine.  Thread_db does not like read-only targets very much.

GDB on a threaded GNU/Linux target, again falls down, sigh!  We can 
hopefully have it working on other platforms.

Andrew

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 01:18:29PM -0400, Andrew Cagney wrote:
> >>BTW, what happens if the target doesn't have execution (i.e. a corefile).
> >>
> 
> >
> >We fall down, just like we do debugging thread_db capable corefiles,
> >I'd imagine.  Thread_db does not like read-only targets very much.
> 
> GDB on a threaded GNU/Linux target, again falls down, sigh!  We can 
> hopefully have it working on other platforms.

I beg your pardon?  This is a thread_db limitation, not a GNU/Linux
limitation.  GNU/Linux copied the interface from Solaris and I believe
we use it on Solaris.  The effect there will be even more extreme than
on GNU/Linux.  <check> Yes, we do use it on Solaris.

If threaded corefiles work on Solaris I doubt it's by merit of
thread_db.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@cygnus.com> writes:
> > I've posted a note to the Dwarf mailing list, describing the
> > DW_OP_push_tls_address approach, and saying that we'll experiment with
> > this as a GNU extension to Dwarf and write back when we've actually
> > got something working.
> 
> Hmm, would you be able to post the prososal here?

Sure thing.  (I re-used most of the language in the Dwarf proposal in
the message you replied to, so you've seen almost all of it already.)

Something to make clear, though --- this isn't a "proposal" proper.
The way the Dwarf committee operates (or the way I think it should
operate, and thus the way I try to deal with it) is:

- you float an idea by them for general approval;
- then you go off and implement it as a vendor extension;
- then once you've got it actually working you come back and make it a
  real proposal;
- if it's accepted you get assigned proper numbers (not in the vendor
  extension range) for your attribute tags, die tags, and so on;
- and finally you switch your emitter to recognize the real numbers.

So this is in the "float an idea by them" stage.  Very preliminary.


$Id: dwarf,v 1.4 2002/06/12 21:44:14 jimb Exp $

(The function of DW_OP_push_tls_address is similar to that of the
run-time function __get_tls_addr, so it was tempting to name the
operation DW_OP_get_tls_addr.  However, DW_OP_push_tls_address is more
consistent with the names of the other operations, especially
DW_OP_push_object_address.)


To section 2.4.1.3, "Stack Operations", add the following paragraphs
after DW_OP_push_object_address:

12. DW_OP_push_tls_address

The DW_OP_push_tls_address operation pushes the base address of the
current thread's thread-local storage block.  If the expression occurs
in the Dwarf information for a dynamically loaded library, then
DW_OP_push_tls_address pushes the base address of that library's block
for the current thread.  If the library's storage for the current
thread has not yet been allocated, a Dwarf consumer may arrange for it
to be allocated now, or report an error to the user.

<rationale italics>
Some implementations of C and C++ support a ``__thread'' storage
class, for variables that occupy distinct memory in distinct threads.
For example, the definition:

       __thread int foo;

declares an integer variable named ``foo'' which has a separate value
and address in each thread, much as a variable declared ``auto'' has a
separate value and address in each invocation of the function
containing its declaration.  Creating a new thread creates a new
instance of ``foo'', and when the thread exits, the storage for
``foo'' is freed.

Typically, a program includes an ``initialization image'' --- a block
of memory containing the initial values for any thread-local variables
it defines.  When the program creates a new thread, the run-time
system allocates a fresh block of memory for those thread-local
variables, and copies the initialization image into it to give the
variables their initialized values.

A dynamically loaded library may also define thread-local variables.
Some implementations delay allocating memory for such variables until
the thread actually refers to them for the first time.  This avoids
the overhead of allocating and initializing the library's thread-local
storage for all the threads present in a program when the library is
loaded, even though only a few threads might actually use the library.

However, when an implementation allocates thread-local storage on
demand, this makes it hard to describe the location of a thread-local
variable using ordinary Dwarf expressions: referencing the storage may
entail allocating memory, copying an initialization image into place,
registering it with the thread, and so on.  A dedicated operation like
DW_OP_push_tls_address leaves this complicated task to the debugger,
which is presumably already familiar with the program's ABI and thread
system, and can handle the request appropriately.
</rationale italics>

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@cygnus.com> writes:
> > For STABS, we can simply invent a new symbol type, whose value is the
> > offset within the thread-local storage block for the current thread
> > for the module containing the stab.  I haven't written up a real
> > proposal for STABS yet.
> > On Linux, Ulrich Drepper has added the following function to
> > libthread_db:
> 
> Has solaris, or even MS, done anything in this area?  The
> LOC_THREAD_LOCAL_STATIC must have come from somewhere, dig dig, you
> may want to look at what HP/UX is getting up to.

I didn't see anything in the Solaris "Stabs Interface Manual"
(distributed with the Solaris toolchain, not publicly) about it.

HP implements something much simpler.  It doesn't deal with
thread-local storage in PIC code; the initialization image is laid out
completely at static link time.  It's thread-local storage in
dynamically loaded libraries that introduces all the hair.

> > If you're not convinced it should be a target method, consider this:
> > Remember that libthread_db isn't clean for cross-debugging.  It's a
> > target library.  So at the moment, there are cases where gdbserver
> > loads and uses libthread_db, not GDB itself.  In those cases, the
> > tls_get_addr request needs to be sent across the network connection to
> > gdbserver, td_thr_tls_get_addr needs to be invoked there, and the
> > answer needs to be sent back.  By making tls_get_addr a target method,
> > it's easy for the remote protocol layer to provide its own definition
> > of the method and send a packet across for the request.
> 
> Similar to this, both SOFTWARE_SINGLESTEP and hardware breakpoints are
> ment to be implemented with support from both the target vector and
> the architecture vector.  By doing that, a sequence like:
> 
> 	can target single step?
> 	  yes, step target
> 	else
> 	  use architecture to software singlestep target
> 
> can be implemented (in both cases it isn't so it can't, ulgh).

Yes, I remember you saying this before; I was trying to follow your
lead here.

> However, in the case of the above, is the architecture method needed?
> Given that th only thing implementing this will be the above GNU/Linux
> thread-db library, and GDB's linux thread code will know to call that
> directly.

I don't think that's so.  As I say, Uli is pretty much just following
what the IA-64 and SPARC people have done for their ABI's, and
introducing a new scheme for the IA-32.  So we should expect this
feature to crop up on other platforms.

As far as the gdbarch method is concerned, I dunno.  It's true that
nobody is actually going to define the method at the moment.  But I
don't see why embedded ABI's wouldn't want to support __thread; the
whole point is that __thread can be faster and have less overhead than
the pthreads alternative, which (it seems to me) would make it very
attractive to the embedded world.  So I would expect the gdbarch
method to be used for the first embedded ABI that supports __thread.

Should we put off adding the gdbarch method until someone is actually
going to define it?

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Jacobowitz <drow@mvista.com> writes:
> On Fri, Jun 21, 2002 at 01:18:29PM -0400, Andrew Cagney wrote:
> > >>BTW, what happens if the target doesn't have execution (i.e. a corefile).
> > >
> > >We fall down, just like we do debugging thread_db capable corefiles,
> > >I'd imagine.  Thread_db does not like read-only targets very much.
> > 
> > GDB on a threaded GNU/Linux target, again falls down, sigh!  We can 
> > hopefully have it working on other platforms.
> 
> I beg your pardon?  This is a thread_db limitation, not a GNU/Linux
> limitation.  GNU/Linux copied the interface from Solaris and I believe
> we use it on Solaris.  The effect there will be even more extreme than
> on GNU/Linux.  <check> Yes, we do use it on Solaris.
> 
> If threaded corefiles work on Solaris I doubt it's by merit of
> thread_db.

We can debug multi-threaded core files on Linux; see below.  I must be
missing the point.  But this works because the core file reader
actually goes and constructs the thread list itself.  We're not using
thread_db to handle that.  Which is sort of a bug --- what you see
when you run GDB on such a core file are LWP's, not threads.

Why aren't we using thread_db, though?  Why can't we run thread_db and
simply serve its memory and register requests from the core file?  I
don't see which part of the interface makes this impossible.  And
we'll need to do it if Linux switches to an NxM thread model, no?


$ ps xf
  PID TTY      STAT   TIME COMMAND
...
 9415 pts/1    S      0:00 -bash
...
28675 pts/1    S      0:00  \_ ./linux-dp
28676 pts/1    S      0:00      \_ ./linux-dp
28677 pts/1    S      0:00          \_ ./linux-dp
28678 pts/1    S      0:00          \_ ./linux-dp
28679 pts/1    S      0:00          \_ ./linux-dp
28680 pts/1    S      0:00          \_ ./linux-dp
28681 pts/1    S      0:00          \_ ./linux-dp
...
$ gdb -nw linux-dp
GNU gdb 2002-06-13-cvs
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(gdb) attach 28675
Attaching to program: /rigel/jimb/cygnus/pentium3/sourceware/gdb/main/native/build/gdb/testsuite/gdb.threads/linux-dp, process 28675
Reading symbols from /lib/i686/libpthread.so.0...done.
[New Thread 1024 (LWP 28675)]
[New Thread 2049 (LWP 28676)]
[New Thread 1026 (LWP 28677)]
[New Thread 2051 (LWP 28678)]
[New Thread 3076 (LWP 28679)]
[New Thread 4101 (LWP 28680)]
[New Thread 5126 (LWP 28681)]
Loaded symbols for /lib/i686/libpthread.so.0
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
0x4011a5a1 in __libc_nanosleep () at __libc_nanosleep:-1
-1      __libc_nanosleep: No such file or directory.
        in __libc_nanosleep
(gdb) gcore linux-dp.core
Saved corefile linux-dp.core
(gdb) The program is running.  Quit anyway (and detach it)? (y or n) y
Detaching from program: /rigel/jimb/cygnus/pentium3/sourceware/gdb/main/native/build/gdb/testsuite/gdb.threads/linux-dp, process 28675
$ readelf -n linux-dp.core

Notes at offset 0x00000354 with length 0x000016dc:
  Owner         Data size       Description
  CORE          0x0000007c      NT_PRPSINFO (prpsinfo structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
  CORE          0x00000090      NT_PRSTATUS (prstatus structure)
  CORE          0x0000006c      NT_FPREGSET (floating point registers)
  LINUX         0x00000200      NT_PRXFPREG (user_xfpregs structure)
$ gdb -nw linux-dp
GNU gdb 2002-06-13-cvs
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(gdb) core linux-dp.core
warning: core file may not match specified executable file.
Core was generated by `/rigel/jimb/cygnus/pentium3/sourceware/gdb/main/native/build/gdb/testsuite/gdb.t'.
Program terminated with signal 17, Child status changed.
Reading symbols from /lib/i686/libpthread.so.0...done.
Loaded symbols for /lib/i686/libpthread.so.0
Reading symbols from /lib/i686/libm.so.6...done.
Loaded symbols for /lib/i686/libm.so.6
Reading symbols from /lib/i686/libc.so.6...done.
Loaded symbols for /lib/i686/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
#0  0x40146afe in __select () at __select:-1
-1      __select: No such file or directory.
        in __select
(gdb) info thread
  7 process 67137539  0x4011a5a1 in __libc_nanosleep () at __libc_nanosleep:-1
  6 process 134311939  0x401453e7 in __poll (fds=0x804bddc, nfds=1, 
    timeout=2000) at ../sysdeps/unix/sysv/linux/poll.c:63
  5 process 67268611  0x4008dba5 in __sigsuspend (set=0x4039a98c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  4 process 134443011  0x40146afe in __select () at __select:-1
  3 process 201617411  0x4008dba5 in __sigsuspend (set=0x4079a98c)
    at ../sysdeps/unix/sysv/linux/sigsuspend.c:45
  2 process 268791811  0x40146afe in __select () at __select:-1
* 1 process 335966211  0x40146afe in __select () at __select:-1
(gdb) 

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 03:08:03PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@mvista.com> writes:
> > On Fri, Jun 21, 2002 at 01:18:29PM -0400, Andrew Cagney wrote:
> > > >>BTW, what happens if the target doesn't have execution (i.e. a corefile).
> > > >
> > > >We fall down, just like we do debugging thread_db capable corefiles,
> > > >I'd imagine.  Thread_db does not like read-only targets very much.
> > > 
> > > GDB on a threaded GNU/Linux target, again falls down, sigh!  We can 
> > > hopefully have it working on other platforms.
> > 
> > I beg your pardon?  This is a thread_db limitation, not a GNU/Linux
> > limitation.  GNU/Linux copied the interface from Solaris and I believe
> > we use it on Solaris.  The effect there will be even more extreme than
> > on GNU/Linux.  <check> Yes, we do use it on Solaris.
> > 
> > If threaded corefiles work on Solaris I doubt it's by merit of
> > thread_db.
> 
> We can debug multi-threaded core files on Linux; see below.  I must be
> missing the point.  But this works because the core file reader
> actually goes and constructs the thread list itself.  We're not using
> thread_db to handle that.  Which is sort of a bug --- what you see
> when you run GDB on such a core file are LWP's, not threads.

Yes.  Search the archives to see the patch I committed to make this
"bug" happen instead of trying to use libthread_db; that message also
describes the problems.

And if you're feeling ambitious, search for the message which describes
why it broke libthread_db support for static executables.  I haven't
thought of a solution yet.

> Why aren't we using thread_db, though?  Why can't we run thread_db and
> simply serve its memory and register requests from the core file?  I
> don't see which part of the interface makes this impossible.  And
> we'll need to do it if Linux switches to an NxM thread model, no?

It's not interface but implementation.  There's all sorts of places
where libthread_db wants to write to the inferior.  If you fake memory
writes, it is possible that this would work; I couldn't figure out how
to do that non-intrusively.

Of much more interest to me is the fact that the LWP view of core files
can be supported cross-platform and the thread_db view of core files
can not be.  That is IMO a very substantial design flaw in
libthread_db.  Not a problem if all the world's a SPARC, but after the
effort I and others went through in order to make cross core debugging
functional I'm reluctant to use libthread_db anywhere I can avoid it.

An M:N implementation of libthread_db is even more likely to want to
write to or signal the inferior process.  I don't know if Sun's handles
this gracefully or if they grub through the corefile looking for
threads like we do.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 03:08:03PM -0500, Jim Blandy wrote:
> Why aren't we using thread_db, though?  Why can't we run thread_db and
> simply serve its memory and register requests from the core file?  I
> don't see which part of the interface makes this impossible.  And
> we'll need to do it if Linux switches to an NxM thread model, no?

I should add to my previous comment that the use of libthread_db to
access TLS data means that such will never be possible in a core file,
either, without significant redesign of libthread_db - possibly as some
sort of data file which can be loaded separately from the coredump and
describe thread structures.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Jacobowitz <drow@mvista.com> writes:
> Yes.  Search the archives to see the patch I committed to make this
> "bug" happen instead of trying to use libthread_db; that message also
> describes the problems.

Great, thanks.

> And if you're feeling ambitious, search for the message which describes
> why it broke libthread_db support for static executables.  I haven't
> thought of a solution yet.

I'm not that ambitious yet.

> > Why aren't we using thread_db, though?  Why can't we run thread_db and
> > simply serve its memory and register requests from the core file?  I
> > don't see which part of the interface makes this impossible.  And
> > we'll need to do it if Linux switches to an NxM thread model, no?
> 
> It's not interface but implementation.  There's all sorts of places
> where libthread_db wants to write to the inferior.  If you fake memory
> writes, it is possible that this would work; I couldn't figure out how
> to do that non-intrusively.

All the calls to ps_p[td]write I see in GNU's libthread_db have to do
with the implementation of events.  Can't we just ignore those writes?
Or better yet, make gdb/thread-db.c smart enough not to set event
breakpoints in the first place when the underyling target doesn't have
execution?

> Of much more interest to me is the fact that the LWP view of core files
> can be supported cross-platform and the thread_db view of core files
> can not be.  That is IMO a very substantial design flaw in
> libthread_db.  Not a problem if all the world's a SPARC, but after the
> effort I and others went through in order to make cross core debugging
> functional I'm reluctant to use libthread_db anywhere I can avoid
> it.

This is the more substantial objection, it seems to me.  But simply
because libthread_db can't be used for cross-platform core files
doesn't mean we shouldn't use it in the native case --- for the same
reasons we use it on live processes.

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 03:54:12PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@mvista.com> writes:
> > Yes.  Search the archives to see the patch I committed to make this
> > "bug" happen instead of trying to use libthread_db; that message also
> > describes the problems.
> 
> Great, thanks.
> 
> > And if you're feeling ambitious, search for the message which describes
> > why it broke libthread_db support for static executables.  I haven't
> > thought of a solution yet.
> 
> I'm not that ambitious yet.
> 
> > > Why aren't we using thread_db, though?  Why can't we run thread_db and
> > > simply serve its memory and register requests from the core file?  I
> > > don't see which part of the interface makes this impossible.  And
> > > we'll need to do it if Linux switches to an NxM thread model, no?
> > 
> > It's not interface but implementation.  There's all sorts of places
> > where libthread_db wants to write to the inferior.  If you fake memory
> > writes, it is possible that this would work; I couldn't figure out how
> > to do that non-intrusively.
> 
> All the calls to ps_p[td]write I see in GNU's libthread_db have to do
> with the implementation of events.  Can't we just ignore those writes?
> Or better yet, make gdb/thread-db.c smart enough not to set event
> breakpoints in the first place when the underyling target doesn't have
> execution?

I tried that; there was some other problem.  It might have been in my
implementation.  Indeed upon looking back at it I see that you are
correct.  Maybe it would work.

> > Of much more interest to me is the fact that the LWP view of core files
> > can be supported cross-platform and the thread_db view of core files
> > can not be.  That is IMO a very substantial design flaw in
> > libthread_db.  Not a problem if all the world's a SPARC, but after the
> > effort I and others went through in order to make cross core debugging
> > functional I'm reluctant to use libthread_db anywhere I can avoid
> > it.
> 
> This is the more substantial objection, it seems to me.  But simply
> because libthread_db can't be used for cross-platform core files
> doesn't mean we shouldn't use it in the native case --- for the same
> reasons we use it on live processes.

Maybe.... I don't think the analogy holds.  When we use it on live
processes there is always a system (somewhere) on which thread_db could
be running.  That's why I was willing to use it in gdbserver.  Of
course, it's ONE MORE library that needs to be on all my targets now,
which I'm not in love with.

Debugging a core dump can't validly require access to a target.  So
making "native debug of a core dump" different from the hopeful "cross
debug of a core dump" seems a bit dodgy.  I'd call the libthread_db
approach broken for this purpose (a little outside its design scope
perhaps).

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Andrew Cagney]

> This is the more substantial objection, it seems to me.  But simply
>> because libthread_db can't be used for cross-platform core files
>> doesn't mean we shouldn't use it in the native case --- for the same
>> reasons we use it on live processes.
> 
> 
> Maybe.... I don't think the analogy holds.  When we use it on live
> processes there is always a system (somewhere) on which thread_db could
> be running.  That's why I was willing to use it in gdbserver.  Of
> course, it's ONE MORE library that needs to be on all my targets now,
> which I'm not in love with.
> 
> Debugging a core dump can't validly require access to a target.  So
> making "native debug of a core dump" different from the hopeful "cross
> debug of a core dump" seems a bit dodgy.

Yes.

>  I'd call the libthread_db
> approach broken for this purpose (a little outside its design scope
> perhaps).

I think it reflects limitations of the current libthread-db interface 
rather than a broken approach.

Andrew

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 05:46:05PM -0400, Andrew Cagney wrote:
> >This is the more substantial objection, it seems to me.  But simply
> >>because libthread_db can't be used for cross-platform core files
> >>doesn't mean we shouldn't use it in the native case --- for the same
> >>reasons we use it on live processes.
> >
> >
> >Maybe.... I don't think the analogy holds.  When we use it on live
> >processes there is always a system (somewhere) on which thread_db could
> >be running.  That's why I was willing to use it in gdbserver.  Of
> >course, it's ONE MORE library that needs to be on all my targets now,
> >which I'm not in love with.
> >
> >Debugging a core dump can't validly require access to a target.  So
> >making "native debug of a core dump" different from the hopeful "cross
> >debug of a core dump" seems a bit dodgy.
> 
> Yes.
> 
> > I'd call the libthread_db
> >approach broken for this purpose (a little outside its design scope
> >perhaps).
> 
> I think it reflects limitations of the current libthread-db interface 
> rather than a broken approach.

I disagree... the concept of having a "libthread_db" with an interface
involves it being a target library, part of the system.  Unless you
change its "interface" to be a data file rather than code, it requires
access to a target in order to interpret target data.  That's my whole
objection to it.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Andrew Cagney]

> /* Get address of thread local variable.  */
> extern td_err_e td_thr_tls_get_addr (const td_thrhandle_t *__th,
>                                      struct link_map *__map, size_t __offset,
>                                      void **__address);
> 
> This takes a thread handle, an entry from the dynamic linker's link
> map, and an offset, and sets *__address to point to the base of that
> thread and module's thread-local storage, plus the offset.  It returns
> an error code if the space hasn't been allocated yet.

What does GDB do if there isn't [yet] any allocated local storage?

Andrew

Re: GDB support for thread-local storage

[Andrew Cagney]

>> > I'd call the libthread_db
>> >approach broken for this purpose (a little outside its design scope
>> >perhaps).
> 
>> 
>> I think it reflects limitations of the current libthread-db interface 
>> rather than a broken approach.
> 
> 
> I disagree... the concept of having a "libthread_db" with an interface
> involves it being a target library, part of the system.  Unless you
> change its "interface" to be a data file rather than code, it requires
> access to a target in order to interpret target data.  That's my whole
> objection to it.

Sorry, I'm lost here.

Say, instead of a libthread_db, we had gdb/libthread-db.c which could be 
compiled on all systems.  It would have some sort of procedural 
interface, and would grub around in target data to find thread X lwp 
maps.  However, it could be written in a way that was host architecture 
netural.

Andrew

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Jacobowitz <drow@mvista.com> writes:
> On Fri, Jun 21, 2002 at 03:08:03PM -0500, Jim Blandy wrote:
> > Why aren't we using thread_db, though?  Why can't we run thread_db and
> > simply serve its memory and register requests from the core file?  I
> > don't see which part of the interface makes this impossible.  And
> > we'll need to do it if Linux switches to an NxM thread model, no?
> 
> I should add to my previous comment that the use of libthread_db to
> access TLS data means that such will never be possible in a core file,
> either, without significant redesign of libthread_db - possibly as some
> sort of data file which can be loaded separately from the coredump and
> describe thread structures.

Well, if the core file support is willing to reach into the notes and
construct a thread list, what's wrong with having it override the
`thread_local_storage_get_address' target method, too?

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@cygnus.com> writes:
> > /* Get address of thread local variable.  */
> > extern td_err_e td_thr_tls_get_addr (const td_thrhandle_t *__th,
> >                                      struct link_map *__map, size_t __offset,
> >                                      void **__address);
> > This takes a thread handle, an entry from the dynamic linker's link
> > map, and an offset, and sets *__address to point to the base of that
> > thread and module's thread-local storage, plus the offset.  It returns
> > an error code if the space hasn't been allocated yet.
> 
> What does GDB do if there isn't [yet] any allocated local storage?

It returns TD_NOTALLOC.

Re: GDB support for thread-local storage

[Andrew Cagney]

> Andrew Cagney <ac131313@cygnus.com> writes:
> 
>> > /* Get address of thread local variable.  */
>> > extern td_err_e td_thr_tls_get_addr (const td_thrhandle_t *__th,
>> >                                      struct link_map *__map, size_t __offset,
>> >                                      void **__address);
>> > This takes a thread handle, an entry from the dynamic linker's link
>> > map, and an offset, and sets *__address to point to the base of that
>> > thread and module's thread-local storage, plus the offset.  It returns
>> > an error code if the space hasn't been allocated yet.
> 
>> 
>> What does GDB do if there isn't [yet] any allocated local storage?
> 
> 
> It returns TD_NOTALLOC.

So, what does *GDB* do if there isn't [yet] any allocated local storage 
(as identified by the above function returning TD_NOTALLOC)?

Andrew

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@cygnus.com> writes:
> > /* Get address of thread local variable.  */
> > extern td_err_e td_thr_tls_get_addr (const td_thrhandle_t *__th,
> >                                      struct link_map *__map, size_t __offset,
> >                                      void **__address);
> > This takes a thread handle, an entry from the dynamic linker's link
> > map, and an offset, and sets *__address to point to the base of that
> > thread and module's thread-local storage, plus the offset.  It returns
> > an error code if the space hasn't been allocated yet.
> 
> What does GDB do if there isn't [yet] any allocated local storage?

Oh, sorry --- what does *GDB* do?

It depends.  For the first cut, it returns an error:

    (gdb) print i
    Storage for the thread-local variable `i' has not been allocated in
    this thread yet.
    (gdb)

But there are ways we could do better than that.

It's tempting to suggest that referring to unallocated thread-local
storage should cause the storage to be allocated.  This emulates the
way variable references in the actual program behave most closely.

However, this is a pretty heavy-duty operation.  It involves calling a
function in the inferior, which will do a malloc, copy in the
initialization image, update thread structures, and so on.  I don't
think simply trying to print a variable should cause that kind of
disturbance in the inferior.

Instead, I think trying to print an unallocated __thread variable
should cause an error, which suggests that the user try a command
`thread allocate VAR', which will do the right inferior-disturbing
magic to make VAR exist.  That way, the user has control over the
process, and GDB doesn't munge her inferior in unexpected ways.

There is a middle ground.  Initializing a thread-local storage block
is simply a memcpy of the initialization image.  There are no relocs
to apply --- stuff like this:

        __thread int i;
        __thread int *pi = &i;

is forbidden.  This means that there's an exact copy of all the bits
each variable *would* have, if it *were* allocated --- they're just
immutable, and at the wrong address.

So a really clever implementation could actually give you the value of
a thread-local variable like `i' even when it hasn't been allocated
yet.  It would be a non-lazy value, not an lvalue, and have no
address, but at least `print i' would give you the right answer.  We
could even set a bit on the value so that an attempt to assign to it
would give a helpful error message:

    (gdb) set var i = 2
    Storage for the thread-local variable `i' has not been allocated
    in this thread yet.  Type `thread allocate i' to allocate it.
    (gdb)

Or something like that.

Anyway, that's why the gdbarch and target methods I proposed are
producing `struct value' objects, and not something more low-level ---
to leave open the option of handling unallocated values this way.

Re: GDB support for thread-local storage

[Jim Blandy]

One of the reasons I really like having libthread_db handle TLS
resolution is that the alternative is to do an inferior function call
when you reference a variable.  Check out Uli's document, at the
pointer I gave --- even the compiler will sometimes have to generate a
call to __tls_get_addr to find a __thread variable's address.

And GDB shouldn't cache this base address while the inferior runs,
either --- remember the "GDB must never lie" rule.  Evaluating `x' in
GDB had better reference the same storage that it would if the compiler
evaluated `x' at the point where the program is stopped.

So if we have trouble keeping the Insight variable window up-to-date
now...

Anyway, in that context, having libthread_db handle it all in-process
seems really nice.

Re: GDB support for thread-local storage

[Jim Blandy]

Jim Blandy <jimb@redhat.com> writes:
> Andrew Cagney <ac131313@cygnus.com> writes:
> > Has solaris, or even MS, done anything in this area?  The
> > LOC_THREAD_LOCAL_STATIC must have come from somewhere, dig dig, you
> > may want to look at what HP/UX is getting up to.
> HP implements something much simpler.  It doesn't deal with
> thread-local storage in PIC code; the initialization image is laid out
> completely at static link time.  It's thread-local storage in
> dynamically loaded libraries that introduces all the hair.

What I wrote is incorrect.  HP does handle TLS in shared libraries.
But in their arrangement, every thread-local variable lives at a
offset from register CR27, and GDB can compute that offset at
symbol-reading time.

I think this means that they don't address a lot of the issues that
the IA-64 / SPARC / Red Hat proposal does.  I don't see how you'd
handle dlopen'd libraries or lazy allocation in their scheme.

But in any case, HP's gdbarch method for finding thread-local storage
would be very simple: just add the offset to CR27, and there's your
address.

Given that hpread.c hard-codes a reference to the macro CR27_REGNUM,
which is only defined in config/pa/tm-hppa.h, I assume that hpread.c
is only used in PA targets.  The PA hasn't been multi-arched, so I
assume it's going away.

So, does anyone care if I break the hpread.c code?

Re: GDB support for thread-local storage

[Andrew Cagney]

> Jim Blandy <jimb@redhat.com> writes:
> 
>> Andrew Cagney <ac131313@cygnus.com> writes:
> 
>> > Has solaris, or even MS, done anything in this area?  The
>> > LOC_THREAD_LOCAL_STATIC must have come from somewhere, dig dig, you
>> > may want to look at what HP/UX is getting up to.
> 
>> HP implements something much simpler.  It doesn't deal with
>> thread-local storage in PIC code; the initialization image is laid out
>> completely at static link time.  It's thread-local storage in
>> dynamically loaded libraries that introduces all the hair.
> 
> 
> What I wrote is incorrect.  HP does handle TLS in shared libraries.
> But in their arrangement, every thread-local variable lives at a
> offset from register CR27, and GDB can compute that offset at
> symbol-reading time.
> 
> I think this means that they don't address a lot of the issues that
> the IA-64 / SPARC / Red Hat proposal does.  I don't see how you'd
> handle dlopen'd libraries or lazy allocation in their scheme.

I don't know.  HP people do monitor this list so may be able to answer.

> But in any case, HP's gdbarch method for finding thread-local storage
> would be very simple: just add the offset to CR27, and there's your
> address.

BTW, why, in your propsal, is the offset incorporated into the address 
that is returned - rather than getting the base address and then adding 
the offset - more like HP did.

Andrew

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@cygnus.com> writes:
> > Jim Blandy <jimb@redhat.com> writes:
> >
> >> Andrew Cagney <ac131313@cygnus.com> writes:
> >
> >> > Has solaris, or even MS, done anything in this area?  The
> >> > LOC_THREAD_LOCAL_STATIC must have come from somewhere, dig dig, you
> >> > may want to look at what HP/UX is getting up to.
> >
> >> HP implements something much simpler.  It doesn't deal with
> >> thread-local storage in PIC code; the initialization image is laid out
> >> completely at static link time.  It's thread-local storage in
> >> dynamically loaded libraries that introduces all the hair.
> > What I wrote is incorrect.  HP does handle TLS in shared libraries.
> > But in their arrangement, every thread-local variable lives at a
> > offset from register CR27, and GDB can compute that offset at
> > symbol-reading time.
> > I think this means that they don't address a lot of the issues that
> > the IA-64 / SPARC / Red Hat proposal does.  I don't see how you'd
> > handle dlopen'd libraries or lazy allocation in their scheme.
> 
> I don't know.  HP people do monitor this list so may be able to answer.
> 
> > But in any case, HP's gdbarch method for finding thread-local storage
> > would be very simple: just add the offset to CR27, and there's your
> > address.
> 
> BTW, why, in your propsal, is the offset incorporated into the address
> that is returned - rather than getting the base address and then
> adding the offset - more like HP did.

That's me imitating a rather weird quirk in the way the TLS run-time
implementation works.  But if you look at my Dwarf 2 sketch proposal,
we can't do it that way.  We need something that gives us the base
address.  So the way HP does it is pretty much the way we'll do it,
except for the assumption that the base address is in a register.

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 06:14:41PM -0500, Jim Blandy wrote:
> 
> One of the reasons I really like having libthread_db handle TLS
> resolution is that the alternative is to do an inferior function call
> when you reference a variable.  Check out Uli's document, at the
> pointer I gave --- even the compiler will sometimes have to generate a
> call to __tls_get_addr to find a __thread variable's address.
> 
> And GDB shouldn't cache this base address while the inferior runs,
> either --- remember the "GDB must never lie" rule.  Evaluating `x' in
> GDB had better reference the same storage that it would if the compiler
> evaluated `x' at the point where the program is stopped.
> 
> So if we have trouble keeping the Insight variable window up-to-date
> now...
> 
> Anyway, in that context, having libthread_db handle it all in-process
> seems really nice.

Well, libthread_db is still pretty expensive.  It generally does a
substantial amount of memory access to the inferior.  Cheaper than a
function call, but not so much.

Part of that is the ridiculous way we use it and abuse the thread_alive
checks at every opportunity.  If you benchmark some operations on
libthread_db/native gdb, and gdb/local gdbserver (which uses thread_db,
but makes some documented assumptions about the threading model, and
also supports partial stops which I haven't worked out how to do in GDB
proper) you'll see that the overhead of the remote protocol is
sometimes less than the overhead of native thread_db.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 06:31:25PM -0400, Andrew Cagney wrote:
> 
> >>> I'd call the libthread_db
> >>>approach broken for this purpose (a little outside its design scope
> >>>perhaps).
> >
> >>
> >>I think it reflects limitations of the current libthread-db interface 
> >>rather than a broken approach.
> >
> >
> >I disagree... the concept of having a "libthread_db" with an interface
> >involves it being a target library, part of the system.  Unless you
> >change its "interface" to be a data file rather than code, it requires
> >access to a target in order to interpret target data.  That's my whole
> >objection to it.
> 
> Sorry, I'm lost here.
> 
> Say, instead of a libthread_db, we had gdb/libthread-db.c which could be 
> compiled on all systems.  It would have some sort of procedural 
> interface, and would grub around in target data to find thread X lwp 
> maps.  However, it could be written in a way that was host architecture 
> netural.

Sure.  But the design of libthread_db says, "I am 100% coupled to the
private structure of this thread implementation.  I must match its
version exactly if you want predictable results.  My details can change
in minor revisions or even more frequently."  That's not part of the
implementation; it's more like the purpose of the design.  It is a
layer between implementation-specific details with no guaranteed
structure and a structured client interface.

Without imposing structure on that data, I don't think it'll ever be
possible to have a gdb/libthread-db.c.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Fri, Jun 21, 2002 at 05:37:27PM -0500, Jim Blandy wrote:
> 
> Daniel Jacobowitz <drow@mvista.com> writes:
> > On Fri, Jun 21, 2002 at 03:08:03PM -0500, Jim Blandy wrote:
> > > Why aren't we using thread_db, though?  Why can't we run thread_db and
> > > simply serve its memory and register requests from the core file?  I
> > > don't see which part of the interface makes this impossible.  And
> > > we'll need to do it if Linux switches to an NxM thread model, no?
> > 
> > I should add to my previous comment that the use of libthread_db to
> > access TLS data means that such will never be possible in a core file,
> > either, without significant redesign of libthread_db - possibly as some
> > sort of data file which can be loaded separately from the coredump and
> > describe thread structures.
> 
> Well, if the core file support is willing to reach into the notes and
> construct a thread list, what's wrong with having it override the
> `thread_local_storage_get_address' target method, too?

Nothing, you're right.  Whether the corefile will have the details to
do this, however, without incorporating a large chunk of knowledge from
various target libthread_db systems, though - that's dubious at best. 
At least there is an ABI to guide us.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

unsuscribe

[phi]

unsuscribe

Re: GDB support for thread-local storage

[Andrew Cagney]

> Sorry, I'm lost here.
>> 
>> Say, instead of a libthread_db, we had gdb/libthread-db.c which could be 
>> compiled on all systems.  It would have some sort of procedural 
>> interface, and would grub around in target data to find thread X lwp 
>> maps.  However, it could be written in a way that was host architecture 
>> netural.
> 
> 
> Sure.  But the design of libthread_db says, "I am 100% coupled to the
> private structure of this thread implementation.  I must match its
> version exactly if you want predictable results.  My details can change
> in minor revisions or even more frequently."  That's not part of the
> implementation; it's more like the purpose of the design.  It is a
> layer between implementation-specific details with no guaranteed
> structure and a structured client interface.

Right.

But what is stopping us picking up that code, compiling it on the host 
(not target), and then using it in GDB?

> Without imposing structure on that data, I don't think it'll ever be
> possible to have a gdb/libthread-db.c.

We'll never be able to have a single libthread-db.c file but we should 
at least be able to get the interface defined, and the code implemented, 
in a way that makes it possible for it to be used on the host.

Andrew

Re: GDB support for thread-local storage

[Andrew Cagney]

> I don't know.  HP people do monitor this list so may be able to answer.
>> 
> 
>> > But in any case, HP's gdbarch method for finding thread-local storage
>> > would be very simple: just add the offset to CR27, and there's your
>> > address.
> 
>> 
>> BTW, why, in your propsal, is the offset incorporated into the address
>> that is returned - rather than getting the base address and then
>> adding the offset - more like HP did.
> 
> 
> That's me imitating a rather weird quirk in the way the TLS run-time
> implementation works.  But if you look at my Dwarf 2 sketch proposal,
> we can't do it that way.  We need something that gives us the base
> address.  So the way HP does it is pretty much the way we'll do it,
> except for the assumption that the base address is in a register.

So an initial work-in-progress interface can be simplified to just:

	target_thread_base_address(thread, pc?, &base_addr)
		-> ok, not-available

dropping that offset et.al.?  Someone else can add them if they later 
prove necessary.

Andrew

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Sat, Jun 22, 2002 at 11:22:49AM -0400, Andrew Cagney wrote:
> >Sorry, I'm lost here.
> >>
> >>Say, instead of a libthread_db, we had gdb/libthread-db.c which could be 
> >>compiled on all systems.  It would have some sort of procedural 
> >>interface, and would grub around in target data to find thread X lwp 
> >>maps.  However, it could be written in a way that was host architecture 
> >>netural.
> >
> >
> >Sure.  But the design of libthread_db says, "I am 100% coupled to the
> >private structure of this thread implementation.  I must match its
> >version exactly if you want predictable results.  My details can change
> >in minor revisions or even more frequently."  That's not part of the
> >implementation; it's more like the purpose of the design.  It is a
> >layer between implementation-specific details with no guaranteed
> >structure and a structured client interface.
> 
> Right.
> 
> But what is stopping us picking up that code, compiling it on the host 
> (not target), and then using it in GDB?

Version skew primarily.  If you examine the way that libthread_db works
in GNU libc, you'll see that it knows the offset to various private
data structures by including the internal linuxthreads implementation
header.  These can change between minor point-releases of glibc.  We
could detect the version string and have lists of offsets that way but
it would mean updating GDB for every new version of glibc.  This is
probably feasible but mitigates the value of thread_db greatly; the
principal advantage was separating the debugger from knowledge of the
threads implementation.  We'd have to build target data structures (the
way that C++ ABI support does) and their types can vary greatly
architecture to architecture.  If we require libpthread.so to contain
debugging information it would be a little more practical, maybe...

On Solaris this isn't even an option of course.  If we don't use the
system libthread_db we have no way to duplicate its functionality.

> >Without imposing structure on that data, I don't think it'll ever be
> >possible to have a gdb/libthread-db.c.
> 
> We'll never be able to have a single libthread-db.c file but we should 
> at least be able to get the interface defined, and the code implemented, 
> in a way that makes it possible for it to be used on the host.

Perhaps for a limited set of (target arch, target lib version) pairs.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Jacobowitz <drow@mvista.com> writes:
> On Fri, Jun 21, 2002 at 06:14:41PM -0500, Jim Blandy wrote:
> > 
> > One of the reasons I really like having libthread_db handle TLS
> > resolution is that the alternative is to do an inferior function call
> > when you reference a variable.  Check out Uli's document, at the
> > pointer I gave --- even the compiler will sometimes have to generate a
> > call to __tls_get_addr to find a __thread variable's address.
> > 
> > And GDB shouldn't cache this base address while the inferior runs,
> > either --- remember the "GDB must never lie" rule.  Evaluating `x' in
> > GDB had better reference the same storage that it would if the compiler
> > evaluated `x' at the point where the program is stopped.
> > 
> > So if we have trouble keeping the Insight variable window up-to-date
> > now...
> > 
> > Anyway, in that context, having libthread_db handle it all in-process
> > seems really nice.
> 
> Well, libthread_db is still pretty expensive.  It generally does a
> substantial amount of memory access to the inferior.  Cheaper than a
> function call, but not so much.

Well, td_thr_tls_get_addr only makes two calls to ps_pdread.  The
first one is a big one, unfortunately --- it reads a pretty large
structure.

> Part of that is the ridiculous way we use it and abuse the thread_alive
> checks at every opportunity.  If you benchmark some operations on
> libthread_db/native gdb, and gdb/local gdbserver (which uses thread_db,
> but makes some documented assumptions about the threading model, and
> also supports partial stops which I haven't worked out how to do in GDB
> proper) you'll see that the overhead of the remote protocol is
> sometimes less than the overhead of native thread_db.

It seems to me that the right place to handle this is in
proc-service.c, or in the layers below that --- those layers may know
the inferior is stopped, and be able to cache memory contents, etc.