gcore and nptl threads on linux

gcore and nptl threads on linux

[J. Johnston]

There is a problem with implementing gcore on linux with nptl thread
support.  Under the linux nptl model, each thread is mapped to an
lwpid which is distinct from the pid of the process that created the
thread.

The current linux gcore code is merging the pid with the tid into an unsigned
long value.  This works ok in the old model because the tid and pids are only
16-bits in length.  This no longer can work because tids in the nptl model are actually
addresses and are not restricted to 16-bits.

The kernel has no knowledge of thread ids and generates the corefile prstatus notes
using the lwpids.  It would be cleaner if the gcore output emulated this
kernel output.  Now, the gcore code has access to the thread list, but at the
thread layer, there is no lwp info in the list (just pid and tid info).

At present, the only exposed interface for fetching the lwp from a thread ptid_t
is the to_pid_to_str() target function.  This converts the ptid to string form
and will contain the LWP in string form.  To fish this out requires parsing
the output and converting string to integer.  I have a working example
of this, but this is considered a bit of an ugly hack.

I would like to propose exposing the lwp_from_thread() functionality currently
present in the existing threading code.  Notably, thread-db.c has lwp_from_thread(),
sol-thread.c has thread_to_lwp(), and uw-thread.c has thr_to_lwp() which are all
static functions.  The default version of this target function would simply return the
ptid_t input to it.

This change would allow linux-proc.c to get the lwp where possible in a cleaner
fashion.

If nobody has any major objections to this proposal, I can post a patch shortly.

-- Jeff J.

Re: gcore and nptl threads on linux

[Daniel Jacobowitz]

On Tue, Mar 04, 2003 at 06:40:51PM -0500, J. Johnston wrote:
> There is a problem with implementing gcore on linux with nptl thread
> support.  Under the linux nptl model, each thread is mapped to an
> lwpid which is distinct from the pid of the process that created the
> thread.
> 
> The current linux gcore code is merging the pid with the tid into an 
> unsigned
> long value.  This works ok in the old model because the tid and pids are 
> only
> 16-bits in length.  This no longer can work because tids in the nptl model 
> are actually
> addresses and are not restricted to 16-bits.

I think this used to cause problems, too - it really should be just the
lwpid, I'd think.  For these cases at least.

> This change would allow linux-proc.c to get the lwp where possible in a 
> cleaner
> fashion.
> 
> If nobody has any major objections to this proposal, I can post a patch 
> shortly.

What do you plan to do when there is not a mapping, i.e. in an M:N
environment?  This interface assumes 1:1.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: gcore and nptl threads on linux

[J. Johnston]

Daniel Jacobowitz wrote:
> On Tue, Mar 04, 2003 at 06:40:51PM -0500, J. Johnston wrote:
> 
>>There is a problem with implementing gcore on linux with nptl thread
>>support.  Under the linux nptl model, each thread is mapped to an
>>lwpid which is distinct from the pid of the process that created the
>>thread.
>>
>>The current linux gcore code is merging the pid with the tid into an 
>>unsigned
>>long value.  This works ok in the old model because the tid and pids are 
>>only
>>16-bits in length.  This no longer can work because tids in the nptl model 
>>are actually
>>addresses and are not restricted to 16-bits.
> 
> 
> I think this used to cause problems, too - it really should be just the
> lwpid, I'd think.  For these cases at least.
> 
> 
>>This change would allow linux-proc.c to get the lwp where possible in a 
>>cleaner
>>fashion.
>>
>>If nobody has any major objections to this proposal, I can post a patch 
>>shortly.
> 
> 
> What do you plan to do when there is not a mapping, i.e. in an M:N
> environment?  This interface assumes 1:1.
> 

I would think the null_ptid would serve in such a case.

-- Jeff J.

Re: gcore and nptl threads on linux

[Daniel Jacobowitz]

On Wed, Mar 05, 2003 at 12:05:44PM -0500, J. Johnston wrote:
> 
> 
> Daniel Jacobowitz wrote:
> >On Tue, Mar 04, 2003 at 06:40:51PM -0500, J. Johnston wrote:
> >
> >>There is a problem with implementing gcore on linux with nptl thread
> >>support.  Under the linux nptl model, each thread is mapped to an
> >>lwpid which is distinct from the pid of the process that created the
> >>thread.
> >>
> >>The current linux gcore code is merging the pid with the tid into an 
> >>unsigned
> >>long value.  This works ok in the old model because the tid and pids are 
> >>only
> >>16-bits in length.  This no longer can work because tids in the nptl 
> >>model are actually
> >>addresses and are not restricted to 16-bits.
> >
> >
> >I think this used to cause problems, too - it really should be just the
> >lwpid, I'd think.  For these cases at least.
> >
> >
> >>This change would allow linux-proc.c to get the lwp where possible in a 
> >>cleaner
> >>fashion.
> >>
> >>If nobody has any major objections to this proposal, I can post a patch 
> >>shortly.
> >
> >
> >What do you plan to do when there is not a mapping, i.e. in an M:N
> >environment?  This interface assumes 1:1.
> >
> 
> I would think the null_ptid would serve in such a case.

I guess the issue is that we should be dumping the set of LWPs to the
generated core file, not the set of threads.  It seems to me like GDB
should be aware of the list of LWPs, and it shouldn't be hidden in each
individual thread package.

I may be alone in that opinion though.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: gcore and nptl threads on linux

[Andrew Cagney]

>> I would think the null_ptid would serve in such a case.
> 
> 
> I guess the issue is that we should be dumping the set of LWPs to the
> generated core file, not the set of threads.  It seems to me like GDB
> should be aware of the list of LWPs, and it shouldn't be hidden in each
> individual thread package.

You mean add them to the `struct thread_info' list?  Why not (ignoring 
technical realities for the moment :-)?

Andrew

Re: gcore and nptl threads on linux

[Daniel Jacobowitz]

On Wed, Mar 05, 2003 at 07:56:01PM -0500, Andrew Cagney wrote:
> 
> >>I would think the null_ptid would serve in such a case.
> >
> >
> >I guess the issue is that we should be dumping the set of LWPs to the
> >generated core file, not the set of threads.  It seems to me like GDB
> >should be aware of the list of LWPs, and it shouldn't be hidden in each
> >individual thread package.
> 
> You mean add them to the `struct thread_info' list?  Why not (ignoring 
> technical realities for the moment :-)?

Well, I wouldn't do it that way.  I haven't really designed this, so
bear with me if it has some squishy spots.

I think there should be two lists:
  all threads
  all lwps

Should the data structures be the same?  I don't know.  The mapping
between them would be defined by the thread stratum; its role would be
to take thread requests, convert them to LWP requests, and pass them
on.  The process stratum would be responsible for managing all of the
LWPs.

This has some advantages, I think.  Here's one: we would have a logical
interface for reporting an event from an LWP that doesn't currently
have a thread.  This happens in LinuxThreads, as I've mentioned
recently.  The thread stratum could see that the inferior ptid was just
an LWP id and pass the request along no questions asked.

Hmm, definitely some loose edges in that one.  Should both an LWP and a
thread have a regcache?  Might work.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: gcore and nptl threads on linux

[Andrew Cagney]

> On Wed, Mar 05, 2003 at 07:56:01PM -0500, Andrew Cagney wrote:
> 
>> 
> 
>> >>I would think the null_ptid would serve in such a case.
> 
>> >
>> >
>> >I guess the issue is that we should be dumping the set of LWPs to the
>> >generated core file, not the set of threads.  It seems to me like GDB
>> >should be aware of the list of LWPs, and it shouldn't be hidden in each
>> >individual thread package.
> 
>> 
>> You mean add them to the `struct thread_info' list?  Why not (ignoring 
>> technical realities for the moment :-)?
> 
> 
> Well, I wouldn't do it that way.  I haven't really designed this, so
> bear with me if it has some squishy spots.
> 
> I think there should be two lists:
>   all threads
>   all lwps

I believe in `zero, one, many':

- lwps
- processes
- threads (as in pthread)
- threads (as in a java interpreter thread)
- tasks (as in ada)

Each has something like:

- an architecture
- a target
- an owner?

For instance, an ada task might be implemented using a p-thread, which 
might in turn be implemented using an lwp-thread.  Only, the task is 
doing an rpc to java interpreter thread running in a separate process.

Each category can either maintain a local private database, or they can 
all share a common database.  If the info is more central, it becomes 
easier for the user to query/manipulate it.

> Should the data structures be the same?  I don't know.  The mapping
> between them would be defined by the thread stratum; its role would be
> to take thread requests, convert them to LWP requests, and pass them
> on.  The process stratum would be responsible for managing all of the
> LWPs.

Things to do today should include throwing out stratum (along with the 
bath water).

> This has some advantages, I think.  Here's one: we would have a logical
> interface for reporting an event from an LWP that doesn't currently
> have a thread.  This happens in LinuxThreads, as I've mentioned
> recently.  The thread stratum could see that the inferior ptid was just
> an LWP id and pass the request along no questions asked.

?  That sounds a bit up-side-down, shouldn't events be propogating up - 
lwp gets to see them before thread?

> Hmm, definitely some loose edges in that one.  Should both an LWP and a
> thread have a regcache?  Might work.

Don't forget that a regcache is just a local performance optimization - 
a look-a-side buffer.

Andrew

Re: gcore and nptl threads on linux

[Daniel Jacobowitz]

On Wed, Mar 05, 2003 at 08:44:49PM -0500, Andrew Cagney wrote:
> >On Wed, Mar 05, 2003 at 07:56:01PM -0500, Andrew Cagney wrote:
> >
> >>
> >
> >>>>I would think the null_ptid would serve in such a case.
> >
> >>>
> >>>
> >>>I guess the issue is that we should be dumping the set of LWPs to the
> >>>generated core file, not the set of threads.  It seems to me like GDB
> >>>should be aware of the list of LWPs, and it shouldn't be hidden in each
> >>>individual thread package.
> >
> >>
> >>You mean add them to the `struct thread_info' list?  Why not (ignoring 
> >>technical realities for the moment :-)?
> >
> >
> >Well, I wouldn't do it that way.  I haven't really designed this, so
> >bear with me if it has some squishy spots.
> >
> >I think there should be two lists:
> >  all threads
> >  all lwps
> 
> I believe in `zero, one, many':
> 
> - lwps
> - processes
> - threads (as in pthread)
> - threads (as in a java interpreter thread)
> - tasks (as in ada)

Makes sense.

> Each has something like:
> 
> - an architecture
> - a target
> - an owner?

Not sure...

> >Should the data structures be the same?  I don't know.  The mapping
> >between them would be defined by the thread stratum; its role would be
> >to take thread requests, convert them to LWP requests, and pass them
> >on.  The process stratum would be responsible for managing all of the
> >LWPs.
> 
> Things to do today should include throwing out stratum (along with the 
> bath water).

Good luck, it's been on my list for a year and a half or so and every
time I try I get scared and find something else to do.

> >This has some advantages, I think.  Here's one: we would have a logical
> >interface for reporting an event from an LWP that doesn't currently
> >have a thread.  This happens in LinuxThreads, as I've mentioned
> >recently.  The thread stratum could see that the inferior ptid was just
> >an LWP id and pass the request along no questions asked.
> 
> ?  That sounds a bit up-side-down, shouldn't events be propogating up - 
> lwp gets to see them before thread?

I disconnected from myself there.  Event -> lwp layer -> thread layer;
GDB request -> thread layer -> lwp layer.

> >Hmm, definitely some loose edges in that one.  Should both an LWP and a
> >thread have a regcache?  Might work.
> 
> Don't forget that a regcache is just a local performance optimization - 
> a look-a-side buffer.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: gcore and nptl threads on linux

[J. Johnston]

Andrew Cagney wrote:
> 
>>> I would think the null_ptid would serve in such a case.
>>
>>
>>
>> I guess the issue is that we should be dumping the set of LWPs to the
>> generated core file, not the set of threads.  It seems to me like GDB
>> should be aware of the list of LWPs, and it shouldn't be hidden in each
>> individual thread package.
> 
> 
> You mean add them to the `struct thread_info' list?  Why not (ignoring 
> technical realities for the moment :-)?
> 

How about a second list made up of thread_info structs for lwps?  That way, the current
thread routines wouldn't have to constantly validate whether the list item was
a thread or lwp.  This also would simplify the numbering system.  There
could be equivalent lwp routines for accessing the list (e.g. iterate_over_lwps() ).

-- Jeff J.

Re: gcore and nptl threads on linux

[Andrew Cagney]

>> You mean add them to the `struct thread_info' list?  Why not (ignoring technical realities for the moment :-)?
> 
> 
> How about a second list made up of thread_info structs for lwps?  That way, the current
> thread routines wouldn't have to constantly validate whether the list item was
> a thread or lwp.  This also would simplify the numbering system.  There
> could be equivalent lwp routines for accessing the list (e.g. iterate_over_lwps() ).

Something like that.  Is a new term needed though?

Have the thread and lwp code each have their own instance of a `struct 
tpid_info' list (struct context_info list)?

Andrew

Re: gcore and nptl threads on linux

[Daniel Jacobowitz]

On Thu, Mar 06, 2003 at 03:17:33PM -0500, Andrew Cagney wrote:
> 
> >>You mean add them to the `struct thread_info' list?  Why not (ignoring 
> >>technical realities for the moment :-)?
> >
> >
> >How about a second list made up of thread_info structs for lwps?  That 
> >way, the current
> >thread routines wouldn't have to constantly validate whether the list item 
> >was
> >a thread or lwp.  This also would simplify the numbering system.  There
> >could be equivalent lwp routines for accessing the list (e.g. 
> >iterate_over_lwps() ).
> 
> Something like that.  Is a new term needed though?
> 
> Have the thread and lwp code each have their own instance of a `struct 
> tpid_info' list (struct context_info list)?

Hmm, I'm not sure.  Let's take a little while to think about a better
name?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: gcore and nptl threads on linux

[J. Johnston]

Daniel Jacobowitz wrote:
> On Thu, Mar 06, 2003 at 03:17:33PM -0500, Andrew Cagney wrote:
> 
>>>>You mean add them to the `struct thread_info' list?  Why not (ignoring 
>>>>technical realities for the moment :-)?
>>>
>>>
>>>How about a second list made up of thread_info structs for lwps?  That 
>>>way, the current
>>>thread routines wouldn't have to constantly validate whether the list item 
>>>was
>>>a thread or lwp.  This also would simplify the numbering system.  There
>>>could be equivalent lwp routines for accessing the list (e.g. 
>>>iterate_over_lwps() ).
>>
>>Something like that.  Is a new term needed though?
>>
>>Have the thread and lwp code each have their own instance of a `struct 
>>tpid_info' list (struct context_info list)?
> 
> 
> Hmm, I'm not sure.  Let's take a little while to think about a better
> name?
> 

How about uow_info with uow standing for "unit of work"?

-- Jeff J.

Re: gcore and nptl threads on linux

[Andrew Cagney]

>> Hmm, I'm not sure.  Let's take a little while to think about a better
>> name?

> 
> 
> How about uow_info with uow standing for "unit of work"?

Throws up in bucket ... :-)

Re: gcore and nptl threads on linux

[Andrew Cagney]

> Daniel Jacobowitz wrote:
> On Thu, Mar 06, 2003 at 03:17:33PM -0500, Andrew Cagney wrote:
> 
> You mean add them to the `struct thread_info' list?  Why not (ignoring technical realities for the moment :-)?
> 
> 
> How about a second list made up of thread_info structs for lwps?  That way, the current
> thread routines wouldn't have to constantly validate whether the list item was
> a thread or lwp.  This also would simplify the numbering system.  There
> could be equivalent lwp routines for accessing the list (e.g. iterate_over_lwps() ).
> 
> Something like that.  Is a new term needed though?
> 
> Have the thread and lwp code each have their own instance of a `struct tpid_info' list (struct context_info list)?
> 
> 
> Hmm, I'm not sure.  Let's take a little while to think about a better
> name?
> 
> 
> How about uow_info with uow standing for "unit of work"?

This came up once before:

http://sources.redhat.com/ml/gdb/2001-03/msg00126.html

shall baring major protest go with `struct ptid_info'?

Andrew