Problem with multiple threads using gdbserver on x86.

Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]

hi, 
      
this is regarding debugging multi-threaded applications using gdbserver.
I am facing a weird situation with respect to the same. When i try and
run gdbserver on a multi-threaded application and try to debug the same,
gdb is unable to stop all the threads on hitting a breakpoint in one of
the threads. 
	
the steps i followed are the following. 
$>gdbserver localhost:1234 thr
$>gdb thr

(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0x40000be0 in ?? ()

(gdb) set remotelogfile logfile
(gdb) b main
Breakpoint 1 at 0x80484b0: file thr.c, line 11.
(gdb) c
Continuing.
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Ignoring packet error, continuing...

Breakpoint 1, main (argc=1, argv=0xbffff144) at thr.c:11
11           printf("Creating...\n");
(gdb) n
12           pthread_create( &id, NULL, work2, ((void*) 0) );
(gdb) n
13           printf("Created thread.\n"); fflush(stdout);
(gdb) n
15           printf("Waiting...\n");


At this point in time if i do an info threads the only thread it shows
me is the main thread though in gdb it shows me both the threads. 
At the same point of time when the main thread is stopped at this
position the thread which has just been created runs away. ! 
and on doing an info threads i do not get the information about the 
newly created thread. 

Is this a limitation of gdbserver with respect to kernel space threads.
 I saw the comment in thread-db.c in the function
thread_db_find_new_threads. which says "iterate" over user space threads
??
The foll. is the information regarding my machine / environment. 

RedHat linux 9.0 .  gdb v 5.3 gcc version 3.2.2 20030222
GNU gdb Red Hat Linux (5.3post-0.20021129.18rh)


I wonder if anyone else is getting a similar problem ..I am attaching
the 2 files that were used for this purpose. 

build the 2 files as
 gcc -g new.c thr.c -lpthread (obviously !)


regards
Ramana

P.S. the same works fine on a native version of gdb. 



 

[-- Attachment #2: new.c --]
[-- Type: text/x-c, Size: 154 bytes --]

#include <stdio.h>
void *work2(void *foo)
{
     while ((int)foo<5) {
	  printf("got %d\n", (int) foo++);
	  sleep(2);
     }
     return (void *)110;
}


[-- Attachment #3: thr.c --]
[-- Type: text/x-c, Size: 457 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

extern int work2(void *);
int main(int argc, char **argv)
{
     pthread_t id; 
     int  retval;

     printf("Creating...\n");
     pthread_create( &id, NULL, work2, ((void*) 0) );
     printf("Created thread.\n"); fflush(stdout);

     printf("Waiting...\n");
     pthread_join(id,(void **)&retval);
     printf("Joined with retval = %d\n",retval);

     return 0;
}

Re: Problem with multiple threads using gdbserver on x86.

[Daniel Jacobowitz]

On Mon, Jun 23, 2003 at 04:26:56PM +0530, Ramana Radhakrishnan wrote:
> hi, 
>       
> this is regarding debugging multi-threaded applications using gdbserver.
> I am facing a weird situation with respect to the same. When i try and
> run gdbserver on a multi-threaded application and try to debug the same,
> gdb is unable to stop all the threads on hitting a breakpoint in one of
> the threads. 
> 	
> the steps i followed are the following. 
> $>gdbserver localhost:1234 thr
> $>gdb thr
> 
> (gdb) target remote localhost:1234
> Remote debugging using localhost:1234
> 0x40000be0 in ?? ()
> 
> (gdb) set remotelogfile logfile
> (gdb) b main
> Breakpoint 1 at 0x80484b0: file thr.c, line 11.
> (gdb) c
> Continuing.
> Ignoring packet error, continuing...
> Ignoring packet error, continuing...
> Ignoring packet error, continuing...
> 
> Breakpoint 1, main (argc=1, argv=0xbffff144) at thr.c:11
> 11           printf("Creating...\n");
> (gdb) n
> 12           pthread_create( &id, NULL, work2, ((void*) 0) );
> (gdb) n
> 13           printf("Created thread.\n"); fflush(stdout);
> (gdb) n
> 15           printf("Waiting...\n");
> 
> 
> At this point in time if i do an info threads the only thread it shows
> me is the main thread though in gdb it shows me both the threads. 
> At the same point of time when the main thread is stopped at this
> position the thread which has just been created runs away. ! 
> and on doing an info threads i do not get the information about the 
> newly created thread. 
> 
> Is this a limitation of gdbserver with respect to kernel space threads.
>  I saw the comment in thread-db.c in the function
> thread_db_find_new_threads. which says "iterate" over user space threads
> ??
> The foll. is the information regarding my machine / environment. 
> 
> RedHat linux 9.0 .  gdb v 5.3 gcc version 3.2.2 20030222
> GNU gdb Red Hat Linux (5.3post-0.20021129.18rh)
> 
> 
> I wonder if anyone else is getting a similar problem ..I am attaching
> the 2 files that were used for this purpose. 

It works perfectly fine for me.  I recommend you ask Red Hat; I know
that gdbserver does not yet support NPTL so it's possible that they
have disabled threading.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

> It works perfectly fine for me.  I recommend you ask Red Hat; I know
> that gdbserver does not yet support NPTL so it's possible that they
> have disabled threading.

I tried it again with gdbserver and gdb-5.3 rebuilt from the sources . 
with no luck . any other ideas. 

regards
Ramana

Re: Problem with multiple threads using gdbserver on x86.

[Daniel Jacobowitz]

On Mon, Jun 23, 2003 at 06:40:40PM +0530, Ramana Radhakrishnan wrote:
> 
> > It works perfectly fine for me.  I recommend you ask Red Hat; I know
> > that gdbserver does not yet support NPTL so it's possible that they
> > have disabled threading.
> 
> I tried it again with gdbserver and gdb-5.3 rebuilt from the sources . 
> with no luck . any other ideas. 

That won't support NPTL either :)

I suspect gdbserver + threads is not usable on RH9 until someone
patches gdbserver to support it.  I don't have a system to try this on
myself, so someone else will have to do it.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

On Mon, 2003-06-23 at 18:58, Daniel Jacobowitz wrote:
> I dunno.  Probably just providing a stub for ps_get_thread_area
> (spelling?) will be enough to make it work.
there does not seem to be any reference to this function anywhere in the
gdbserver code that i have. maybe  i have a very old version but the
source has not changed for the last 12 months or so for the thread-db.c
file. just checked that on cvs. the error seems to come in
thread_db_init() and precisely when it tries to find the new threads. 


could someone explain why this code sits inside such a function . isnt
this strange??
static void
linux_look_up_symbols (void)
{
#ifdef USE_THREAD_DB
  if (using_threads)
    return;

  using_threads = thread_db_init ();
#endif
}



	
> 
> On Mon, Jun 23, 2003 at 06:59:51PM +0530, ramana wrote:
> > could you give me a hint as to what needs to be done. i am sure i can find 
> > free time to patch it up myself. would give me a chance to fool around with 
> > some gdb stuff, things that i have been itching to do .
> > regards
> > Ramana
> > 
> > On Mon, 23 Jun 2003 09:16:06 -0400, Daniel Jacobowitz <drow@mvista.com> 
> > wrote:
> > 
> > >On Mon, Jun 23, 2003 at 06:40:40PM +0530, Ramana Radhakrishnan wrote:
> > >>
> > >>> It works perfectly fine for me.  I recommend you ask Red Hat; I know
> > >>> that gdbserver does not yet support NPTL so it's possible that they
> > >>> have disabled threading.
> > >>
> > >>I tried it again with gdbserver and gdb-5.3 rebuilt from the sources . 
> > >>with no luck . any other ideas.
> > >
> > >That won't support NPTL either :)
> > >
> > >I suspect gdbserver + threads is not usable on RH9 until someone
> > >patches gdbserver to support it.  I don't have a system to try this on
> > >myself, so someone else will have to do it.
> > >
> > 
> > 
> > 
> > -- 
> > Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
> > 

Re: Problem with multiple threads using gdbserver on x86.

[Daniel Jacobowitz]

On Mon, Jun 23, 2003 at 08:29:42PM +0530, Ramana Radhakrishnan wrote:
> On Mon, 2003-06-23 at 18:58, Daniel Jacobowitz wrote:
> > I dunno.  Probably just providing a stub for ps_get_thread_area
> > (spelling?) will be enough to make it work.
> there does not seem to be any reference to this function anywhere in the
> gdbserver code that i have. maybe  i have a very old version but the
> source has not changed for the last 12 months or so for the thread-db.c
> file. just checked that on cvs. the error seems to come in
> thread_db_init() and precisely when it tries to find the new threads. 

You will have to look in the source to libthread_db, which is a part of
glibc and in this case NPTL, in order to figure out why this is not
working.

Debugging libthread_db is, in general, a pain in the ass.


-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

On Mon, 2003-06-23 at 20:29, Daniel Jacobowitz wrote:

> You will have to look in the source to libthread_db, which is a part of
> glibc and in this case NPTL, in order to figure out why this is not
> working.
> 
> Debugging libthread_db is, in general, a pain in the ass.
> 

Thanx for the info. will look into the same
.
 but that still does not answer my question regarding
linux_look_up_symbols.. I was under the 
impression that looking up symbols was a file format  / debug format
specific thing and not an OS specific functionality and would like to
know the reason behind having a function like look_up_symbol in the
target_ops structure and seem to use it to initialize the thread
information in the implementation ?




regards
Ramana


--
God made machine language - the rest is the work of man. 
				

Re: Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

hi all,

this is with respect to my previous post. i would rather like to
rephrase my question. 

since symbol lookups are not necessarily a functionality of gdbserver
(infact gdbserver would very well work with a stripped down version of
the final executable) why is it that the target_ops structure has a
member named look_up_symbols ? or am i missing something very very basic
over here. ?and the linux implementation seems to capture information
regarding threads in linux_low.c

or is it one of the cases of the implementation not being related to the
intuitive sense one tries to make out of the name of the interface. ?

regards
Ramana

Re: Problem with multiple threads using gdbserver on x86.

[Daniel Jacobowitz]

On Mon, Jun 23, 2003 at 09:56:01PM +0530, Ramana Radhakrishnan wrote:
> hi all,
> 
> this is with respect to my previous post. i would rather like to
> rephrase my question. 
> 
> since symbol lookups are not necessarily a functionality of gdbserver
> (infact gdbserver would very well work with a stripped down version of
> the final executable) why is it that the target_ops structure has a
> member named look_up_symbols ? or am i missing something very very basic
> over here. ?and the linux implementation seems to capture information
> regarding threads in linux_low.c
> 
> or is it one of the cases of the implementation not being related to the
> intuitive sense one tries to make out of the name of the interface. ?

Gdbserver needs to know the location of the symbols.  The way it does
this is by querying GDB on the host side for the addresses.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Problem with multiple threads using gdbserver on x86.

[Ramana Radhakrishnan]

hi daniel,

> Gdbserver needs to know the location of the symbols.  The way it does
> this is by querying GDB on the host side for the addresses.

but the only piece of code within the target specific function seems to
be getting information regarding threads .maybe i am a bit dense but
this really doesnt look like querying symbol specific information from
gdb. 

The following snippet is the code from 

linux_look_up_symbols()
{
#ifdef USE_THREAD_DB
  if (using_threads)
    return;

  using_threads = thread_db_init ();
#endif
}

regards
Ramana

Re: Problem with multiple threads using gdbserver on x86.

[Daniel Jacobowitz]

On Mon, Jun 23, 2003 at 10:05:10PM +0530, Ramana Radhakrishnan wrote:
> hi daniel,
> 
> > Gdbserver needs to know the location of the symbols.  The way it does
> > this is by querying GDB on the host side for the addresses.
> 
> but the only piece of code within the target specific function seems to
> be getting information regarding threads .maybe i am a bit dense but
> this really doesnt look like querying symbol specific information from
> gdb. 
> 
> The following snippet is the code from 
> 
> linux_look_up_symbols()
> {
> #ifdef USE_THREAD_DB
>   if (using_threads)
>     return;
> 
>   using_threads = thread_db_init ();
> #endif
> }

Look at thread_db_init and the code in libthread_db in the glibc
source.  Then look at ps_pglobal_lookup in proc-service.c.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer