gdbserver: thread getmsg err on mips[el]-linux

gdbserver: thread getmsg err on mips[el]-linux

[Atsushi Nemoto]

Hi.  When I tried to debug multi thread program with gdbserver
(6.3.50_20050104) on mips[el]-linux, I got following error:

[[gdbserver side]]
$ ./gdbserver host:50000 ./ttest
Process ./ttest created; pid = 3328
Listening on port 50000
Remote debugging from host 127.0.0.1
thread getmsg err: no event message for getmsg
Segmentation fault

[[gdb side]]
$ ./gdb ./ttest
GNU gdb 6.3.50_20050104
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "mipsel-unknown-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
 
(gdb) target remote localhost:50000
Remote debugging using localhost:50000
0x2aaa8b00 in ?? ()
(gdb) b main
Breakpoint 1 at 0x4008c4: file ttest.c, line 14.
(gdb) c
Continuing.
[New Thread 16384]
[Switching to Thread 16384]
 
Breakpoint 1, main (argc=1, argv=0x7fff7dd4) at ttest.c:14
14              ret = pthread_create(&id, NULL, func, 0);
(gdb) c
Continuing.
Watchdog has expired.  Target detached.


My environment is Debian sarge (glibc 2.3.2.ds1-20) with kernel
2.6.10.  Strangely, rarely I did not get this error on same
environment.  Also I did not get this error with kernel 2.4.28.

The error was printed in gdbserver/thread-db.c:thread_db_create_event().

  err = td_ta_event_getmsg (thread_agent, &msg);
  if (err != TD_OK)
    fprintf (stderr, "thread getmsg err: %s\n",
	     thread_db_err_str (err));

Why td_ta_event_getmsg() return TD_NOMSG?

If I modified above code as following, the error did not happen.

  do {
    err = td_ta_event_getmsg (thread_agent, &msg);
  } while (err == TD_NOMSG);

So it seems there is some race around here?


Here is a session log with debug_threads = 1.

[[gdbserver side]]
$ ./gdbserver host:50000 ./ttest
Process ./ttest created; pid = 3328
Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Listening on port 50000
[[gdb side]]
$ ./gdb ./ttest
GNU gdb 6.3.50_20050104
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "mipsel-unknown-linux-gnu"...Using host libthread_db library "/lib/libthread_db.so.1".
 
(gdb) target remote localhost:50000
Remote debugging using localhost:50000
0x2aaa8b00 in ?? ()
[[gdbserver side]]
Remote debugging from host 127.0.0.1
[[gdb side]]
(gdb) b main
Breakpoint 1 at 0x4008c4: file ttest.c, line 14.
(gdb) c
Continuing.
[New Thread 16384]
[Switching to Thread 16384]
 
Breakpoint 1, main (argc=1, argv=0x7fff7dd4) at ttest.c:14
14              ret = pthread_create(&id, NULL, func, 0);
[[gdbserver side]]
Writing 0d to 2aab7310
Writing 0d to 004008c4
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing 05 to 2aab7310
Writing 18 to 004008c4
Writing ffffff80 to 2ab54cb0
Writing 0d to 2ab11050
Writing 01 to 2ab54cc4
Writing 0d to 2aab7314
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 70 to 2aab7314
Writing 0d to 2aab7310
Writing 0d to 004008c4
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 05 to 2aab7310
Writing 18 to 004008c4
Writing 0d to 2aab7314
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 70 to 2aab7314
Writing 0d to 2aab7310
Writing 0d to 004008c4
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 05 to 2aab7310
Writing 18 to 004008c4
Writing 0d to 2aab7314
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 70 to 2aab7314
Writing 0d to 2aab7310
Writing 0d to 004008c4
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 05 to 2aab7310
Writing 18 to 004008c4
[[gdb side]]
(gdb) c
Continuing.
Watchdog has expired.  Target detached.
 
(gdb)
[[gdbserver side]]
Writing 0d to 004008c8
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Writing resume reply for 16384
 
Writing 21 to 004008c8
Writing 0d to 2aab7310
Writing 0d to 004008c4
Resuming, no pending status
Resuming process 3328 (continue, signal 0, stop not expected)
Checking for breakpoint.
Removed breakpoint.
  Got an event from 3328 (57f)
Hit a (non-reinsert) breakpoint.
Thread creation event.
thread getmsg err: no event message for getmsg
Segmentation fault


And this is the test program.

#include <stdio.h>
#include <pthread.h>
static void *func(void *arg)
{
        while (1)
                sleep(1);
}
 
int
main(int argc, char *argv[])
{
        pthread_t id;
        int ret;
        ret = pthread_create(&id, NULL, func, 0);
        printf("ID %d ret = %d\n", id, ret);
        while (1)
                sleep(1);
}


P.S.  I'm not on this ML.  Please CC to me.
---
Atsushi Nemoto

Re: gdbserver: thread getmsg err on mips[el]-linux

[Atsushi Nemoto]

>>>>> On Fri, 14 Jan 2005 17:48:02 +0900 (JST), Atsushi Nemoto <anemo@mba.ocn.ne.jp> said:
anemo> Hi.  When I tried to debug multi thread program with gdbserver
anemo> (6.3.50_20050104) on mips[el]-linux, I got following error:

anemo> [[gdbserver side]]
anemo> $ ./gdbserver host:50000 ./ttest
anemo> Process ./ttest created; pid = 3328
anemo> Listening on port 50000
anemo> Remote debugging from host 127.0.0.1
anemo> thread getmsg err: no event message for getmsg
anemo> Segmentation fault

Does anybody see above error?  Anyway, here is a patch for this
problem.  Please apply to CVS if no objection.  Thank you.


--- gdb-6.3.org/gdb/gdbserver/thread-db.c	2004-10-17 02:42:00.000000000 +0900
+++ gdb-6.3/gdb/gdbserver/thread-db.c	2005-01-27 12:19:29.000000000 +0900
@@ -21,6 +21,7 @@
    Foundation, Inc., 59 Temple Place - Suite 330,
    Boston, MA 02111-1307, USA.  */
 
+#include <unistd.h>
 #include "server.h"
 
 #include "linux-low.h"
@@ -142,6 +143,7 @@
   td_event_msg_t msg;
   td_err_e err;
   struct inferior_linux_data *tdata;
+  int timeout;
 
   if (debug_threads)
     fprintf (stderr, "Thread creation event.\n");
@@ -152,7 +154,13 @@
      In the LinuxThreads implementation, this is safe,
      because all events come from the manager thread
      (except for its own creation, of course).  */
-  err = td_ta_event_getmsg (thread_agent, &msg);
+  for (timeout = 0; timeout < 50000; timeout++)
+    {
+      err = td_ta_event_getmsg (thread_agent, &msg);
+      if (err != TD_NOMSG)
+	break;
+      usleep(1000);
+    }
   if (err != TD_OK)
     fprintf (stderr, "thread getmsg err: %s\n",
 	     thread_db_err_str (err));


P.S.  I'm not on this ML.  Please CC to me.
---
Atsushi Nemoto

Re: gdbserver: thread getmsg err on mips[el]-linux

[Atsushi Nemoto]

>>>>> On Fri, 14 Jan 2005 17:48:02 +0900 (JST), Atsushi Nemoto <anemo@mba.ocn.ne.jp> said:

anemo> Hi.  When I tried to debug multi thread program with gdbserver
anemo> (6.3.50_20050104) on mips[el]-linux, I got following error:

anemo> [[gdbserver side]]
anemo> ...
anemo> thread getmsg err: no event message for getmsg
anemo> Segmentation fault

I finally see what's wrong.  This is not gdb's fault but kernel's.
The fix is now in linux-mips CVS.

http://www.linux-mips.org/archives/linux-mips/2005-02/msg00103.html
http://www.linux-mips.org/archives/linux-cvs-patches/2005-02/msg00041.html

---
Atsushi Nemoto

Re: gdbserver: thread getmsg err on mips[el]-linux

[Daniel Jacobowitz]

On Wed, Feb 09, 2005 at 11:53:31PM +0900, Atsushi Nemoto wrote:
> >>>>> On Fri, 14 Jan 2005 17:48:02 +0900 (JST), Atsushi Nemoto <anemo@mba.ocn.ne.jp> said:
> 
> anemo> Hi.  When I tried to debug multi thread program with gdbserver
> anemo> (6.3.50_20050104) on mips[el]-linux, I got following error:
> 
> anemo> [[gdbserver side]]
> anemo> ...
> anemo> thread getmsg err: no event message for getmsg
> anemo> Segmentation fault
> 
> I finally see what's wrong.  This is not gdb's fault but kernel's.
> The fix is now in linux-mips CVS.
> 
> http://www.linux-mips.org/archives/linux-mips/2005-02/msg00103.html
> http://www.linux-mips.org/archives/linux-cvs-patches/2005-02/msg00041.html

Thanks!  Data cache flushing bugs are really nasty.

-- 
Daniel Jacobowitz
CodeSourcery, LLC