multithreaded programs on arm-linux

multithreaded programs on arm-linux

[frank]

Hello,

after having failed to solve my problems remote debugging a
program linked against shared libraries i tried to work
around it be running the gdb on the target system.

This works for single threaded programs but i ran into
problems with threaded programs.

GNU gdb 6.3
[...]
This GDB was configured as "arm-linux"...Using host libthread_db library
"/lib/libthread_db.so.1".

(gdb) r
Program received signal SIG32, Real-time event 32.
renderThread starts up
0x40680aac in sigsuspend () from /lib/libc.so.6
(gdb) c
Continuing.

Program received signal SIGINT, Interrupt.
0x4071a1fc in select () from /lib/libc.so.6
(gdb) info threads
(gdb)

libthread_db is available on the system (/lib/libthread_db.so.1).
gdb successfully ran "thread_db_load()".

Any ideas how i can debug this?

Regards,
  Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Tue, Nov 01, 2005 at 07:10:51PM +0100, frank@betaversion.net wrote:
> libthread_db is available on the system (/lib/libthread_db.so.1).
> gdb successfully ran "thread_db_load()".
> 
> Any ideas how i can debug this?

Has libpthread.so been stripped?  Debug info can be removed, but you
must leave the static symbol table (.symtab) to use native libthread_db.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[frank]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Tue, Nov 01, 2005 at 07:10:51PM +0100, frank@betaversion.net wrote:
>> libthread_db is available on the system (/lib/libthread_db.so.1).
>> gdb successfully ran "thread_db_load()".
>>
>> Any ideas how i can debug this?
>
> Has libpthread.so been stripped?  Debug info can be removed, but you
> must leave the static symbol table (.symtab) to use native libthread_db.

No, it hasn't been stripped. And the the section .symtab also seems
to be available.

$ file libthread_db.so.1
libthread_db.so.1: ELF 32-bit LSB shared object, ARM, version 1 (ARM), not
stripped

$ readelf -a libthread_db.so.1
Section Headers:
[...]
[26] .symtab           SYMTAB          00000000 0047ec 000f80 10     27 189  4
[...]
Symbol table '.symtab' contains 248 entries:
    Num:    Value  Size Type    Bind   Vis      Ndx Name
      0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
      1: 00000100     0 SECTION LOCAL  DEFAULT    1
      2: 00000120     0 SECTION LOCAL  DEFAULT    2
      3: 0000041c     0 SECTION LOCAL  DEFAULT    3
[...]
    245: 00003254   268 FUNC    GLOBAL DEFAULT   13 td_ta_clear_event
    246: 000022a0   204 FUNC    GLOBAL DEFAULT   13 td_thr_setfpregs
    247: 00000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
[...]

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Andreas Schwab]

frank@betaversion.net writes:

> No, it hasn't been stripped. And the the section .symtab also seems
> to be available.

Does it contain the symbol nptl_version?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: multithreaded programs on arm-linux

[frank]

Quoting Andreas Schwab <schwab@suse.de>:

> frank@betaversion.net writes:
>> No, it hasn't been stripped. And the the section .symtab also seems
>> to be available.
>
> Does it contain the symbol nptl_version?

No it doesn't.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Andreas Schwab]

frank@betaversion.net writes:

> Quoting Andreas Schwab <schwab@suse.de>:
>
>> frank@betaversion.net writes:
>>> No, it hasn't been stripped. And the the section .symtab also seems
>>> to be available.
>>
>> Does it contain the symbol nptl_version?
>
> No it doesn't.

Then it is not usable.  All symbols must be preserved, even local ones.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Wed, Nov 02, 2005 at 02:28:29PM +0100, Andreas Schwab wrote:
> frank@betaversion.net writes:
> 
> > Quoting Andreas Schwab <schwab@suse.de>:
> >
> >> frank@betaversion.net writes:
> >>> No, it hasn't been stripped. And the the section .symtab also seems
> >>> to be available.
> >>
> >> Does it contain the symbol nptl_version?
> >
> > No it doesn't.
> 
> Then it is not usable.  All symbols must be preserved, even local ones.

While this is true, a LinuxThreads library won't require
nptl_version...

And an ARM NPTL library will require additional gdbserver changes.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Andreas Schwab]

Daniel Jacobowitz <drow@false.org> writes:

> While this is true, a LinuxThreads library won't require
> nptl_version...

I was assuming NPTL, because AFAIK LinuxThreads does not have the symbol
stripping problem (the dynamic symbol table of the library should be good
enough).

If this is indeed LinuxThreads, then it is most likely a different
problem.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Wed, Nov 02, 2005 at 03:33:17PM +0100, Andreas Schwab wrote:
> Daniel Jacobowitz <drow@false.org> writes:
> 
> > While this is true, a LinuxThreads library won't require
> > nptl_version...
> 
> I was assuming NPTL, because AFAIK LinuxThreads does not have the symbol
> stripping problem (the dynamic symbol table of the library should be good
> enough).

Oh.  Huh - I didn't realize that.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Wed, Nov 02, 2005 at 03:33:17PM +0100, Andreas Schwab wrote:
>> Daniel Jacobowitz <drow@false.org> writes:
>>
>> > While this is true, a LinuxThreads library won't require
>> > nptl_version...
>>
>> I was assuming NPTL, because AFAIK LinuxThreads does not have the symbol
>> stripping problem (the dynamic symbol table of the library should be good
>> enough).
>
> Oh.  Huh - I didn't realize that.

Does anyone have an idea left how i could end up with
a thread enabled gdb?

What steps could i try to narrow down the problem?

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Wed, Nov 02, 2005 at 04:55:58PM +0100, Frank Meerkoetter wrote:
> Quoting Daniel Jacobowitz <drow@false.org>:
> 
> >On Wed, Nov 02, 2005 at 03:33:17PM +0100, Andreas Schwab wrote:
> >>Daniel Jacobowitz <drow@false.org> writes:
> >>
> >>> While this is true, a LinuxThreads library won't require
> >>> nptl_version...
> >>
> >>I was assuming NPTL, because AFAIK LinuxThreads does not have the symbol
> >>stripping problem (the dynamic symbol table of the library should be good
> >>enough).
> >
> >Oh.  Huh - I didn't realize that.
> 
> Does anyone have an idea left how i could end up with
> a thread enabled gdb?
> 
> What steps could i try to narrow down the problem?

How about answering whether you are using NPTL or LinuxThreads first?

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Wed, Nov 02, 2005 at 04:55:58PM +0100, Frank Meerkoetter wrote:
>> Quoting Daniel Jacobowitz <drow@false.org>:
>>
>> >On Wed, Nov 02, 2005 at 03:33:17PM +0100, Andreas Schwab wrote:
>> >>Daniel Jacobowitz <drow@false.org> writes:
>> >>
>> >>> While this is true, a LinuxThreads library won't require
>> >>> nptl_version...
>> >>
>> >>I was assuming NPTL, because AFAIK LinuxThreads does not have the symbol
>> >>stripping problem (the dynamic symbol table of the library should be good
>> >>enough).
>> >
>> >Oh.  Huh - I didn't realize that.
>>
>> Does anyone have an idea left how i could end up with
>> a thread enabled gdb?
>>
>> What steps could i try to narrow down the problem?
>
> How about answering whether you are using NPTL or LinuxThreads first?

Sorry, that was a misunderstanding on my side. I'm running
LinuxThreads.

# uname -a
Linux (none) 2.4.19-rmk7-omap1 #97 Wed Sep 21 15:59:38 CEST 2005 armv4l 
unknown

# /lib/libc.so.6
GNU C Library stable release version 2.2.5, by Roland McGrath et al.
Copyright (C) 1992-2001, 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 2.95.3 20010315 (release).
Compiled on a Linux 2.4.19 system on 2003-10-30.
Available extensions:
         GNU libio by Per Bothner
         crypt add-on version 2.1 by Michael Glad and others
         linuxthreads-0.9 by Xavier Leroy
         BIND-8.2.3-T5B
         libthread_db work sponsored by Alpha Processor Inc
         NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Thu, Nov 03, 2005 at 09:33:06AM +0100, Frank Meerkoetter wrote:
> Sorry, that was a misunderstanding on my side. I'm running
> LinuxThreads.

OK, let's start at the top then.

1.  Is gdbserver linked to libthread_db.so during build?

2.  Does gdbserver output anything while trying to debug a
multithreaded program?

3.  What does the complete remote protocol session look like?
"set debug remote 1", "target host:port", "break main", "continue".

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

Hi,

> OK, let's start at the top then.
>
> 1.  Is gdbserver linked to libthread_db.so during build?

I'm not using the gdbserver but rather running the gdb on
the target. The reason is that i didn't managed to
remote debug a program which is dynamically linked.

I assume gdb is linked against libthread_db.so

frank@linux:~/gdb/gdb-host-arm/gdb> arm-linux-readelf --syms gdb | grep
thread_db
   1896: 0005be00    28 FUNC    GLOBAL DEFAULT   13 thread_db_init
   3751: 0005d7f4    80 FUNC    GLOBAL DEFAULT   13 _initialize_thread_db
    647: 0005b6e4   400 FUNC    LOCAL  DEFAULT   13 thread_db_err_str
    649: 0005b874   168 FUNC    LOCAL  DEFAULT   13 thread_db_state_str
    653: 0005bac4   168 FUNC    LOCAL  DEFAULT   13 thread_db_map_id2thr
    656: 0005bb6c   140 FUNC    LOCAL  DEFAULT   13 thread_db_get_info
    662: 0005be58  1044 FUNC    LOCAL  DEFAULT   13 thread_db_load
    686: 0005c5d0   428 FUNC    LOCAL  DEFAULT   13 thread_db_new_objfile
    687: 0005d3e8   108 FUNC    LOCAL  DEFAULT   13 thread_db_find_new_threads
    688: 0026d4fc     0 NOTYPE  LOCAL  DEFAULT   22 using_thread_db
    690: 0026d3b0     0 NOTYPE  LOCAL  DEFAULT   22 thread_db_ops
    692: 0005c908    76 FUNC    LOCAL  DEFAULT   13 thread_db_attach
    694: 0005c9e8    88 FUNC    LOCAL  DEFAULT   13 thread_db_detach
    696: 0005ca74   168 FUNC    LOCAL  DEFAULT   13 thread_db_resume
    698: 0005cca0   304 FUNC    LOCAL  DEFAULT   13 thread_db_wait
    699: 0005cdd0   204 FUNC    LOCAL  DEFAULT   13 thread_db_xfer_memory
    700: 0005ce9c   272 FUNC    LOCAL  DEFAULT   13 thread_db_fetch_registers
    701: 0005cfac   340 FUNC    LOCAL  DEFAULT   13 thread_db_store_registers
    702: 0005d100    52 FUNC    LOCAL  DEFAULT   13 thread_db_kill
    703: 0005d134    96 FUNC    LOCAL  DEFAULT   13 thread_db_create_inferior
    704: 0005d194    68 FUNC    LOCAL  DEFAULT   13
thread_db_post_startup_inferior
    705: 0005d1d8    80 FUNC    LOCAL  DEFAULT   13 thread_db_mourn_inferior
    706: 0005d228   248 FUNC    LOCAL  DEFAULT   13 thread_db_thread_alive
    709: 0005d454   292 FUNC    LOCAL  DEFAULT   13 thread_db_pid_to_str
    710: 0005d578   372 FUNC    LOCAL  DEFAULT   13
thread_db_get_thread_local_address
    711: 0005d6ec   264 FUNC    LOCAL  DEFAULT   13 init_thread_db_ops
   7239: 0005be00    28 FUNC    GLOBAL DEFAULT   13 thread_db_init
   9093: 0005d7f4    80 FUNC    GLOBAL DEFAULT   13 _initialize_thread_db

The libthread_db.so of the toolchain and the target have matching md5sums.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Thu, Nov 03, 2005 at 04:00:09PM +0100, Frank Meerkoetter wrote:
> Quoting Daniel Jacobowitz <drow@false.org>:
> 
> Hi,
> 
> >OK, let's start at the top then.
> >
> >1.  Is gdbserver linked to libthread_db.so during build?
> 
> I'm not using the gdbserver but rather running the gdb on
> the target. The reason is that i didn't managed to
> remote debug a program which is dynamically linked.
> 
> I assume gdb is linked against libthread_db.so

GDB uses dlopen to get at libthread_db.so.  Does it do that?  Does
initialization fail?

You'll need to attach a debugger to gdb and look at thread_db_init.
The first call will generally fail but a later one should succeed.


-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:
[...]
>> I'm not using the gdbserver but rather running the gdb on
>> the target. The reason is that i didn't managed to
>> remote debug a program which is dynamically linked.
>>
>> I assume gdb is linked against libthread_db.so
>
> GDB uses dlopen to get at libthread_db.so.  Does it do that?  Does
> initialization fail?

thread_db_load() which contains the dlopen() call is successfull run.
The end of thread_db_load() is reached and 1 is returned.

> You'll need to attach a debugger to gdb and look at thread_db_init.
> The first call will generally fail but a later one should succeed.

frank@linux:~/gdb/gdb-6.3> global thread_db_init
gdb/gdbserver/thread-db.c
gdb/thread-db.c

thread_db_init of the gdbserver could fail but not thread_db_init
of the gdb.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Thu, Nov 03, 2005 at 04:51:07PM +0100, Frank Meerkoetter wrote:
> Quoting Daniel Jacobowitz <drow@false.org>:
> [...]
> >>I'm not using the gdbserver but rather running the gdb on
> >>the target. The reason is that i didn't managed to
> >>remote debug a program which is dynamically linked.
> >>
> >>I assume gdb is linked against libthread_db.so
> >
> >GDB uses dlopen to get at libthread_db.so.  Does it do that?  Does
> >initialization fail?
> 
> thread_db_load() which contains the dlopen() call is successfull run.
> The end of thread_db_load() is reached and 1 is returned.
> 
> >You'll need to attach a debugger to gdb and look at thread_db_init.
> >The first call will generally fail but a later one should succeed.
> 
> frank@linux:~/gdb/gdb-6.3> global thread_db_init
> gdb/gdbserver/thread-db.c
> gdb/thread-db.c
> 
> thread_db_init of the gdbserver could fail but not thread_db_init
> of the gdb.

Try thread_db_new_objfile.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Thu, Nov 03, 2005 at 04:51:07PM +0100, Frank Meerkoetter wrote:
>> >You'll need to attach a debugger to gdb and look at thread_db_init.
>> >The first call will generally fail but a later one should succeed.
>>
>> frank@linux:~/gdb/gdb-6.3> global thread_db_init
>> gdb/gdbserver/thread-db.c
>> gdb/thread-db.c
>>
>> thread_db_init of the gdbserver could fail but not thread_db_init
>> of the gdb.
>
> Try thread_db_new_objfile.

thread_db_new_objfile() is called two times.
On the first run it prints the library name and
than branches on the falg using_thread_db
to the label quit.
There it returns without executing target_new_objfile_chain (objfile).

The second time the same path is followed but without branching into
the block protected by the flag dejavu.

In neither cases is this line executed:
    /* Now attempt to open a connection to the thread library.  */
    err = td_ta_new_p (&proc_handle, &thread_agent);

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Thu, Nov 03, 2005 at 05:28:09PM +0100, Frank Meerkoetter wrote:
> thread_db_new_objfile() is called two times.
> On the first run it prints the library name and
> than branches on the falg using_thread_db
> to the label quit.
> There it returns without executing target_new_objfile_chain (objfile).
> 
> The second time the same path is followed but without branching into
> the block protected by the flag dejavu.
> 
> In neither cases is this line executed:
>    /* Now attempt to open a connection to the thread library.  */
>    err = td_ta_new_p (&proc_handle, &thread_agent);

Then what set using_thread_db?

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Thu, Nov 03, 2005 at 05:28:09PM +0100, Frank Meerkoetter wrote:
>> thread_db_new_objfile() is called two times.
>> On the first run it prints the library name and
>> than branches on the falg using_thread_db
>> to the label quit.
>> There it returns without executing target_new_objfile_chain (objfile).
>>
>> The second time the same path is followed but without branching into
>> the block protected by the flag dejavu.
>>
>> In neither cases is this line executed:
>>    /* Now attempt to open a connection to the thread library.  */
>>    err = td_ta_new_p (&proc_handle, &thread_agent);
>
> Then what set using_thread_db?

I misinterpreted the indentation level at the first try.
So the paths reported above are a little bit different.
The problem is that we end up in the block below.

static void
thread_db_new_objfile (struct objfile *objfile)
    [...]
    /* Don't attempt to use thread_db on targets which can not run
      (core files).  */
    if (objfile == NULL || !target_has_execution)
     {
       /* All symbols have been discarded.  If the thread_db target is
          active, deactivate it now.  */
       if (using_thread_db)
         {
           gdb_assert (proc_handle.pid == 0);
           unpush_target (&thread_db_ops);
           using_thread_db = 0;
         }
       goto quit;
     }

objfile is NULL, why i couldn't figure out.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
> objfile is NULL, why i couldn't figure out.

That should be true the first time but not the second, if you've used
"file".

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
>> objfile is NULL, why i couldn't figure out.
>
> That should be true the first time but not the second, if you've used
> "file".

Ok. The first time objfile is null. The second time target_has_execution
is false.

# ./gdb ./gdb
GNU gdb 6.3
This GDB was configured as "arm-linux"...Using host libthread_db library
"/lib/libthread_db.so.1".

(gdb) l ../../gdb-6.3/gdb/thread-db.c:679
warning: Source file is more recent than executable.

674           }
675       }
676
677       /* Don't attempt to use thread_db on targets which can not run
678          (core files).  */
679       if (objfile == NULL || !target_has_execution)
680         {
681           /* All symbols have been discarded.  If the thread_db target is
682              active, deactivate it now.  */
683           if (using_thread_db)
(gdb)
684             {
685               gdb_assert (proc_handle.pid == 0);
686               unpush_target (&thread_db_ops);
687               using_thread_db = 0;
688             }
689
690           goto quit;
691         }
692
693       if (using_thread_db)
(gdb) b ../../gdb-6.3/gdb/thread-db.c:679
Breakpoint 1 at 0x5c62c: file ../../gdb-6.3/gdb/thread-db.c, line 679.
(gdb) r
Starting program: /progfs/opt/Qtopia/bin/gdb

GNU gdb 6.3
This GDB was configured as "arm-linux".
(gdb) file apdf
Reading symbols from /progfs/opt/Qtopia/bin/apdf...done.
Using host libthread_db library "/lib/libthread_db.so.1".

// first round

Breakpoint 1, thread_db_new_objfile (objfile=0x0) at
../../gdb-6.3/gdb/thread-db.c:679
679       if (objfile == NULL || !target_has_execution)
(gdb) p objfile
$1 = (struct objfile *) 0x0
(gdb) n
683           if (using_thread_db)
(gdb) p using_thread_db
$2 = 0
(gdb) n
729       if (target_new_objfile_chain)
(gdb) p target_new_objfile_chain
$3 = (void (*)()) 0
(gdb) n
731     }
(gdb) c
Continuing.

// second round

Breakpoint 1, thread_db_new_objfile (objfile=0x2ceb08) at
../../gdb-6.3/gdb/thread-db.c:679
679       if (objfile == NULL || !target_has_execution)
(gdb) p objfile
$4 = (struct objfile *) 0x2ceb08
// current_target.to_has_execution = 0
(gdb) n
683           if (using_thread_db)
(gdb) n
729       if (target_new_objfile_chain)
(gdb) n
731     }

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Fri, Nov 04, 2005 at 10:23:21AM +0100, Frank Meerkoetter wrote:
> Quoting Daniel Jacobowitz <drow@false.org>:
> 
> >On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
> >>objfile is NULL, why i couldn't figure out.
> >
> >That should be true the first time but not the second, if you've used
> >"file".
> 
> Ok. The first time objfile is null. The second time target_has_execution
> is false.

It doesn't get called after you say "run"?  Is this a static executable
you're debugging?


-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Fri, Nov 04, 2005 at 10:23:21AM +0100, Frank Meerkoetter wrote:
>> Quoting Daniel Jacobowitz <drow@false.org>:
>>
>> >On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
>> >>objfile is NULL, why i couldn't figure out.
>> >
>> >That should be true the first time but not the second, if you've used
>> >"file".
>>
>> Ok. The first time objfile is null. The second time target_has_execution
>> is false.
>
> It doesn't get called after you say "run"?  Is this a static executable
> you're debugging?

It does get called. Sorry, i tested the wrong thing.

[...]
/* Now attempt to open a connection to the thread library.  */
   err = td_ta_new_p (&proc_handle, &thread_agent);

It returns TD_NOLIBTHREAD.

I know from stepping through thread_db_load() that
That td_ta_new_p = verbose_dlsym (handle, "td_ta_new");
was successfull.

verbose_dlsym() also produced now warning that it couldn't
find a certain symbol.

td_ta_new seems to fail.
I've downloaded the linuxThreads source but can't find any
references to "td_ta_new".

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Fri, Nov 04, 2005 at 04:39:30PM +0100, Frank Meerkoetter wrote:
> Quoting Daniel Jacobowitz <drow@false.org>:
> 
> >On Fri, Nov 04, 2005 at 10:23:21AM +0100, Frank Meerkoetter wrote:
> >>Quoting Daniel Jacobowitz <drow@false.org>:
> >>
> >>>On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
> >>>>objfile is NULL, why i couldn't figure out.
> >>>
> >>>That should be true the first time but not the second, if you've used
> >>>"file".
> >>
> >>Ok. The first time objfile is null. The second time target_has_execution
> >>is false.
> >
> >It doesn't get called after you say "run"?  Is this a static executable
> >you're debugging?
> 
> It does get called. Sorry, i tested the wrong thing.
> 
> [...]
> /* Now attempt to open a connection to the thread library.  */
>   err = td_ta_new_p (&proc_handle, &thread_agent);
> 
> It returns TD_NOLIBTHREAD.
> 
> I know from stepping through thread_db_load() that
> That td_ta_new_p = verbose_dlsym (handle, "td_ta_new");
> was successfull.
> 
> verbose_dlsym() also produced now warning that it couldn't
> find a certain symbol.
> 
> td_ta_new seems to fail.
> I've downloaded the linuxThreads source but can't find any
> references to "td_ta_new".

It's in libthread_db, which is linuxthreads_db/ in the LinuxThreads
source.

It sounds like your libpthread.so has been stripped of some required
symbols.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:
>> /* Now attempt to open a connection to the thread library.  */
>>   err = td_ta_new_p (&proc_handle, &thread_agent);
>>
>> It returns TD_NOLIBTHREAD.
>>
>> I know from stepping through thread_db_load() that
>> That td_ta_new_p = verbose_dlsym (handle, "td_ta_new");
>> was successfull.
>>
>> verbose_dlsym() also produced now warning that it couldn't
>> find a certain symbol.
>>
>> td_ta_new seems to fail.
>> I've downloaded the linuxThreads source but can't find any
>> references to "td_ta_new".
>
> It's in libthread_db, which is linuxthreads_db/ in the LinuxThreads
> source.
>
> It sounds like your libpthread.so has been stripped of some required
> symbols.

I found it.

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Fri, Nov 04, 2005 at 04:39:30PM +0100, Frank Meerkoetter wrote:
>> Quoting Daniel Jacobowitz <drow@false.org>:
>>
>> >On Fri, Nov 04, 2005 at 10:23:21AM +0100, Frank Meerkoetter wrote:
>> >>Quoting Daniel Jacobowitz <drow@false.org>:
>> >>
>> >>>On Thu, Nov 03, 2005 at 06:50:47PM +0100, Frank Meerkoetter wrote:
>> >>>>objfile is NULL, why i couldn't figure out.
>> >>>
>> >>>That should be true the first time but not the second, if you've used
>> >>>"file".
>> >>
>> >>Ok. The first time objfile is null. The second time target_has_execution
>> >>is false.
>> >
>> >It doesn't get called after you say "run"?  Is this a static executable
>> >you're debugging?
>>
>> It does get called. Sorry, i tested the wrong thing.
>>
>> [...]
>> /* Now attempt to open a connection to the thread library.  */
>>   err = td_ta_new_p (&proc_handle, &thread_agent);
>>
>> It returns TD_NOLIBTHREAD.
>>
>> I know from stepping through thread_db_load() that
>> That td_ta_new_p = verbose_dlsym (handle, "td_ta_new");
>> was successfull.
>>
>> verbose_dlsym() also produced now warning that it couldn't
>> find a certain symbol.
>>
>> td_ta_new seems to fail.
>> I've downloaded the linuxThreads source but can't find any
>> references to "td_ta_new".
>
> It's in libthread_db, which is linuxthreads_db/ in the LinuxThreads
> source.
>
> It sounds like your libpthread.so has been stripped of some required
> symbols.

I found it. glibc-linuxthreads-2.2.5.tar.gz contains
linuxthreads/ and linuxthreads_db/ both at the root level...
So i missed it at the first look.

td_err_e
td_ta_new (struct ps_prochandle *ps, td_thragent_t **ta)
{
[...]
   /* Get the global event mask.  This is one of the variables which
      are new in the thread library to enable debugging.  If it is
      not available we cannot debug.  */
   if (td_lookup (ps, PTHREAD_THREADS_EVENTS, &addr) != PS_OK)
     return TD_NOLIBTHREAD;

It seems that td_lookup fails. td_lookup is a wrapper
around ps_pglobal_lookup defined in td_symbol_list.c.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Fri, Nov 04, 2005 at 05:06:30PM +0100, Frank Meerkoetter wrote:
> It seems that td_lookup fails. td_lookup is a wrapper
> around ps_pglobal_lookup defined in td_symbol_list.c.

So why is GDB not finding the symbol?

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Daniel Jacobowitz]

On Fri, Nov 04, 2005 at 04:39:30PM +0100, Frank Meerkoetter wrote:
> It returns TD_NOLIBTHREAD.

A good place to try a breakpoint now would be ps_pglobal_lookup.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: multithreaded programs on arm-linux

[Frank Meerkoetter]

Quoting Daniel Jacobowitz <drow@false.org>:

> On Fri, Nov 04, 2005 at 04:39:30PM +0100, Frank Meerkoetter wrote:
>> It returns TD_NOLIBTHREAD.
>
> A good place to try a breakpoint now would be ps_pglobal_lookup.

The problem is solved. The culprit was that __pthread_threads_events
was missing from libpthread. After rebuilding libpthread gdb
thread support is working as expected.

Thank you very much for your extensive help.

Regards,
   Frank

Re: multithreaded programs on arm-linux

[Andreas Schwab]

Frank Meerkoetter <frank@betaversion.net> writes:

> Does anyone have an idea left how i could end up with
> a thread enabled gdb?

If you are using NPTL, then you need to make sure that the symbol table of
libpthread is complete.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."