gdb does not show full backtrace for deadlocked pthread program.

gdb does not show full backtrace for deadlocked pthread program.

[Ben Greear]

Hello!

I have a program that uses pthreads and evidently has a
nifty little thread deadlock.

The interesting thing is that gdb will not show me the full
backtrace of the deadlocked threads.  Both tested system's running
a slightly modified 2.6.11 kernel.  The FC2 system is a dual-xeon with
SMP kernel.  The FC4 machine is a laptop with UP kernel.
FC2's gdb doesn't even decode frame 3, but on FC4 I get this:

[after attaching to deadlocked process]

(gdb) thread apply all bt

Thread 2 (Thread -1208132688 (LWP 9895)):
#0  0xffffe410 in ?? ()
#1  0xb7fd50e8 in ?? ()
#2  0x00000002 in ?? ()
#3  0x004b4e7e in __lll_mutex_lock_wait () from /lib/libpthread.so.0

Thread 1 (Thread -1208026656 (LWP 9892)):
#0  0xffffe410 in ?? ()
#1  0xbffff118 in ?? ()
#2  0x00000002 in ?? ()
#3  0x004b4e7e in __lll_mutex_lock_wait () from /lib/libpthread.so.0


Frames 0 and 1 appear to be in linux-gate.so, most likely whatever
kernel call allows nptl to work??

Any idea how to get a full backtrace for these threads?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: gdb does not show full backtrace for deadlocked pthread program.

[Daniel Jacobowitz]

On Mon, Sep 26, 2005 at 12:24:06AM -0700, Ben Greear wrote:
> Hello!
> 
> I have a program that uses pthreads and evidently has a
> nifty little thread deadlock.
> 
> The interesting thing is that gdb will not show me the full
> backtrace of the deadlocked threads.  Both tested system's running
> a slightly modified 2.6.11 kernel.  The FC2 system is a dual-xeon with
> SMP kernel.  The FC4 machine is a laptop with UP kernel.
> FC2's gdb doesn't even decode frame 3, but on FC4 I get this:
> 
> [after attaching to deadlocked process]
> 
> (gdb) thread apply all bt
> 
> Thread 2 (Thread -1208132688 (LWP 9895)):
> #0  0xffffe410 in ?? ()
> #1  0xb7fd50e8 in ?? ()
> #2  0x00000002 in ?? ()
> #3  0x004b4e7e in __lll_mutex_lock_wait () from /lib/libpthread.so.0

GDB knows how to load symbols for the kernel DSO and backtrace from it. 
It also knows how to load separate debug info packages, which I assume
Red Hat provides for libc.  I would have expected the shipped GDB to do
this, but if not, try building your own from current CVS.  If you
install the debug info packages and want to use them, you may need to
configure your own gdb with --prefix=/usr.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: gdb does not show full backtrace for deadlocked pthread program.

[Ben Greear]

Daniel Jacobowitz wrote:
> On Mon, Sep 26, 2005 at 12:24:06AM -0700, Ben Greear wrote:
> 
>>Hello!
>>
>>I have a program that uses pthreads and evidently has a
>>nifty little thread deadlock.
>>
>>The interesting thing is that gdb will not show me the full
>>backtrace of the deadlocked threads.  Both tested system's running
>>a slightly modified 2.6.11 kernel.  The FC2 system is a dual-xeon with
>>SMP kernel.  The FC4 machine is a laptop with UP kernel.
>>FC2's gdb doesn't even decode frame 3, but on FC4 I get this:
>>
>>[after attaching to deadlocked process]
>>
>>(gdb) thread apply all bt
>>
>>Thread 2 (Thread -1208132688 (LWP 9895)):
>>#0  0xffffe410 in ?? ()
>>#1  0xb7fd50e8 in ?? ()
>>#2  0x00000002 in ?? ()
>>#3  0x004b4e7e in __lll_mutex_lock_wait () from /lib/libpthread.so.0
> 
> 
> GDB knows how to load symbols for the kernel DSO and backtrace from it. 
> It also knows how to load separate debug info packages, which I assume
> Red Hat provides for libc.  I would have expected the shipped GDB to do
> this, but if not, try building your own from current CVS.  If you
> install the debug info packages and want to use them, you may need to
> configure your own gdb with --prefix=/usr.

The latest cvs did indeed print out useful stack traces.  Would
be lovely to have whatever magic makes that work in the more
official releases!

Thanks,
Ben

> 


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com