Reading dynamic symbols from Linux core files?

Reading dynamic symbols from Linux core files?

[John Gilmore]

Martin Langhoff of laptop.org is trying to debug an intermittent bug
in the One Laptop Per Child that results in a core dump of a Python
process.  The bug report (including the gzipped core file) is here:

  http://dev.laptop.org/ticket/11698

Martin and Jan Kratochvil are currently chasing a problem with finding
and getting "debug symbol" packages in Fedora 14 (which this OLPC
software release is based on).  But in the process of helping them, I
also noticed that merely running:

  $ gdb - core.522

on the core file doesn't provide access to most symbols from the dynamic
symbol table.

Oddly, "info functions" does produce three non-debugging symbols, but
nothing else.  Gdb knows that the second thread is stopped in
__kernel_vsyscall() but doesn't know much else.  See attached
transcript.

The internal structure of GDB has changed enough that I'm not sure
exactly where in the code it should be trying to find the dynamic
symbol table from the core file and extracting the symbols from it.  I
remember making this work many years ago in GDB, which made it much
more useful when debugging executables that had no symbols, and core
files for which matching executables were hard to come by.

Have any clues for me?

	John

$ gdb - /tmp/core.522
GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
-: No such file or directory.
[New Thread 522]
[New Thread 603]
Core was generated by `python /usr/bin/sugar-session'.
Program terminated with signal 11, Segmentation fault.
#0  0xa7183e2e in ?? ()
(gdb) i fun
All defined functions:

Non-debugging symbols:
0xa78be400  __kernel_sigreturn
0xa78be40c  __kernel_rt_sigreturn
0xa78be414  __kernel_vsyscall
(gdb) bt
#0  0xa7183e2e in ?? ()
#1  0x08f352d0 in ?? ()
#2  0xa6f0ae57 in ?? ()
#3  0x08cea354 in ?? ()
#4  0xa7771878 in ?? ()
#5  0x00000000 in ?? ()
(gdb) i threads
  2 Thread 603  0xa78be424 in __kernel_vsyscall ()
* 1 Thread 522  0xa7183e2e in ?? ()
(gdb) thread 2
[Switching to thread 2 (Thread 603)]#0  0xa78be424 in __kernel_vsyscall ()
(gdb) bt
#0  0xa78be424 in __kernel_vsyscall ()
#1  0xa75f6c36 in ?? ()
#2  0xffffffff in ?? ()
#3  0x00000001 in ?? ()
#4  0xa724b27c in ?? ()
#5  0xa718edac in ?? ()
#6  0xa55023c0 in ?? ()
#7  0xa717e8b7 in ?? ()
#8  0xa55023c0 in ?? ()
#9  0xa717f04b in ?? ()
#10 0x00000001 in ?? ()
#11 0x00000001 in ?? ()
#12 0xa5e18328 in ?? ()
#13 0xa70cbed5 in ?? ()
#14 0x088bd0b0 in ?? ()
#15 0xa71a7bd0 in ?? ()
#16 0x00000000 in ?? ()
(gdb) 

Re: Reading dynamic symbols from Linux core files?

[Jan Kratochvil]

On Fri, 16 Mar 2012 19:38:40 +0100, John Gilmore wrote:
> Martin and Jan Kratochvil are currently chasing a problem with finding
> and getting "debug symbol" packages in Fedora 14

There may be some prelink related problem but I have no more ideas:
	http://lists.fedoraproject.org/pipermail/devel/2012-March/164047.html


> I'm not sure exactly where in the code it should be trying to find the
> dynamic symbol table from the core file and extracting the symbols from it.

gdb/symfile-mem.c
bfd/elfcode.h NAME(_bfd_elf,bfd_from_remote_memory)

The problem usually was - at least in some Fedoras + RHELs - that vDSO has
overgrown 4KB.  The loadable segment remained in the first 4KB part but
debugging symbols spanned more pages which led to bfd_from_remote_memory
cutting off the debug symbols.  Recent Fedora patch for it is:
	http://pkgs.fedoraproject.org/gitweb/?p=gdb.git;a=blob_plain;f=gdb-6.6-bfd-vdso8k.patch;hb=master


Regards,
Jan

Re: Reading dynamic symbols from Linux core files?

[Martin Langhoff]

On Fri, Mar 16, 2012 at 2:48 PM, Jan Kratochvil
<jan.kratochvil@redhat.com> wrote:
> The problem usually was - at least in some Fedoras + RHELs - that vDSO has
> overgrown 4KB.  The loadable segment remained in the first 4KB part but
> debugging symbols spanned more pages which led to bfd_from_remote_memory
> cutting off the debug symbols.  Recent Fedora patch for it is:
>        http://pkgs.fedoraproject.org/gitweb/?p=gdb.git;a=blob_plain;f=gdb-6.6-bfd-vdso8k.patch;hb=master

So we're SOL?

BTW, thanks Jan and John for taking this upstream with a background
knowledge I can't match!

One thing to note is that we cannot entirely rule out some binaries
being corrupt. This problem has already crossed several "that's
impossible" boundaries. One of our avenues of investigation is an SD
card duplicator introducing some odd fs corruption, although rpm -Va
and other checksums tell us the files are good.

Also - http://lists.fedoraproject.org/pipermail/devel/2012-March/164105.html



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff