GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[William Adair]

Can anyone recommend a way to start debugging GDB? I've found that when I
compile a 32-bit shared library that GDB is unable to correctly resolve the
symbols in the image and set breakpoints. Even when I reach a segfault and
print the backtrace, the symbols are incorrect or flat out missing. I
already verified that the image has .debug_* sections as would be required
for DWARF debugging, so I'm perplexed as to why GDB cannot correctly
resolve it and am looking for guidance.

Thank you.

Re: GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[Hannes Domani]

 Am Sonntag, 1. November 2020, 00:26:34 MEZ hat William Adair via Gdb <gdb@sourceware.org> Folgendes geschrieben:

> Can anyone recommend a way to start debugging GDB? I've found that when I
> compile a 32-bit shared library that GDB is unable to correctly resolve the
> symbols in the image and set breakpoints. Even when I reach a segfault and
> print the backtrace, the symbols are incorrect or flat out missing. I
> already verified that the image has .debug_* sections as would be required
> for DWARF debugging, so I'm perplexed as to why GDB cannot correctly
> resolve it and am looking for guidance.
>
> Thank you.

Do you have a (small) example where this problem can be reproduced?


Hannes

Re: GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[William Adair]

Hannes,

Unfortunately, I do not. It appears I'm left with little choice but to
debug GDB itself while it's debugging the executable that loads the shared
library. I'll add that I am using a custom linker command file and that may
be ruining some predefined expectation from GDB on what data should be in
certain sections, but I haven't dug far enough into GDB source to figure
out how it loads symbols from the binary image, yet.

On Sun, Nov 1, 2020 at 9:53 AM Hannes Domani <ssbssa@yahoo.de> wrote:

>  Am Sonntag, 1. November 2020, 00:26:34 MEZ hat William Adair via Gdb <
> gdb@sourceware.org> Folgendes geschrieben:
>
> > Can anyone recommend a way to start debugging GDB? I've found that when I
> > compile a 32-bit shared library that GDB is unable to correctly resolve
> the
> > symbols in the image and set breakpoints. Even when I reach a segfault
> and
> > print the backtrace, the symbols are incorrect or flat out missing. I
> > already verified that the image has .debug_* sections as would be
> required
> > for DWARF debugging, so I'm perplexed as to why GDB cannot correctly
> > resolve it and am looking for guidance.
> >
> > Thank you.
>
> Do you have a (small) example where this problem can be reproduced?
>
>
> Hannes
>

Re: GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[Andrew Dinn]

On 31/10/2020 23:26, William Adair via Gdb wrote:
> Can anyone recommend a way to start debugging GDB? I've found that when I
> compile a 32-bit shared library that GDB is unable to correctly resolve the
> symbols in the image and set breakpoints. Even when I reach a segfault and
> print the backtrace, the symbols are incorrect or flat out missing. I
> already verified that the image has .debug_* sections as would be required
> for DWARF debugging, so I'm perplexed as to why GDB cannot correctly
> resolve it and am looking for guidance.
Well you can, of course, use your currently installed gdb to debug gdb.

Build gdb from a downloaded src tree using

  make install PREFIX=/path/to/install/dir CFLAGS=-g CXXFLAGS=-g.

Then run

  gdb --fullname /path/to/my/newly/built/gdb.

When you type run you will enter a nested gdb that you can debug.

Hint: before typing run type

(gdb) set prompt (outer)

That way you can distinguish prompts in the outer gdb from prompts in
the inner gdb.

n.b.b. make sure to type a space after (outer) or your typing will start
right next to the closing bracket
regards,

After that all you need to do is familiarize wiht the gdb source!


Andrew Dinn
-----------

Re: GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[William Adair]

Andrew,

Great feedback. It was a bit tricky to figure out how GDB was executing at
first in main(), but I realized that backtrace_command_1 was the function
I'm looking for in the stack.c file. The stack trace eventually hits
print_frame_info()->print_frame() [which relies on find_frame_sal()] ->
find_frame_funname -> lookup_minimal_symbol_by_pc() ->
lookup_minimal_symbol_by_pc_section()
A few immediate questions come to mind: What is the minimal symbol table?
How can I verify that PC is pointing to the section that the actual symbol
being executed resides in? Lastly, there's a comment: This code assumes
that the minimal symbols are sorted by ascending address values. Does this
imply that if the actual symbol for the PC has a lower address than the PC
that the symbol will be wrong? (I wonder if this is what is happening in my
case...)

Thanks everyone for your time.

On Sun, Nov 1, 2020 at 1:28 PM Andrew Dinn <adinn@redhat.com> wrote:

> On 31/10/2020 23:26, William Adair via Gdb wrote:
> > Can anyone recommend a way to start debugging GDB? I've found that when I
> > compile a 32-bit shared library that GDB is unable to correctly resolve
> the
> > symbols in the image and set breakpoints. Even when I reach a segfault
> and
> > print the backtrace, the symbols are incorrect or flat out missing. I
> > already verified that the image has .debug_* sections as would be
> required
> > for DWARF debugging, so I'm perplexed as to why GDB cannot correctly
> > resolve it and am looking for guidance.
> Well you can, of course, use your currently installed gdb to debug gdb.
>
> Build gdb from a downloaded src tree using
>
>   make install PREFIX=/path/to/install/dir CFLAGS=-g CXXFLAGS=-g.
>
> Then run
>
>   gdb --fullname /path/to/my/newly/built/gdb.
>
> When you type run you will enter a nested gdb that you can debug.
>
> Hint: before typing run type
>
> (gdb) set prompt (outer)
>
> That way you can distinguish prompts in the outer gdb from prompts in
> the inner gdb.
>
> n.b.b. make sure to type a space after (outer) or your typing will start
> right next to the closing bracket
> regards,
>
> After that all you need to do is familiarize wiht the gdb source!
>
>
> Andrew Dinn
> -----------
>
>

Re: GDB Incorrectly Reads & Resolves Shared Library Symbols [MinGW-w64]

[Andrew Dinn]

Hi William,

On 03/11/2020 23:20, William Adair wrote:
> Great feedback. It was a bit tricky to figure out how GDB was executing
> at first in main(), but I realized that backtrace_command_1 was the
> function I'm looking for in the stack.c file. The stack trace eventually
> hits print_frame_info()->print_frame() [which relies on
> find_frame_sal()] -> find_frame_funname -> lookup_minimal_symbol_by_pc()
> -> lookup_minimal_symbol_by_pc_section()
> A few immediate questions come to mind: What is the minimal symbol
> table? How can I verify that PC is pointing to the section that the
> actual symbol being executed resides in? Lastly, there's a comment: This
> code assumes that the minimal symbols are sorted by ascending address
> values. Does this imply that if the actual symbol for the PC has a lower
> address than the PC that the symbol will be wrong? (I wonder if this is
> what is happening in my case...)

Well, as I said "After that all you need to do is familiarize with the
gdb source!" ;-)

I have not looked into shared lib code much but I believe minimal
symbols are used to record details of symbols in those libs for which
gdb does not have (maybe cannot have?) full info.

The type definition is in symtab.h along with related types symbol and
general_symbol_info. Note that symbol and minimal_symbol both (publicly)
inherit common data from general_symbol_info which crops up all over the
code base. n.b. your quickest way to see what is these types are is to
use gdb:

(outer) ptype minimal_symbol
. . .
(outer) ptype symbol
. . .
(outer) ptype general_symbol_info
. . .

The stuff about address ordering may be the issue but it sounds to me
like this is a comment about how gdb expects its data to be organized.
If something is missing/out of order in gdb's data that may be to do
with the order in the shared lib or it may be because gdb has failed to
read or sort something from the lib.

gdb has a host of _debug vars you can set to trace what it is doing as
it executes a program. This command will help you find them

(outer) info var _debug

The debug vars you probably want to set for tracing symtab processing
can be enabled like this

(outer) break main
...
(outer) commands
 > set symbol_lookup_debug = 1
 > set symtab_create_debug
 > end

The above breakpoint with associated commands will set these debug vars
when gdb starts running.

I suggest you also look at the contents of the shared library outside of
gdb using objdump and/or nm. That will tell you what gdb is trying to
process and you may be able to compare it with the debug output to see
whether any symbols is getting lost or discarded.

> Thanks everyone for your time.
You are welcome. Good luck.

regards,


Andrew Dinn
-----------