Re: Can't debug x86_64 C++ programs.

[Keith Seitz <keiths@redhat.com>]

John Fine wrote:
> Can you give me some hints where in the GDB/Insight source code certain 
> operations are done?  I think I might be able to fix or at least 
> understand some of these bugs myself.

Yes, I can provide some pointers about where to look for certain things, 
but my knowledge of gdb is rusty. The good news is that it is now my job 
to make gdb a better c++ debugger, so my knowledge grows every day.

> I'd especially like to have the nexti button actually work (rather than 
> usually step in).  I can't imagine how that bug could be the GUI's 
> fault.  It only makes sense if the GDB code for nexti can't recognize a 
> call instruction in x86_64 asm code.  I'd really like to look at the 
> code in gdb that tries to determine whether the next instruction is a 
> call.  (Maybe it is confused by the possible encodings of call or maybe 
> it is confused by the 64-bit virtual address and sometimes looking in 
> the wrong place.)  But I don't know how to find that code.

First things first: verify that this is a problem with core gdb (which 
it would have to be IMO). Run the command-line gdb (insight -nw or 
insight --i=console), load your file (pass as argument or "file MYAPP" 
at gdb prompt), set a break somewhere ("break SOMEWHERE"), and try to 
nexti. If it doesn't work here, it is definitely a gdb problem.

Debugging this, though, is going to be a pain. A real pain. You should 
set a breakpoint at "nexti_command" in infcmd.c and step from there. I 
haven't looked at this code in almost a decade, but I do remember it 
being particularly nasty.

>> Isolated test cases are very useful.
> I'll see what I can find time to create.  So far I haven't tried to 
> debug anything small, so I don't know whether Insight would behave 
> differently.  I haven't found any target programs in which insight 
> doesn't have these malfunctions.  There are calls in each program for 
> which gdb/Insight can correctly step over or out.  There are lines in 
> the source code of each for which gdb/Insight seems to understand the 
> correspondence between source and asm.  But in every program, there are 
> lines where it doesn't.

If you can find a test case -- ANY test case -- that you can send me, 
that would greatly expedite this process.

> A lot of the debugging I attempted, and some of the worst gdb/Insight 
> behavior I saw) was with the program opannotate that is part of oprofile 
> package.

I am quite familiar with opannotate (having written the eclipse-oprofile 
  RPM), so I'll take a look at this and see if I cannot get any further 
with your problems.

> It is Oprofile version 0.9.3 (because the build for the newer Oprofile 
> was unhappy with some older libraries on our Centos system).  It was 
> built with gcc version 3.4.6 with switches -g O2.

gcc 3.4.6? Wow, that's over two and one-half years old -- ancient by 
gcc/gdb standards *for C++*. I will try to grab a copy of this and see 
how executables built by this behave with gdb. I am going to guess that 
this is a big part of the problem.

I don't suppose you could try gcc 4.1 or 4.4?

> In Opannotate, I just wanted to understand some of the basic flow of the 
> program that I couldn't figure out from the source code.  So I just 
> wanted to do a bunch of step over and a few step into operations.  But I 
> spent all my time trying to get out of the functions that I stepped into 
> using the step over button and never reached the parts I wanted to see.

I'll give this a try.

> Most of what I'm attempting to debug is in a large closed source project 
> compiled with the Intel 10.0 compiler.

I don't know much about the Intel 10.0 compiler or how well gdb works 
with it. I'll ask around.

> Another place I'd really like some hints about gdb/Insight source code 
> location is in the construction of the text that goes into the asm 
> window.  I probably won't understand the code well enough to make the 
> change I'd like, but I want to look and see if I can.  On each line of 
> asm code, I'd like display the line number of the source line that the 
> debugger thinks the debug info tells it corresponds.  If I really 
> understood how to change things, I'd also color code that, to flag the 
> places where the source filename is different.  If practical, I would 
> also drop the display of the function+offset version of the address.  In 
> C++ code function names get too messy and the plain virtual address 
> should be good enough.

That occurs in gdb_load_assembly in src/gdb/gdbtk/generic/gdbtk-cmds.c.

> I'm not certain, but I think it happens when the debugger thinks the 
> source file has changed.  I do a step across an asm instruction and 
> (because of inlining) the source file might change.

If I am reading this right, the executable has changed and the asm pane 
is now obsolete. There could be a bug here w.r.t. updating the contents 
of the asm pane (a bug I've squashed several times over the years), but 
I cannot think how this could cause the panes to swap places off the top 
of my head. Again, I'll take a look and see if I can reproduce this 
given the new information.

> If it shouldn't swap, where should it stay (top or bottom)?

SRC pane should always be on top, ASM always on the bottom.

> When you set a breakpoint in asm view, Insight sometimes displays the 
> red mark in both asm view and source view.  Is it really gdb code that 
> tells it where to display the red mark in source view?

Yes, because only the backend (gdb) knows how to translate between the 
two. Insight is actually pretty stupid most of the time: it was designed 
this way so that getting a new architecture to run on it would require 
getting gdb running. Even today, with every new arch added to gdb, the 
only time insight needs modification is if a new target is introduced.

So, there we are: more investigation is necessary. I hope to get to some 
of this tonight.

Keith