gdb "automation" question

gdb "automation" question

[Steffen Dettmer]

Hi,

I've got questions regarding gdb "automation" which I want to use
to pass "log file" information from a remote target.

I am using arm-elf-gdb-6.8.

How do I get "log" information from my application via remote gdb?

Here an overview what I'm doing currently and I would like to archive.

I have an embedded device and run remote gdb. I can load my
applicate with "remote put" (after "target remote"). Remote
device needs to be rebooted and gdb needs to reconnect (via
"target remote"). I use user "define"d commands to do so. The
user defined commands also disable all breakpoints, because
otherwise gdb does not connect (maybe a limitation of the used
remote monitor).

The logging library (e.g. on linux) can print via STDERR, so when
debugging native programs locally, I can see the log messages
easily. Of course this does not work with remote targets.
So I use a small log handler that concatenates the logmessages to
a global static char globalLogBuffer[big_enough]. This can be
shown via gdb. I can for example use:

define show_log
  if globalLog[0] != 0
     printf "%s", globalLog
  endif
  set globalLog[0] = 0
end

However, it would be better to automatically show the messages.

Best would be to show this right after a message was added. So in
the log handler I added a call to an empty static function
logHook() and tried:

define ena_log
  b logHook
  commands
    silent
    show_log
    continue
  end
end

This has the big disadvantage that it breaks stepping (commands
"next" and so on), because when in "next" the logHook break point
is reached (as usual for most high-level debugging), continue is
invoked and the position of the interactive debug is lost
(without the continue, it stops in the logHook, also not
helpful). Another problem is that to reconnect all breakpoints
must be disabled ("dis") but afterwards cannot be enabled ("ena"
would enable all breakpoints, including the breakpoints disabled
before the "dis" - of course!). The numbers of the break points
of course change, so the script cannot do something like "ena 12"
or "enable breakpoint m/logHook/" or so.

Without the single-stepping issue this would be lovely. It works
nicely (except the show-stopping "next" issue).

I though simplest workaround would be

define hook-stop
  show_log
end

Unfortunately directly after rebooting (before "target remote"
and "continue"), the hook cannot work. I have to CTRL-C all the
time to cancel hook execution. So I tried to add this to my user
defined commands. They call a command "defaction" which I
extended:

define defaction
  .... other default actions ...
  define hook-stop
     show_log
  end
end

but I get an error ("This command cannot be used at top level").

any ideas how I could get around those issues?

Best would be a kind of "at this position perform an action and
continue" that would not interference with user breakpoints and
stepping with "next".

Also if someone knows some workaround or has an completely
different approach I would appreciate to read about :-)

oki,

Steffen

Re: gdb "automation" question

[Tom Tromey]

>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:

Steffen> I've got questions regarding gdb "automation" which I want to use
Steffen> to pass "log file" information from a remote target.

What version of gdb are you using?

Steffen> Best would be to show this right after a message was added. So in
Steffen> the log handler I added a call to an empty static function
Steffen> logHook() and tried:
Steffen> define ena_log
Steffen>   b logHook
Steffen>   commands
Steffen>     silent
Steffen>     show_log
Steffen>     continue
Steffen>   end
Steffen> end

Steffen> This has the big disadvantage that it breaks stepping (commands
Steffen> "next" and so on)

Yeah, we've recently been discussing this use case (as a tangent from
the main discussion) on the patch list.

Unless I really misread it, I think there is some agreement that we
should have a new flag on a breakpoint that would tell the gdb internals
"after the commands are done, continue stepping".

This would solve this problem plus a few other similar things.


I did find a trick you can use to accomplish this with a python-enabled
gdb.  What you do is write a new convenience function in Python.  Do all
the work in this function, then have the function always return 0.
Finally, make your breakpoint condition call this function.

This is ugly, but it does seem to work.

Steffen> Another problem is that to reconnect all breakpoints
Steffen> must be disabled ("dis") but afterwards cannot be enabled ("ena"
Steffen> would enable all breakpoints, including the breakpoints disabled
Steffen> before the "dis" - of course!). The numbers of the break points
Steffen> of course change, so the script cannot do something like "ena 12"
Steffen> or "enable breakpoint m/logHook/" or so.

With a recent enough GDB you can write a Python script to do the
enabling, which will give you more control over what happens.

Steffen> define hook-stop
Steffen>   show_log
Steffen> end

Steffen> Unfortunately directly after rebooting (before "target remote"
Steffen> and "continue"), the hook cannot work. I have to CTRL-C all the
Steffen> time to cancel hook execution.

I guess you could make a flag and a new phony "target":

set var $flag = 0

define target ours
  target remote etc
  set var $flag = 1
end

define hook-stop
  if $flag
    show_log
  endif
end

Then have users use "target ours" instead of "target remote".

Steffen> define defaction
Steffen>   .... other default actions ...
Steffen>   define hook-stop
Steffen>      show_log
Steffen>   end
Steffen> end
Steffen> but I get an error ("This command cannot be used at top level").

At least in the current sources this error only comes from the
tracepoint code.  I don't have a tree before 7.0 handy, so if you're
using something older, maybe upgrading would help this.  Or maybe the
"..." includes tracepoint commands?

Steffen> Best would be a kind of "at this position perform an action and
Steffen> continue" that would not interference with user breakpoints and
Steffen> stepping with "next".

A patch would be great :-).  But you could also CC yourself on the bug:
http://sourceware.org/bugzilla/show_bug.cgi?id=11669

FWIW, giving Python better access to "inferior control" (stuff like
noticing when the inferior stops, starts, is killed, etc) is also on our
wish list.  There were some initial patches from last year's Summer of
Code, but we haven't integrated those yet, and in any case I don't think
they went quite far enough.

Tom

Re: gdb "automation" question

[Steffen Dettmer]

Hi,

thank you for your quick response.

On Tue, Jun 22, 2010 at 6:09 PM, Tom Tromey <tromey@redhat.com> wrote:
>>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:
>
> Steffen> I've got questions regarding gdb "automation" which I want to use
> Steffen> to pass "log file" information from a remote target.
>
> What version of gdb are you using?

I am using arm-elf-gdb-6.8.

> Yeah, we've recently been discussing this use case (as a tangent from
> the main discussion) on the patch list.

Ohh, so this means that this is not yet possible...

> Unless I really misread it, I think there is some agreement that we
> should have a new flag on a breakpoint that would tell the gdb internals
> "after the commands are done, continue stepping".
>
> This would solve this problem plus a few other similar things.

Yes, I think this would solve this.

> I did find a trick you can use to accomplish this with a
> python-enabled gdb.

Ohh, until now I did not even knew that there is a python-enabled
gdb. I found http://sourceware.org/gdb/current/onlinedocs/gdb/Python.html.
http://sourceware.org/gdb/wiki/PythonGdb sounds as this would be
an experimental feature. Are the parts I need stable?
Should I work on branch origin/archer-tromey-python on a git
clone?

> What you do is write a new convenience function in Python.  Do
> all the work in this function, then have the function always
> return 0.  Finally, make your breakpoint condition call this
> function.

sounds promising.
Where can I learn more?

> This is ugly, but it does seem to work.

(python extended gdb seems as it could become a lovely feature! :))
I'd have interesting things to learn (GIT, python, gdb...).
Is there some recommended tutorial?

> With a recent enough GDB you can write a Python script to do the
> enabling, which will give you more control over what happens.

Is it needed from repo (as wiki/PythonGdb):

  git clone git://sourceware.org/git/archer.git
  cd archer; git checkout --track -b python origin/archer-tromey-python

or is there already a released version recent enough?




> Steffen> define hook-stop
> Steffen>   show_log
> Steffen> end
>
> Steffen> Unfortunately directly after rebooting (before "target remote"
> Steffen> and "continue"), the hook cannot work. I have to CTRL-C all the
> Steffen> time to cancel hook execution.
>
> I guess you could make a flag and a new phony "target":
>
> set var $flag = 0
>
> define target ours
>  target remote etc
>  set var $flag = 1
> end
>
> define hook-stop
>  if $flag
>    show_log
>  endif
> end
>
> Then have users use "target ours" instead of "target remote".

(the "target" inside the define uses the original "target"?
Shouldn't it recurse...)

Not sure if I get this...

shall I add:

define kill mykill
 set var $flag = 0
 kill
end

but what for the crashes? Sometimes gdb shows my that some
trapReset was caught. Should I hook it in some way?



> Steffen> define defaction
> Steffen>   .... other default actions ...
> Steffen>   define hook-stop
> Steffen>      show_log
> Steffen>   end
> Steffen> end
> Steffen> but I get an error ("This command cannot be used at top level").
>
> At least in the current sources this error only comes from the
> tracepoint code.  I don't have a tree before 7.0 handy, so if you're
> using something older, maybe upgrading would help this.  Or maybe the
> "..." includes tracepoint commands?

No, in my tests there were no other default actions. It was 6.8.
I hope I can re-check with 7.1 tomorrow.

> Steffen> Best would be a kind of "at this position perform an action and
> Steffen> continue" that would not interference with user breakpoints and
> Steffen> stepping with "next".
>
> A patch would be great :-).  But you could also CC yourself on the bug:
> http://sourceware.org/bugzilla/show_bug.cgi?id=11669
>
> FWIW, giving Python better access to "inferior control" (stuff like
> noticing when the inferior stops, starts, is killed, etc) is also on our
> wish list.  There were some initial patches from last year's Summer of
> Code, but we haven't integrated those yet, and in any case I don't think
> they went quite far enough.

It is great to watch gdb improving. Also really great that now
such great documentation exists. And so many new exciting
features added in the last years. Just great.

A big THANK YOU to everyone involved!!!

oki,

Steffen

Re: gdb "automation" question

[Steffen Dettmer]

* On Tue, Jun 22, 2010 at 6:09 PM, Tom Tromey <tromey@redhat.com> wrote:
> Steffen> define defaction
> Steffen>   .... other default actions ...
> Steffen>   define hook-stop
> Steffen>      show_log
> Steffen>   end
> Steffen> end
> Steffen> but I get an error ("This command cannot be used at top level").
>
> At least in the current sources this error only comes from the
> tracepoint code.  I don't have a tree before 7.0 handy, so if you're
> using something older, maybe upgrading would help this.  Or maybe the
> "..." includes tracepoint commands?

No, I did not use tracepoints (not supported on platform).

I re-tested with version 7.1, which was the most recent I could
find on http://ftp.gnu.org/gnu/gdb/.
I observed the same issue.

Here a minimal "This command cannot be used at the top level"
example:

------------------------------------------------------------------->8=======
steffen@host:dir/arm-elf $ cat gdb-commands.txt
# Do not edit, automatically generated by make
define defactions
  define hook-stop
    show_log
  end
end
document defactions
  Default actions
end

steffen@host:dir/arm-elf $ /usr/local/build/gdb-7.1/build/gdb/gdb
GNU gdb (GDB) 7.1
 [...]
This GDB was configured as "--host=i686-pc-linux-gnu --target=arm-elf".
 [...]
(gdb) so gdb-commands.txt
/path/dir/arm-elf/gdb-commands.txt:6: Error in sourced command file:
This command cannot be used at the top level.
=======8<-------------------------------------------------------------------

This shows the error without any tracepoint.


Similar attempt, but this blocks / hangs.

First what works:

------------------------------------------------------------------->8=======
steffen@host:dir/arm-elf $ cat gdb-commands.txt
# Do not edit, automatically generated by make
define tr
  disable
  target remote 1.2.3.4:4000
  monitor set appName 829901
  symbol-file 8299011999.elf
end
steffen@host:dir/arm-elf $ /usr/local/build/gdb-7.1/build/gdb/gdb
 [...]
(gdb) so gdb-commands.txt
(gdb) q
=======8<-------------------------------------------------------------------


So this works, but when adding "define hook-stop\n  end" to get rid
of the hook, it hangs:

------------------------------------------------------------------->8=======
steffen@host:dir/arm-elf $ cat gdb-commands.txt
# Do not edit, automatically generated by make
define tr
  disable
  define hook-stop
  end
  target remote 1.2.3.4:4000
  monitor set appName 829901
  symbol-file 8299011999.elf
end
steffen@host:dir/arm-elf $ /usr/local/build/gdb-7.1/build/gdb/gdb
 [...]
(gdb) so gdb-commands.txt
<<<<<<<<<<<<<<<<<< PRESSED CTRL-C >>>>>>>>>>>>>>>>>>
/path/dir/arm-elf/gdb-commands.txt:6: Error in sourced command file:
1.2.3.4:4000: Interrupted system call.
=======8<-------------------------------------------------------------------

Why does it attempt to do anything remotely when adding a
"define"? Shouldn't this just define a new command?



BTW, I cleant my ~/.gdbinit for the test:

------------------------------------------------------------------->8=======
define show_log
  if clogStaticLogBuffer[0] != 0
    printf "%s", clogStaticLogBuffer
  end
  set clogStaticLogBuffer[0] = 0
end
=======8<-------------------------------------------------------------------

Re: gdb "automation" question

[Tom Tromey]

>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:

Steffen> Ohh, until now I did not even knew that there is a
Steffen> python-enabled gdb. I found
Steffen> http://sourceware.org/gdb/current/onlinedocs/gdb/Python.html.
Steffen> http://sourceware.org/gdb/wiki/PythonGdb sounds as this would
Steffen> be an experimental feature. Are the parts I need stable?
Steffen> Should I work on branch origin/archer-tromey-python on a git
Steffen> clone?

You don't need the git branch; you can probably do what you want with
the 7.1 release, or current gdb CVS.

>> What you do is write a new convenience function in Python.

Steffen> sounds promising.
Steffen> Where can I learn more?

Convenience functions are documented in the manual.

>> This is ugly, but it does seem to work.

Steffen> (python extended gdb seems as it could become a lovely feature! :))
Steffen> I'd have interesting things to learn (GIT, python, gdb...).
Steffen> Is there some recommended tutorial?

I wrote a series about it on my blog.  A lot of that is still relevant,
though some details have changed, and not all the changes have been
merged.

Start here: http://tromey.com/blog/?p=494

>> define target ours
>> target remote etc
[...]

Steffen> (the "target" inside the define uses the original "target"?
Steffen> Shouldn't it recurse...)

This define adds a new subcommand to target, called "ours".
It doesn't override "target" itself.

Tom

Re: gdb "automation" question

[Tom Tromey]

>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:

Steffen> Here a minimal "This command cannot be used at the top level"
Steffen> example:

Ok, I see.  The problem is more obvious if you enter the commands
manually:

(gdb) define z
Type commands for definition of "z".
End with a line saying just "end".
>define hook-stop
>echo hi
>end
(gdb) 

"define" doesn't properly recognize a nested define.

Could you file a bug report for this?

Tom

Re: gdb "automation" question

[Steffen Dettmer]

On Wed, Jun 23, 2010 at 11:01 PM, Tom Tromey <tromey@redhat.com> wrote:
> "define" doesn't properly recognize a nested define.
> Could you file a bug report for this?

I created #11750 (http://sourceware.org/bugzilla/show_bug.cgi?id=11750).
Maybe you could review/revise to ensure I wrote correctly and
included the needed information?

oki,

Steffen

Re: gdb "automation" question

[Tom Tromey]

>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:

Steffen> I created #11750
Steffen> (http://sourceware.org/bugzilla/show_bug.cgi?id=11750).  Maybe
Steffen> you could review/revise to ensure I wrote correctly and
Steffen> included the needed information?

It looks good to me, thanks for doing that.

Tom

Re: gdb "automation" question

[Steffen Dettmer]

Hi,

thanks to Tom and others I made progress to automatically display
my remote log messages using python-enabled gdb-7.1.

:-)

Thank you, I have a much better situation now.

However, a bit more tuning would be great :)



First let me summarize what I did:

in C code:

   static char *logStaticLogBuffer ...
   static void logTrace() { }

at the end of some log function added:

   strncat(logStaticLogBuffer, formattedMessage, len);
   logTrace(); /* empty function, just to have a symbol to break at */

This can be used with (from `~/.gdbinit')

  define show_log
    if logStaticLogBuffer != 0 && logStaticLogBuffer[0] != 0
      printf "%s", logStaticLogBuffer
      set logStaticLogBuffer[0] = 0
    end
  end

to get it printed right after a record was added (from `~/.gdbinit.py'):

  class ShowClogPy (gdb.Function):
      """Shows the static log buffer"""

      def __init__ (self):
          super (ShowClogPy, self).__init__ ("show_log_py")

      def invoke (self):
          gdb.execute("show_log")
          return False

  ShowClogPy()

interactively this can be `enabled' by:

  (gdb) break logTrace if $show_log_py()

and this works. My connect command

  disable
  target remote ...
  monitor ...
  symbol-file image.elf"

unfortunately must `disable' - otherwise, `target remote' fails with:

  Warning:
  Cannot insert breakpoint 1.
  Error accessing memory address 0xd2b44: Unknown error 4294967295.

the error remains for any following commands.




Now my questions how I can improve:

#1 When several messages were printed, I get
     ---Type <return> to continue, or q <return> to quit---
   is there a way to avoid it?



#2 How do I write show_log in Python? According to the online manual
   `Python representation of symbols' I understood that I have to
   use gdb.lookup_symbol. Is this right? I tried
     gdb.lookup_symbol("logStaticLogBuffer")
   but this only leads to
     AttributeError: 'module' object has no attribute 'lookup_symbol'



#3 my connect command has to disable breakpoints.
   I cannot re-enable them, because it does not know which
   breakpoints were enabled and which not.
   I've read about `save breakpoints' and tried it:

     (gdb) i b
     Num     Type           Disp Enb Address    What
     1       breakpoint     keep y   0x000d2b44 in logTrace
                                            at xxx.c:123
             stop only if $show_log_py()

     (gdb) save breakpoints tmpfile
     warning: save-tracepoints: no tracepoints to save.

     (gdb) source tmpfile
     tmpfile: No such file or directory.

   Would `save breakpoints' help to re-enable my breakpoints?
   How do I use it correctly?



#4 How do I source ~/.gdbinit.py if and only if I have a python-enabled
   gdb?
   If not possible:
   Would it be an idea for a future version to add such a feature?



#5 How can I automake `break logTrace if $show_log_py()' to be executed
   only if
     - Python avialable (would be solved by #4)
     - symbol logTrace and logStaticLogBuffer both exist
   I though I'd need some `if gdb.lookup_symbol("logTrace")',
   but failed because of #2



#6 BTW, why does `source ~/.gdbinit.py' work? Is the file name
   extension telling that it contains Python syntax?
   Otherwise I got errors (and had expected that I have to start
   with some `python' command and end with `end').



I'm sorry for my long (and frequent) posting, but I hope to give
all details that could be helpful in case someone finds time to
point me to the right place.

oki,

Steffen

Re: gdb "automation" question

[Pedro Alves]

I'll answer this one:

On Monday 28 June 2010 16:11:44, Steffen Dettmer wrote:
> #3 my connect command has to disable breakpoints.
>    I cannot re-enable them, because it does not know which
>    breakpoints were enabled and which not.
>    I've read about `save breakpoints' and tried it:
> 
>      (gdb) i b
>      Num     Type           Disp Enb Address    What
>      1       breakpoint     keep y   0x000d2b44 in logTrace
>                                             at xxx.c:123
>              stop only if $show_log_py()
> 
>      (gdb) save breakpoints tmpfile
>      warning: save-tracepoints: no tracepoints to save.
> 
>      (gdb) source tmpfile
>      tmpfile: No such file or directory.
> 
>    Would `save breakpoints' help to re-enable my breakpoints?
>    How do I use it correctly?

"save breakpoints" is new, starting with gdb 7.2.  7.1 only had
"save-tracepoints".  So in < 7.2, "save breakpoints tmpfile"
is the same as "save-tracepoints breakpoints tmpfile", that is,
you're attempting to save tracepoints to a file named
"breakpoints tmpfile".  This is why you see the warning above.

-- 
Pedro Alves

Re: gdb "automation" question

[Tom Tromey]

>>>>> "Steffen" == Steffen Dettmer <steffen.dettmer@googlemail.com> writes:

Steffen> thanks to Tom and others I made progress to automatically display
Steffen> my remote log messages using python-enabled gdb-7.1.

Steffen> #1 When several messages were printed, I get
Steffen>      ---Type <return> to continue, or q <return> to quit---
Steffen>    is there a way to avoid it?

"set pagination off".

If you're referring to script-generated output, I don't think we
currently have a way to print unfiltered.

Steffen> #2 How do I write show_log in Python? According to the online manual
Steffen>    `Python representation of symbols' I understood that I have to
Steffen>    use gdb.lookup_symbol. Is this right? I tried
Steffen>      gdb.lookup_symbol("logStaticLogBuffer")
Steffen>    but this only leads to
Steffen>      AttributeError: 'module' object has no attribute 'lookup_symbol'

You need a newer gdb.  lookup_symbol was added after 7.1.

Steffen> #3 my connect command has to disable breakpoints.
Steffen>    I cannot re-enable them, because it does not know which
Steffen>    breakpoints were enabled and which not.

I think with a newer gdb you could compute this info from Python.

Steffen> #4 How do I source ~/.gdbinit.py if and only if I have a python-enabled
Steffen>    gdb?
Steffen>    If not possible:
Steffen>    Would it be an idea for a future version to add such a feature?

I don't think there is a way.
It would be reasonable to add something.

I think question #5 is implicitly answered by the above...

Steffen> #6 BTW, why does `source ~/.gdbinit.py' work? Is the file name
Steffen>    extension telling that it contains Python syntax?

Yes, see "set script-extension".

Tom