debuginfo server progress update

[Aaron Merey <amerey@redhat.com>]

(resending this email, this time with better formatting!)

Hello,

The debuginfo server, a web service that offers lightweight on-demand lookups
of debuginfo-related data, along with its elfutils client have begun to take
shape so I want to provide an update on where we currently are in its
development and how to use it.

We plan on shipping debuginfo server with elfutils and it is currently
available on the branch 'fche/dbgserver' of the elfutils git repo. The
http web server itself is able to scan a directory tree for any debuginfo,
executable and source files and index them by build-id. The elfutils client
hooks into libdwfl and provides a fallback so that if the library is not able
to locate a debuginfo file, the client, if enabled, will attempt to query
debuginfo servers for debuginfo containing a particular build-id.

As an example, suppose we are trying to trace some program with systemtap.
One of the stap probes we want to run requires DWARF debuginfo but the
executable has been stripped of this information and the corresponding
debuginfo file is not in any of the default directories that stap checks.

    $ stap -e 'probe process("/path/to/exec").function("*") { [...] }'
    semantic error: while resolving probe point: \
      identifier 'process' at t.stp:1:7
            source: probe process("/path/to/exec").function("*") {
    semantic error: no match \
      (similar functions: exit, open, textdomain, tfind, dup)

Systemtap uses elfutils to locate and read debuginfo files, so it is able
to make use of debuginfo server's elfutils client. First we run debuginfo
server and point it to the directory containing the stripped debuginfo.

    $ dbgserver -vvv -p 8002 -F ~/debuginfo_dir &
    [Tue Jul  2 15:00:37] (20459/20462): file-path scanning \
      /home/amerey/debuginfo_dir
    [Tue Jul  2 15:00:37] (20459/20459): Started http server on port 8002, \
      database path /home/amerey/.dbgserver.sqlite
    [Tue Jul  2 15:00:37] (20459/20462): fts traversing \
      /home/amerey/debuginfo_dir
    [Tue Jul  2 15:00:37] (20459/20462): fts traversing \
      /home/amerey/debuginfo_dir/debuginfo
    [Tue Jul  2 15:00:37] (20459/20462): recorded \
      buildid=0a0cd15e690a378ec77359bb2eeb76ea0f8d67f8 \
      file=/home/amerey/debuginfo_dir/debuginfo mtime=1561060856 \
      as executable and debuginfo
    [...]

In order to enable the elfutils client, there must be a $DBGSERVER_URLS
environment variable containing at least one URL of a debuginfo server
(multiple URLs should be separated by spaces). When we run stap again,
elfutils will query the debuginfo server for debuginfo with the build-id
of the executable.

    $ export DBGSERVER_URLS=localhost:8002
    $ stap -v -e 'probe process("/path/to/exec").function("*") { [...] }'
    [...]
    Pass 5: starting run

This time systemtap successfully acquires the debuginfo and begins running
the probe. Although in this example everything is running locally, the server
and the debuginfo could just as well be on a remote host.

Debuginfo server development is ongoing. Some of the features we have planned
include a gdb client that could query debuginfo servers for debuginfo or source
code and the ability for the server to automatically search rpms for any of the
files requested by a client. We also are planning a talk/demo at GNU Cauldron
this September in Montréal, Canada (https://gcc.gnu.org/wiki/cauldron2019).

Hopefully this helped give you a sense of how to use debuginfo server, where it
is in its development, and what is to come. Feedback is always appreciated!

Aaron Merey