From: William Cohen <wcohen@redhat.com>
To: Martin Hunt <hunt@redhat.com>
Cc: "systemtap@sources.redhat.com" <systemtap@sources.redhat.com>
Subject: Suggestions for localizing problems with identifiers in SystemTap.
Date: Mon, 27 Feb 2006 02:24:00 -0000 [thread overview]
Message-ID: <440262BF.8080303@redhat.com> (raw)
In-Reply-To: <1140465215.3791.11.camel@monkey2>
Martin Hunt wrote:
>>FC5 i686 Linux 2.6.15-1.1948_FC5smp
>>
>>Stap translator summary of failed tests
>>buildok-seven.stp:
>> Unable to find identifier $type in function find_pid
>> Unable to find identifier $task in function detach_pid
>
>
> There don't appear to be significant changes in the sources to account
> for this. That makes it a likely gcc bug.
>
> Can someone write up the proper procedure to use to report these? I
> think it should be on the web page.
>
> Martin
I have jotted down some notes on how to track down these types of
problems with the identifiers. The notes are below. There are still lots
of holes.
-Will
Suggestions for localizing problems with identifiers in SystemTap.
Possible causes for this problems symbols in SystemTap scripts:
1) source code doesn't have function, variable, or argument
2) GCC tool chain not generating the proper debug information
3) system tap translator is unable them in the debugging information
-----
Source code doesn't have function, variable, or argument
This can be determined by examining the source code.
1) Find out which kernel is being used and location of source
code. You will need to have the cscope rpm installed
uname -r
get the matching source SRPM.
install srpm on the machine with:
rpm -Uvh kernel-<version>.src.rpm
cd into SPEC directory
rpmbuild -bp kernel-2.6.rpm
cd into BUILD/kernel-<version>/linux-<version> directory
make mrproper
cp /boot/config-`uname -r` .config; make oldconfig; make cscope
2) Find where the problem function is located and examine code for
variable or arument name. Ccsope is extremely useful for navigating
locating the functions.
Start up cscope and use the following cscope entries:
"Find this global definition:" or "Find this egrep pattern:"
Type in function name or pattern
If function shows up, hit return to look at function by hitting enter.
cscope will start an editor. Take a look for the problem symbol
if the function is "static" or "static inline",
the compiler could optimize these away if not called.
cscope can also show where the function is used
-----
GCC toolchain not generating the proper debug information
After establishing that the the symbol exists in source code,
determine whether the compiler is generating the proper debug
information. The program "readelf" in the binutils RPM can be used
the examine the debugging information in the kernel and modules.
readelf --debug-dump /usr/lib/debug/lib/modules/`uname -r`/vmlinux >
/tmp/debuginfo
This will generate a very large file /tmp/debuginfo that can be
examined. For example want to find out why $task is not being found
in function detach_pid for src/testsuite/buildok/seven.stp.
Looking for the section of the output in /tmp/debuginfo for the
detach_pid function we find the following section:
<1><3554ae>: Abbrev Number: 83 (DW_TAG_subprogram)
DW_AT_sibling : <35550b>
DW_AT_external : 1
DW_AT_name : (indirect string, offset: 0x1c783): detach_pid
DW_AT_decl_file : 1
DW_AT_decl_line : 193
DW_AT_prototyped : 1
DW_AT_low_pc : 0xffffffff80143458
DW_AT_high_pc : 0xffffffff80143499
DW_AT_frame_base : 0x8ee43 (location list)
<2><3554cf>: Abbrev Number: 76 (DW_TAG_formal_parameter)
DW_AT_name : (indirect string, offset: 0xba9d3): task
DW_AT_decl_file : 1
DW_AT_decl_line : 192
DW_AT_type : <35184d>
DW_AT_location : 0x8eea3 (location list)
<2><3554de>: Abbrev Number: 76 (DW_TAG_formal_parameter)
DW_AT_name : (indirect string, offset: 0x4fb9b): type
DW_AT_decl_file : 1
DW_AT_decl_line : 192
DW_AT_type : <35062a>
DW_AT_location : 0x8eec6 (location list)
<2><3554ed>: Abbrev Number: 72 (DW_TAG_variable)
DW_AT_name : tmp
DW_AT_decl_file : 1
DW_AT_decl_line : 194
DW_AT_type : <34e505>
DW_AT_location : 0x8ef0f (location list)
<2><3554fc>: Abbrev Number: 72 (DW_TAG_variable)
DW_AT_name : nr
DW_AT_decl_file : 1
DW_AT_decl_line : 194
DW_AT_type : <34e505>
DW_AT_location : 0x8ef45 (location list)
It is clear the "task" identifier is in the debugging information and
that it is a argument being passed into the function. The type of the
identifier can be determined by by searching for the "DW_AT_type", in
this case "><35184d>":
<1><35184d>: Abbrev Number: 7 (DW_TAG_pointer_type)
DW_AT_byte_size : 8
DW_AT_type : <350f16>
The chain can be followed searching for "><350f16>":
<1><350f16>: Abbrev Number: 6 (DW_TAG_typedef)
DW_AT_name : (indirect string, offset: 0x782ad): task_t
DW_AT_decl_file : 9
DW_AT_decl_line : 184
DW_AT_type : <34e8a0>
Searching for "><34e8a0>" yield task "task_struct" followed by its fields:
<1><34e8a0>: Abbrev Number: 15 (DW_TAG_structure_type)
DW_AT_sibling : <34f062>
DW_AT_name : (indirect string, offset: 0x15daa): task_struct
DW_AT_byte_size : 1888
DW_AT_decl_file : 10
DW_AT_decl_line : 13
<2><34e8ad>: Abbrev Number: 16 (DW_TAG_member)
DW_AT_name : (indirect string, offset: 0xb09cc): state
DW_AT_decl_file : 9
DW_AT_decl_line : 701
DW_AT_type : <34f60a>
DW_AT_data_member_location: 2 byte block: 23 0
(DW_OP_plus_uconst: 0)
<2><34e8bc>: Abbrev Number: 16 (DW_TAG_member)
DW_AT_name : (indirect string, offset: 0x8efe): thread_info
DW_AT_decl_file : 9
DW_AT_decl_line : 702
DW_AT_type : <35191d>
DW_AT_data_member_location: 2 byte block: 23 8
(DW_OP_plus_uconst: 8)
-----
SystemTap (stap) is not finding the identiers in the debugging information
After eliminating the source code and compiler, the SystemTap
translator is the likely suspect. The systemtap translator might be
able to find the identifier but is unable to process it. For example
src/testsuite/buildok/seve.stp instruments the detach_pid funtion in
the kernel and on the x86_64 produces the following error message:
semantic error: unresolved target-symbol expression: identifier '$task'
at
/home/wcohen/stap_testing_200602261550/src/testsuite/buildok/seven.stp:19:35
One can use one (or more) "-v" (verbose) options with the SystemTap
translator to get get a clearer picture of what the translator is
doing and getting some idea of why the symbol is not being resolved
correctly. The following command was used:
stap -p4 -vvvv ../src/testsuite/buildok/seven.stp
The previous command's output included:
pattern 'detach_pid' matches function 'detach_pid'
selected function detach_pid
finding prologue for 'detach_pid' entrypc=0xffffffff80143458
highpc=0xffffffff80143499
finding location for local 'task' near address ffffffff8014345f, module
bias 0
finding location for local 'type' near address ffffffff8014345f, module
bias 0
finding location for local 'task' near address ffffffff8014345f, module
bias 0
finding location for local 'type' near address ffffffff8014345f, module
bias 0
Thus, the 'task' local is found. However, the SystemTap translator
appears to be unable to handle it.
next prev parent reply other threads:[~2006-02-27 2:24 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-02-20 16:40 Summary of nightly tests 20060220 William Cohen
2006-02-20 19:52 ` Martin Hunt
2006-02-20 20:05 ` William Cohen
2006-02-20 20:36 ` Frank Ch. Eigler
2006-02-27 2:24 ` William Cohen [this message]
2006-02-27 4:40 ` Suggestions for localizing problems with identifiers in SystemTap Roland McGrath
2006-02-27 15:57 ` William Cohen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=440262BF.8080303@redhat.com \
--to=wcohen@redhat.com \
--cc=hunt@redhat.com \
--cc=systemtap@sources.redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).