Unable to access function parameters

Unable to access function parameters

[Sergey Oboguev]

Hello,

I am a newbie at SystemTap and just installed it, the latest version from git.
I seem to be unable to access the values of function parameters, please see examples below.

I am using custom-built 3.9 kernel that was built with gcc 4.7.2 with debug infromation included and vmlinux-(id) copied alongside vmlinuz-(id) into /boot.

Some potentially relevant config-(id) settings:

CONFIG_DEBUG_FS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_RODATA=y
CONFIG_KPROBES=y
CONFIG_KPROBES_ON_FTRACE=y
CONFIG_RELAY=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENT=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y

Other relevant data and invocation examples follow below in this message.

I'd appreciate very much an advise as to how this could be fixed.

Thanks,
Sergey

==============================

$ uname -a
Linux csb 3.9.0-1-my #2 SMP PREEMPT Tue May 7 06:56:42 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux

$ gcc --version 
gcc (Ubuntu/Linaro 4.7.2-2ubuntu1) 4.7.2

$ ls -ld /boot
drwxr-xr-x 3 root root 4096 Nov  5 12:03 /boot

$ ls -l /boot/vm*3.9*
-rwxr-xr-x 1 root root 177418914 Nov  5 12:03 /boot/vmlinux-3.9.0-1-my
-rw-r--r-- 1 root root   5464672 May  7  2013 /boot/vmlinuz-3.9.0-1-my

$ objdump --dwarf /boot/vmlinux-3.9.0-1-my >~/q.q
objdump: Warning: There is a hole [0x12ddd - 0x12e2d] in .debug_loc section.
objdump: Warning: There is a hole [0x3e9f5 - 0x3ea45] in .debug_loc section.
objdump: Warning: There is a hole [0x3ea55 - 0x3eaa5] in .debug_loc section.
objdump: Warning: There is a hole [0x3eab5 - 0x3eb05] in .debug_loc section.
objdump: Warning: There is a hole [0x3eb15 - 0x3eb65] in .debug_loc section.
objdump: Warning: There is a hole [0x3eb75 - 0x3ebc5] in .debug_loc section.
objdump: Warning: There is a hole [0x3ebd5 - 0x3ec25] in .debug_loc section.
objdump: Warning: There is a hole [0x3ec35 - 0x3ec85] in .debug_loc section.
objdump: Warning: There is a hole [0x3ec95 - 0x3ece5] in .debug_loc section.
objdump: Warning: There is a hole [0x3ecf5 - 0x3ed45] in .debug_loc section.
objdump: Warning: There is a hole [0x3ed55 - 0x3eda5] in .debug_loc section.
objdump: Warning: There is a hole [0x3edb5 - 0x3ee05] in .debug_loc section.
objdump: Warning: There is a hole [0x425e4 - 0x42634] in .debug_loc section.
objdump: Warning: There is a hole [0x7592a - 0x7597a] in .debug_loc section.
objdump: Warning: There is a hole [0x1673c9 - 0x167419] in .debug_loc section.
objdump: Warning: There is a hole [0xd22a10 - 0xd22a50] in .debug_loc section.
objdump: Warning: There is a hole [0xd2acd1 - 0xd2acf9] in .debug_loc section.
objdump: Warning: There is a hole [0xd4af37 - 0xd4af5f] in .debug_loc section.
objdump: Warning: There is a hole [0xd58d99 - 0xd58dc1] in .debug_loc section.

$ ls -l ~/q.q
-rw-rw-r-- 1 sergey sergey 2772062415 Nov  6 09:51 /home/sergey/q.q

$ grep filename q.q
[...]
    <8ca43>   DW_AT_name        : (indirect string, offset: 0x5e107): filename    
    <8a60af>   DW_AT_name        : (indirect string, offset: 0x5e107): filename    
    <8a619c>   DW_AT_name        : (indirect string, offset: 0x5e107): filename    
    <8a623a>   DW_AT_name        : (indirect string, offset: 0x5e107): filename    
[... goes on ...]

$ sudo stap --version
Systemtap translator/driver (version 2.4/0.153, commit release-2.3-233-g5c0aabb + changes)

$ sudo stap -e 'probe syscall.open { printf("hit\n"); }'
hit
hit
hit
hit

$ sudo stap -v -e 'probe syscall.open { printf("hit: %s\n", filename); }'
Pass 1: parsed user script and 157 library script(s) using 102200virt/42448res/2468shr/40872data kb, in 150usr/0sys/159real ms.
semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/linux/syscalls2.stp:128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/syscalls2.stp:127:25
        source:     filename = user_string($filename)
                                           ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :127:25
        source:     filename = user_string($filename)
                                           ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/linux/syscalls2.stp:128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/syscalls2.stp:127:25
        source:     filename = user_string($filename)
                                           ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :127:25
        source:     filename = user_string($filename)
                                           ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/linux/syscalls2.stp:128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :128:32
        source:     filename = user_string_quoted($filename)
                                                  ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/syscalls2.stp:127:25
        source:     filename = user_string($filename)
                                           ^

semantic error: not accessible at this address [man error::dwarf] (0xffffffff811a0340, dieoffset: 0x1834708): identifier '$filename' at :127:25
        source:     filename = user_string($filename)


$ sudo stap -vvv -e 'probe syscall.open { printf("hit: %s\n", filename); }'
[...]
Resolution problem with probe kernel.function("compat_sys_open@fs/compat.c:1280").call?
{
(filename) = (user_string_quoted($filename))
printf("hit: %s\\n", filename)
}
semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/linux/syscalls2.stp:128:32
        source:     filename = user_string_quoted($filename)
                                                  ^
[...]


$ sudo stap -e 'probe syscall.open { printf("hit: %s\n", $filename); }'
semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at <input>:1:42
        source: probe syscall.open { printf("hit: %s\n", $filename); }


$ sudo stap -L syscall.open
syscall.open name:string filename:string flags:long mode:unknown argstr:unknown

$ sudo stap -e 'probe syscall.open { printf("hit: %s\n", $$vars); }'
hit: filename=? flags=? mode=? ret=0x2
hit: filename=? flags=? mode=? ret=0x2
hit: filename=? flags=? mode=? ret=0x2
hit: filename=? flags=? mode=? ret=0x2
hit: filename=? flags=? mode=? ret=0x2

$ sudo stap -e 'probe syscall.exec* { printf("hit: %s: %s\n", name, $$vars); }'
hit: execve: filename=? argv=? envp=? path=? error=?
hit: execve: filename=? __argv=? __envp=? argv={...} envp={...}
hit: execve: filename=? argv=? envp=? path=? error=?
hit: execve: filename=? __argv=? __envp=? argv={...} envp={...}

$ sudo stap -e 'probe syscall.read* { printf("hit: %s: %s\n", name, $$vars); }'
hit: read: fd=? buf=? count=? f={...} ret=?
hit: read: fd=? buf=? count=? f={...} ret=?
hit: read: fd=? buf=? count=? f={...} ret=?

Re: Unable to access function parameters

[Josh Stone]

On 11/06/2013 11:01 AM, Sergey Oboguev wrote:
> Hello,
> 
> I am a newbie at SystemTap and just installed it, the latest version from git.
> I seem to be unable to access the values of function parameters, please see examples below.
> 
> I am using custom-built 3.9 kernel that was built with gcc 4.7.2 with debug infromation included and vmlinux-(id) copied alongside vmlinuz-(id) into /boot.
[...]
> $ objdump --dwarf /boot/vmlinux-3.9.0-1-my >~/q.q
> objdump: Warning: There is a hole [0x12ddd - 0x12e2d] in .debug_loc section.

I suspect you're hitting this:
https://bugzilla.redhat.com/show_bug.cgi?id=904252
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56154

However, even though 56154 seems to indicate it should be fixed in 4.8,
the example in 904252 still fails with gcc-4.8.2-1.fc19.x86_64, so we
may need to ping Alexandre and Jakub on this again.

Re: Unable to access function parameters

[Frank Ch. Eigler]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 1132 bytes --]

Sergey Oboguev <oboguev@yahoo.com> writes:

> [...]
> I am a newbie at SystemTap and just installed it, the latest version from git.

Welcome!

> I seem to be unable to access the values of function parameters, please see examples below. [...]

OK, thanks for the information.

> $ sudo stap -v -e 'probe syscall.open { printf("hit: %s\n", filename); }'
> Pass 1: parsed user script and 157 library script(s) using 102200virt/42448res/2468shr/40872data kb, in 150usr/0sys/159real ms.
> semantic error: not accessible at this address [man error::dwarf] (0xffffffff811f35c0, dieoffset: 0x1c34036): identifier '$filename' at /usr/share/systemtap/tapset/linux/syscalls2.stp:128:32
>         source:     filename = user_string_quoted($filename)
>                                                   ^

This sounds familiar, possibly
https://sourceware.org/bugzilla/show_bug.cgi?id=15123

If you're using that older gcc, and your kernel includes
CONFIG_HAVE_FENTRY, and thus is compiled with CFLAGS+=-mfentry, but
not also -grecord-gcc-switches, then try

# export PR15123_ASSUME_MFENTRY=1
# stap ....


- FChE

Re: Unable to access function parameters

[Sergey Oboguev]

Hi Frank,

> If you're using that older gcc, and your kernel includes
> CONFIG_HAVE_FENTRY, and thus is compiled with CFLAGS+=-mfentry, but
> not also -grecord-gcc-switches, then try

> # export PR15123_ASSUME_MFENTRY=1
> # stap ....

This did the trick.

Thank you very much for the help.

- Sergey