Re: Systemtap on Debian squeeze

Re: Systemtap on Debian squeeze

[Javier Ubillos]

Hi.

I'm trying to get systemtap & stap to run in my debian-squeese
installation, but not having much luck.

http://pastebin.com/paCA5Fcd

My best guess is that it has to do with the line:
probe vfs_read@/build/buildd-linux-2.6_3.2.18-1~bpo60+1-amd64-t61foW/linux-2.6-3.2.18/debian/build/source_amd64_none/fs/read_write.c:364
kernel reloc=.dynamic pc=0xffffffff81106420
As I'm running kernel 3.2.0-0.bpo.2-amd64.

Judging by the rest of the output, stap seems to have figured out my
kernelversion correctly, I can't figure out why it would try to go for
2.6-3.2.18 all of a sudden.
Does any one have any suggestions about what I should look at?

Below is the full output and some more info.
// Javier
(spam warning)




jav@dib:~$ uname -a
Linux dib 3.2.0-0.bpo.2-amd64 #1 SMP Sun Jun 3 21:40:57 UTC 2012
x86_64 GNU/Linux


jav@dib:~$ sudo stap -vvv -e 'probe vfs.read {printf("read
performed\n"); exit()}'
SystemTap translator/driver (version 1.2/0.153 non-git sources)
Copyright (C) 2005-2010 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
Session arch: x86_64 release: 3.2.0-0.bpo.2-amd64
Created temporary directory "/tmp/stapOSQDuk"
Parsed kernel "/lib/modules/3.2.0-0.bpo.2-amd64/build/.config", number
of tuples: 4034
Parsed kernel "/lib/modules/3.2.0-0.bpo.2-amd64/build/Module.symvers",
number of vmlinux exports: 5159
Searched "/usr/share/systemtap/tapset/x86_64/*.stp", found 3
Searched "/usr/share/systemtap/tapset/*.stp", found 65
Pass 1: parsed user script and 68 library script(s) using
52456virt/21072res/1864shr kb, in 70usr/0sys/73real ms.
blacklist regexps:
blfn: ^(atomic_notifier_call_chain|default_do_nmi|__die|die_nmi|do_debug|do_general_protection|do_int3|do_IRQ|do_page_fault|do_sparc64_fault|do_trap|dummy_nmi_callback|flush_icache_range|ia64_bad_break|ia64_do_page_fault|ia64_fault|io_check_error|mem_parity_error|nmi_watchdog_tick|notifier_call_chain|oops_begin|oops_end|program_check_exception|single_step_exception|sync_regs|unhandled_fault|unknown_nmi_error|.*raw_.*lock.*|.*read_.*lock.*|.*write_.*lock.*|.*spin_.*lock.*|.*rwlock_.*lock.*|.*rwsem_.*lock.*|.*mutex_.*lock.*|raw_.*|.*seq_.*lock.*|atomic_.*|atomic64_.*|get_bh|put_bh|.*apic.*|.*APIC.*|.*softirq.*|.*IRQ.*|.*_intr.*|__delay|.*kernel_text.*|get_current|current_.*|.*exception_tables.*|.*setup_rt_frame.*|.*preempt_count.*|preempt_schedule|__switch_to|special_mapping_.*)$
blfn_ret: ^(do_exit|sys_exit|sys_exit_group)$
blfile: ^(kernel/kprobes\.c|arch/.*/kernel/kprobes\.c|include/asm/io\.h|include/asm/bitops\.h|arch/.*/include/asm/io\.h|arch/.*/include/asm/bitops\.h|drivers/ide/ide-iops\.c)$
blsection: ^(\.init\.|\.exit\.|\.devinit\.|\.devexit\.|\.cpuinit\.|\.cpuexit\.|\.meminit\.|\.memexit\.)
parsed 'vfs_read', func 'vfs_read'
focused on module 'kernel = [0xffffffff81000000-0xffffffff81947000,
bias 0x0] file /usr/lib/debug/boot/vmlinux-3.2.0-0.bpo.2-amd64 ELF
machine |x86_64 (code 62)
focused on module 'kernel'
selected function vfs_read
probe vfs_read@/build/buildd-linux-2.6_3.2.18-1~bpo60+1-amd64-t61foW/linux-2.6-3.2.18/debian/build/source_amd64_none/fs/read_write.c:364
kernel reloc=.dynamic pc=0xffffffff81106420
entry-pc lookup (dwarf_entrypc dieoffset: 0x1146330) = 0xffffffff81106420 (rc 0)
finding location for local 'file' near address 0xffffffff81106420,
module bias 0x0
get_cfa_ops @0xffffffff81106420, module_start @0xffffffff81000000

		

 jav@dib:~$ dpkg -l |egrep -i "linux-image|linux-headers"
ii  linux-headers-2.6.32-5.spotify-common           2.6.32-41+spotify2
                    Common header files for Linux 2.6.32-5.spotify
ii  linux-headers-3.2.0-0.bpo.1-common              3.2.4-1~bpo60+1
                    Common header files for Linux 3.2.0-0.bpo.1
ii  linux-headers-3.2.0-0.bpo.2-amd64               3.2.18-1~bpo60+1
                    Header files for Linux 3.2.0-0.bpo.2-amd64
ii  linux-headers-3.2.0-0.bpo.2-common              3.2.18-1~bpo60+1
                    Common header files for Linux 3.2.0-0.bpo.2
ii  linux-image-2.6-amd64                           3.2+44~bpo60+1
                    Linux for 64-bit PCs (dummy package)
ii  linux-image-2.6.32-5-amd64                      2.6.32-45
                    Linux 2.6.32 for 64-bit PCs
rc  linux-image-2.6.32-5.spotify-amd64              2.6.32-41+spotify2
                    Linux 2.6.32 for 64-bit PCs
ii  linux-image-3.2.0-0.bpo.1-amd64                 3.2.4-1~bpo60+1
                    Linux 3.2 for 64-bit PCs
ii  linux-image-3.2.0-0.bpo.2-amd64                 3.2.18-1~bpo60+1
                    Linux 3.2 for 64-bit PCs
ii  linux-image-3.2.0-0.bpo.2-amd64-dbg             3.2.18-1~bpo60+1
                    Debugging infos for Linux 3.2.0-0.bpo.2-amd64
ii  linux-image-amd64                               3.2+44~bpo60+1
                    Linux for 64-bit PCs (meta-package)

Re: Systemtap on Debian squeeze

[Javier Ubillos]

For what it's worth.
The following example works:
stap -ve 'probe begin { log("hello world") exit () }'
(outputs "hello world")

However, the next example does not work:
stap -c df -e 'probe syscall.* { if (target()==pid()) log(name." ".argstr) }'

(more spam output)
Systemtap translator/driver (version 1.7/0.153 Debian version 1.7-1+b1 (sid))
Copyright (C) 2005-2012 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: AVAHI LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS
Created temporary directory "/tmp/stapKnevnf"
Session arch: x86_64 release: 3.2.0-0.bpo.2-amd64
Parsed kernel "/lib/modules/3.2.0-0.bpo.2-amd64/build/.config",
containing 4034 tuples
Parsed kernel /lib/modules/3.2.0-0.bpo.2-amd64/build/Module.symvers,
which contained 5159 vmlinux exports
Searched: " /usr/share/systemtap/tapset/x86_64/*.stp ", found: 4, processed: 4
Searched: " /usr/share/systemtap/tapset/*.stp ", found: 77, processed: 77
Pass 1: parsed user script and 81 library script(s) using
78032virt/22456res/2564shr kb, in 60usr/10sys/76real ms.
wildcard '*' matched 'accept'
Attempting to extract kernel debuginfo build ID from
/lib/modules/3.2.0-0.bpo.2-amd64/build/vmlinux.id
Attempting to extract kernel debuginfo build ID from /sys/kernel/notes
blacklist regexps:
blfn: ^(atomic_notifier_call_chain|default_do_nmi|__die|die_nmi|do_debug|do_general_protection|do_int3|do_IRQ|do_page_fault|do_sparc64_fault|do_trap|dummy_nmi_callback|flush_icache_range|ia64_bad_break|ia64_do_page_fault|ia64_fault|io_check_error|mem_parity_error|nmi_watchdog_tick|notifier_call_chain|oops_begin|oops_end|program_check_exception|single_step_exception|sync_regs|unhandled_fault|unknown_nmi_error|xen_[gs]et_debugreg|xen_irq_.*|xen_.*_fl_direct.*|check_events|xen_adjust_exception_frame|xen_iret.*|xen_sysret64.*|test_ti_thread_flag.*|inat_get_opcode_attribute|system_call_after_swapgs|.*raw_.*_lock.*|.*raw_.*_unlock.*|.*raw_.*_trylock.*|.*read_lock.*|.*read_unlock.*|.*read_trylock.*|.*write_lock.*|.*write_unlock.*|.*write_trylock.*|.*write_seqlock.*|.*write_sequnlock.*|.*spin_lock.*|.*spin_unlock.*|.*spin_trylock.*|.*spin_is_locked.*|rwsem_.*lock.*|.*mutex_.*lock.*|raw_.*|atomic_.*|atomic64_.*|get_bh|put_bh|.*apic.*|.*APIC.*|.*softirq.*|.*IRQ.*|.*_intr.*|__delay|.*kernel_text.*|get_current|current_.*|.*exception_tables.*|.*setup_rt_frame.*|.*preempt_count.*|preempt_schedule|__switch_to|special_mapping_.*|.*_pte_.*)$
blfn_ret: ^(do_exit|sys_exit|sys_exit_group)$
blfile: ^(kernel/kprobes\.c|arch/.*/kernel/kprobes\.c|.*/include/asm/io\.h|.*/include/asm/io_64\.h|.*/include/asm/bitops\.h|drivers/ide/ide-iops\.c|arch/.*/kernel/paravirt\.c|.*/include/asm/paravirt\.h|fs/seq_file\.c)$
blsection: ^(\.init\.|\.exit\.|\.devinit\.|\.devexit\.|\.cpuinit\.|\.cpuexit\.|\.meminit\.|\.memexit\.)
parse 'sys_accept4', func 'sys_accept4'
focused on module 'kernel' = [0xffffffff81000000-0xffffffff81947000,
bias 0 file /usr/lib/debug/boot/vmlinux-3.2.0-0.bpo.2-amd64 ELF
machine |x86_64 (code 62)
focused on module 'kernel'
selected function sys_accept4
selected function sys_accept4
probe sys_accept4@/build/buildd-linux-2.6_3.2.18-1~bpo60+1-amd64-t61foW/linux-2.6-3.2.18/debian/build/source_amd64_none/net/socket.c:1487
kernel reloc=.dynamic pc=0xffffffff8129603b
finding location for local 'flags' near address 0xffffffff8129603b,
module bias 0
get_cfa_ops @0xffffffff8129603b, module_start @0xffffffff81000000


Yours
// Javier

2012/6/11 Javier Ubillos <javier@ubillos.org>:
> Hi.
>
> I'm trying to get systemtap & stap to run in my debian-squeese
> installation, but not having much luck.
>
> http://pastebin.com/paCA5Fcd
>
> My best guess is that it has to do with the line:
> probe vfs_read@/build/buildd-linux-2.6_3.2.18-1~bpo60+1-amd64-t61foW/linux-2.6-3.2.18/debian/build/source_amd64_none/fs/read_write.c:364
> kernel reloc=.dynamic pc=0xffffffff81106420
> As I'm running kernel 3.2.0-0.bpo.2-amd64.
>
> Judging by the rest of the output, stap seems to have figured out my
> kernelversion correctly, I can't figure out why it would try to go for
> 2.6-3.2.18 all of a sudden.
> Does any one have any suggestions about what I should look at?
>
> Below is the full output and some more info.
> // Javier
> (spam warning)
>
>
>
>
> jav@dib:~$ uname -a
> Linux dib 3.2.0-0.bpo.2-amd64 #1 SMP Sun Jun 3 21:40:57 UTC 2012
> x86_64 GNU/Linux
>
>
> jav@dib:~$ sudo stap -vvv -e 'probe vfs.read {printf("read
> performed\n"); exit()}'
> SystemTap translator/driver (version 1.2/0.153 non-git sources)
> Copyright (C) 2005-2010 Red Hat, Inc. and others
> This is free software; see the source for copying conditions.
> Session arch: x86_64 release: 3.2.0-0.bpo.2-amd64
> Created temporary directory "/tmp/stapOSQDuk"
> Parsed kernel "/lib/modules/3.2.0-0.bpo.2-amd64/build/.config", number
> of tuples: 4034
> Parsed kernel "/lib/modules/3.2.0-0.bpo.2-amd64/build/Module.symvers",
> number of vmlinux exports: 5159
> Searched "/usr/share/systemtap/tapset/x86_64/*.stp", found 3
> Searched "/usr/share/systemtap/tapset/*.stp", found 65
> Pass 1: parsed user script and 68 library script(s) using
> 52456virt/21072res/1864shr kb, in 70usr/0sys/73real ms.
> blacklist regexps:
> blfn: ^(atomic_notifier_call_chain|default_do_nmi|__die|die_nmi|do_debug|do_general_protection|do_int3|do_IRQ|do_page_fault|do_sparc64_fault|do_trap|dummy_nmi_callback|flush_icache_range|ia64_bad_break|ia64_do_page_fault|ia64_fault|io_check_error|mem_parity_error|nmi_watchdog_tick|notifier_call_chain|oops_begin|oops_end|program_check_exception|single_step_exception|sync_regs|unhandled_fault|unknown_nmi_error|.*raw_.*lock.*|.*read_.*lock.*|.*write_.*lock.*|.*spin_.*lock.*|.*rwlock_.*lock.*|.*rwsem_.*lock.*|.*mutex_.*lock.*|raw_.*|.*seq_.*lock.*|atomic_.*|atomic64_.*|get_bh|put_bh|.*apic.*|.*APIC.*|.*softirq.*|.*IRQ.*|.*_intr.*|__delay|.*kernel_text.*|get_current|current_.*|.*exception_tables.*|.*setup_rt_frame.*|.*preempt_count.*|preempt_schedule|__switch_to|special_mapping_.*)$
> blfn_ret: ^(do_exit|sys_exit|sys_exit_group)$
> blfile: ^(kernel/kprobes\.c|arch/.*/kernel/kprobes\.c|include/asm/io\.h|include/asm/bitops\.h|arch/.*/include/asm/io\.h|arch/.*/include/asm/bitops\.h|drivers/ide/ide-iops\.c)$
> blsection: ^(\.init\.|\.exit\.|\.devinit\.|\.devexit\.|\.cpuinit\.|\.cpuexit\.|\.meminit\.|\.memexit\.)
> parsed 'vfs_read', func 'vfs_read'
> focused on module 'kernel = [0xffffffff81000000-0xffffffff81947000,
> bias 0x0] file /usr/lib/debug/boot/vmlinux-3.2.0-0.bpo.2-amd64 ELF
> machine |x86_64 (code 62)
> focused on module 'kernel'
> selected function vfs_read
> probe vfs_read@/build/buildd-linux-2.6_3.2.18-1~bpo60+1-amd64-t61foW/linux-2.6-3.2.18/debian/build/source_amd64_none/fs/read_write.c:364
> kernel reloc=.dynamic pc=0xffffffff81106420
> entry-pc lookup (dwarf_entrypc dieoffset: 0x1146330) = 0xffffffff81106420 (rc 0)
> finding location for local 'file' near address 0xffffffff81106420,
> module bias 0x0
> get_cfa_ops @0xffffffff81106420, module_start @0xffffffff81000000
>
>
>
>  jav@dib:~$ dpkg -l |egrep -i "linux-image|linux-headers"
> ii  linux-headers-2.6.32-5.spotify-common           2.6.32-41+spotify2
>                    Common header files for Linux 2.6.32-5.spotify
> ii  linux-headers-3.2.0-0.bpo.1-common              3.2.4-1~bpo60+1
>                    Common header files for Linux 3.2.0-0.bpo.1
> ii  linux-headers-3.2.0-0.bpo.2-amd64               3.2.18-1~bpo60+1
>                    Header files for Linux 3.2.0-0.bpo.2-amd64
> ii  linux-headers-3.2.0-0.bpo.2-common              3.2.18-1~bpo60+1
>                    Common header files for Linux 3.2.0-0.bpo.2
> ii  linux-image-2.6-amd64                           3.2+44~bpo60+1
>                    Linux for 64-bit PCs (dummy package)
> ii  linux-image-2.6.32-5-amd64                      2.6.32-45
>                    Linux 2.6.32 for 64-bit PCs
> rc  linux-image-2.6.32-5.spotify-amd64              2.6.32-41+spotify2
>                    Linux 2.6.32 for 64-bit PCs
> ii  linux-image-3.2.0-0.bpo.1-amd64                 3.2.4-1~bpo60+1
>                    Linux 3.2 for 64-bit PCs
> ii  linux-image-3.2.0-0.bpo.2-amd64                 3.2.18-1~bpo60+1
>                    Linux 3.2 for 64-bit PCs
> ii  linux-image-3.2.0-0.bpo.2-amd64-dbg             3.2.18-1~bpo60+1
>                    Debugging infos for Linux 3.2.0-0.bpo.2-amd64
> ii  linux-image-amd64                               3.2+44~bpo60+1
>                    Linux for 64-bit PCs (meta-package)

Re: Systemtap on Debian squeeze

[Timo Juhani Lindfors]

Javier Ubillos <javier@ubillos.org> writes:
> I'm trying to get systemtap & stap to run in my debian-squeese
> installation, but not having much luck.

What does

http://lindi.iki.fi/lindi/systemtap/diagnose-error-cases.bash

print for you?

Re: Systemtap on Debian squeeze

[Javier Ubillos]

It outputs nothing.
Should I give it some commandline argument?

// Jav

2012/6/12 Timo Juhani Lindfors <timo.lindfors@iki.fi>:
> Javier Ubillos <javier@ubillos.org> writes:
>> I'm trying to get systemtap & stap to run in my debian-squeese
>> installation, but not having much luck.
>
> What does
>
> http://lindi.iki.fi/lindi/systemtap/diagnose-error-cases.bash
>
> print for you?

Re: Systemtap on Debian squeeze

[Adrien Kunysz]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

I guess something like this could help to figure out what's wrong
(if not with your system, with that script):

$ bash -x diagnose-error-cases.bash

On Tue, Jun 12, 2012 at 12:04:44PM +0200, Javier Ubillos wrote:
> It outputs nothing.
> Should I give it some commandline argument?
> 
> // Jav
> 
> 2012/6/12 Timo Juhani Lindfors <timo.lindfors@iki.fi>:
> > Javier Ubillos <javier@ubillos.org> writes:
> >> I'm trying to get systemtap & stap to run in my debian-squeese
> >> installation, but not having much luck.
> >
> > What does
> >
> > http://lindi.iki.fi/lindi/systemtap/diagnose-error-cases.bash
> >
> > print for you?

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Systemtap on Debian squeeze

[Javier Ubillos]

I got some help earlier today on IRC.

It turned out my problem was I did not have elfutils installed (and
that required a downgrade of libdw1).

We found that out by running gdb on stap and printing a stacktrace (IIRC).

// Jav

2012/6/12 Adrien Kunysz <adrien@kunysz.be>:
> I guess something like this could help to figure out what's wrong
> (if not with your system, with that script):
>
> $ bash -x diagnose-error-cases.bash
>
> On Tue, Jun 12, 2012 at 12:04:44PM +0200, Javier Ubillos wrote:
>> It outputs nothing.
>> Should I give it some commandline argument?
>>
>> // Jav
>>
>> 2012/6/12 Timo Juhani Lindfors <timo.lindfors@iki.fi>:
>> > Javier Ubillos <javier@ubillos.org> writes:
>> >> I'm trying to get systemtap & stap to run in my debian-squeese
>> >> installation, but not having much luck.
>> >
>> > What does
>> >
>> > http://lindi.iki.fi/lindi/systemtap/diagnose-error-cases.bash
>> >
>> > print for you?
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iEYEARECAAYFAk/XrBEACgkQKLX03ZSPZGx+nQCfcLnnrIj2Qts4Un+35qaj1vS3
> FVEAnRLx/lg62bmguMU3msWhgilKkN6B
> =a6Ub
> -----END PGP SIGNATURE-----
>

Re: Systemtap on Debian squeeze

[Timo Juhani Lindfors]

Javier Ubillos <javier@ubillos.org> writes:
> It turned out my problem was I did not have elfutils installed (and
> that required a downgrade of libdw1).

So was apt-get install elfutils enough to fix the problem?

Re: Systemtap on Debian squeeze

[Timo Juhani Lindfors]

Javier Ubillos <javier@ubillos.org> writes:
> Maybe make it a dependency for the .deb?

We can't change systemtap in squeeze anymore. systemtap in wheezy works
just fine here without elfutils.