no access to local variables in function

no access to local variables in function

[Lentes, Bernd]

Hi,

i'm new to systemtap. I try to check some local variables of kernel functions. I have a SLES 10 SP4, kernel 2.6.16.60-0.103.1-smp. I'm using systemtap 0.8.
I'm following the instructions from https://sourceware.org/systemtap/SystemTap_Beginners_Guide/targetvariables.html .

If I try to use the example from this page I get:
idcc-devel:~/systemtap # stap -L 'kernel.function("vfs_read")'
kernel.function("vfs_read@fs/read_write.c:248")

Checking other functions with local variables also does not work:
idcc-devel:~/systemtap # stap -L 'kernel.function("local_bh_enable")'
kernel.function("local_bh_enable@kernel/softirq.c:139")

idcc-devel:~/systemtap # stap -L 'kernel.function("__tasklet_schedule")'
kernel.function("__tasklet_schedule@kernel/softirq.c:224")

idcc-devel:~/systemtap # stap -L 'kernel.function("tasklet_kill_immediate")'
kernel.function("tasklet_kill_immediate@kernel/softirq.c:439")

I don't see any local variable. What can I do ?

Btw: is there a chance to monitor the variables of a blacklisted function ?

Thanks.


Bernd

--
Bernd Lentes

Systemadministration
Institut für Entwicklungsgenetik
Gebäude 35.34 - Raum 208
HelmholtzZentrum münchen
bernd.lentes@helmholtz-muenchen.de
phone: +49 89 3187 1241
fax:   +49 89 3187 2294
http://www.helmholtz-muenchen.de/idg

Je suis Charlie


Helmholtz Zentrum München
Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir´in Bärbel Brumme-Bothe
Geschäftsführer: Prof. Dr. Günther Wess, Dr. Nikolaus Blum, Dr. Alfons Enhsen
Registergericht: Amtsgericht München HRB 6466
USt-IdNr: DE 129521671

RE: no access to local variables in function

[Lentes, Bernd]

Bernd wrote:

> -----Original Message-----
> From: systemtap-owner@sourceware.org [mailto:systemtap-
> owner@sourceware.org] On Behalf Of Lentes, Bernd
> Sent: Thursday, May 28, 2015 2:07 PM
> To: 'systemtap@sourceware.org'
> Subject: no access to local variables in function
>
> Hi,
>
> i'm new to systemtap. I try to check some local variables of kernel
> functions. I have a SLES 10 SP4, kernel 2.6.16.60-0.103.1-smp. I'm using
> systemtap 0.8.
> I'm following the instructions from
> https://sourceware.org/systemtap/SystemTap_Beginners_Guide/target
> variables.html .
>
> If I try to use the example from this page I get:
> idcc-devel:~/systemtap # stap -L 'kernel.function("vfs_read")'
> kernel.function("vfs_read@fs/read_write.c:248")
>
> Checking other functions with local variables also does not work:
> idcc-devel:~/systemtap # stap -L 'kernel.function("local_bh_enable")'
> kernel.function("local_bh_enable@kernel/softirq.c:139")
>
> idcc-devel:~/systemtap # stap -L 'kernel.function("__tasklet_schedule")'
> kernel.function("__tasklet_schedule@kernel/softirq.c:224")
>
> idcc-devel:~/systemtap # stap -L
> 'kernel.function("tasklet_kill_immediate")'
> kernel.function("tasklet_kill_immediate@kernel/softirq.c:439")
>
> I don't see any local variable. What can I do ?
>
> Btw: is there a chance to monitor the variables of a blacklisted function ?

Maybe it's an idea to use $$vars, $$parms ...
But I don't know how to interpret this:

$$vars: t=0xffff81003ead2d68    $$parms: t=0xffff81003ead2d68   $$locals:

My problem is that I have constant between 5 and 15% si load on that machine (Soft Interrupt). But I don't know the reason.
Maybe probe::softirq.entry is helpful but I don't know how to use it.

Thanks for any help.


Bernd

Helmholtz Zentrum München
Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir´in Bärbel Brumme-Bothe
Geschäftsführer: Prof. Dr. Günther Wess, Dr. Nikolaus Blum, Dr. Alfons Enhsen
Registergericht: Amtsgericht München HRB 6466
USt-IdNr: DE 129521671

Re: no access to local variables in function

[David Smith]

On 05/28/2015 07:06 AM, Lentes, Bernd wrote:
> Hi,
> 
> i'm new to systemtap. I try to check some local variables of kernel functions. I have a SLES 10 SP4, kernel 2.6.16.60-0.103.1-smp. I'm using systemtap 0.8.
> I'm following the instructions from https://sourceware.org/systemtap/SystemTap_Beginners_Guide/targetvariables.html .
> 
> If I try to use the example from this page I get:
> idcc-devel:~/systemtap # stap -L 'kernel.function("vfs_read")'
> kernel.function("vfs_read@fs/read_write.c:248")
> 
> Checking other functions with local variables also does not work:
> idcc-devel:~/systemtap # stap -L 'kernel.function("local_bh_enable")'
> kernel.function("local_bh_enable@kernel/softirq.c:139")
> 
> idcc-devel:~/systemtap # stap -L 'kernel.function("__tasklet_schedule")'
> kernel.function("__tasklet_schedule@kernel/softirq.c:224")
> 
> idcc-devel:~/systemtap # stap -L 'kernel.function("tasklet_kill_immediate")'
> kernel.function("tasklet_kill_immediate@kernel/softirq.c:439")
> 
> I don't see any local variable. What can I do ?

On my fedora 21 system, I see this:

# stap -L 'kernel.function("vfs_read")'
kernel.function("vfs_read@fs/read_write.c:478") $file:struct file*
$buf:char* $count:size_t $pos:loff_t*

The first thing I'd do is make sure you've got kernel debuginfo
installed that *exactly* matches that kernel.

Wow, you are using an old kernel (2.6.16) and a *really* old version of
systemtap (0.8). Systemtap 0.8 was released in 2008-11-13. We've had 23
releases since then. Is there any chance you can upgrade to a newer version?

> Btw: is there a chance to monitor the variables of a blacklisted function ?

If you use guru mode ('stap -g'), you can probe blacklisted functions.
However, functions are usually blacklisted for a good reason...

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

RE: no access to local variables in function

[Lentes, Bernd]

David wrote:
>
> On 05/28/2015 07:06 AM, Lentes, Bernd wrote:
> > Hi,
> >
> > i'm new to systemtap. I try to check some local variables of kernel
> functions. I have a SLES 10 SP4, kernel 2.6.16.60-0.103.1-smp. I'm using
> systemtap 0.8.
> > I'm following the instructions from
> https://sourceware.org/systemtap/SystemTap_Beginners_Guide/target
> variables.html .
> >
> > If I try to use the example from this page I get:
> > idcc-devel:~/systemtap # stap -L 'kernel.function("vfs_read")'
> > kernel.function("vfs_read@fs/read_write.c:248")
> >
> > Checking other functions with local variables also does not work:
> > idcc-devel:~/systemtap # stap -L 'kernel.function("local_bh_enable")'
> > kernel.function("local_bh_enable@kernel/softirq.c:139")
> >
> > idcc-devel:~/systemtap # stap -L
> 'kernel.function("__tasklet_schedule")'
> > kernel.function("__tasklet_schedule@kernel/softirq.c:224")
> >
> > idcc-devel:~/systemtap # stap -L
> 'kernel.function("tasklet_kill_immediate")'
> > kernel.function("tasklet_kill_immediate@kernel/softirq.c:439")
> >
> > I don't see any local variable. What can I do ?
>
> On my fedora 21 system, I see this:
>
> # stap -L 'kernel.function("vfs_read")'
> kernel.function("vfs_read@fs/read_write.c:478") $file:struct file*
> $buf:char* $count:size_t $pos:loff_t*
>
> The first thing I'd do is make sure you've got kernel debuginfo installed
> that *exactly* matches that kernel.
>

idcc-devel:~/systemtap # rpm -qa|grep -i kernel
kernel-syms-2.6.16.60-0.103.1
kernel-smp-2.6.16.60-0.103.1
kernel-debug-2.6.16.60-0.103.1
kernel-smp-debuginfo-2.6.16.60-0.103.1
kernel-source-2.6.16.60-0.103.1

> Wow, you are using an old kernel (2.6.16) and a *really* old version of
> systemtap (0.8). Systemtap 0.8 was released in 2008-11-13. We've had 23
> releases since then. Is there any chance you can upgrade to a newer
> version?
>

I managed to install 2.7 without any error.
But running my scripts now create errors I did note have before:
E.g.:

Script:
probe kernel.function("irq_exit") {printf("$$vars: %s\t$$parms: %s\t$$locals: %s \n",$$vars,$$parms,$$locals)}
probe kernel.function("tasklet_kill_immediate") {printf("$$vars: %s\t$$parms: %s\t$$locals: %s \n",$$vars,$$parms,$$locals)}
probe kernel.function("dump_clear_tasklet") {printf("$$vars: %s\t$$parms: %s\t$$locals: %s \n",$$vars,$$parms,$$locals)}

errors:
/usr/local/share/systemtap/runtime/transport/transport.c: In function ‘_stp_handle_start’:
/usr/local/share/systemtap/runtime/transport/transport.c:175: error: implicit declaration of function ‘atomic_notifier_chain_register’
/usr/local/share/systemtap/runtime/transport/transport.c: In function ‘_stp_cleanup_and_exit’:
/usr/local/share/systemtap/runtime/transport/transport.c:235: error: implicit declaration of function ‘atomic_notifier_chain_unregister’
make[3]: *** [/tmp/stapnWl1Uo/stap_206249b60ef8ac28da63674b8ffc8443_3123_src.o] Error 1
make[2]: *** [_module_/tmp/stapnWl1Uo] Error 2
make[1]: *** [modules] Error 2
make: *** [modules] Error 2
Spawn waitpid result (0x200): 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_206249b60ef8ac28da63674b8ffc8443_3123.ko" in 1470usr/520sys/2978real ms.
Pass 4: compilation failed.  [man error::pass4]
Running rm -rf /tmp/stapnWl1Uo
Spawn waitpid result (0x0): 0
Removed temporary directory "/tmp/stapnWl1Uo"


script:

// Softirq based probes.
/**
 * probe softirq.entry - Execution of handler for a pending softirq starting
 * @h: struct softirq_action* for current pending softirq
 * @vec: softirq_action vector
 * @action: pointer to softirq handler just about to execute
 * @vec_nr:  softirq vector number
 */
probe softirq.entry = kernel.trace("irq_softirq_entry") !,
                      kernel.trace("softirq_entry") ?
{
        # kernels < 2.6.37
        h = (@defined($h) ? $h : 0)
        vec = (@defined($vec) ? $vec : 0)
        action = (@defined($h) ? @cast($h,"softirq_action","kernel<linux/interrupt.h>")->action : 0)
        # kernels >= 2.6.37
        # vec_nr = (@defined($vec_nr) ? $vec_nr : 0)
}

I found that script on the internet and did copy/paste.

errors:

Pass 1: parsed user script and 106 library script(s) using 45984virt/30588res/2068shr/29320data kb, in 200usr/200sys/2059real ms.
semantic error: no probes found
   thrown from: elaborate.cxx:2004
Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) using 46512virt/31432res/2268shr/29848data kb, in 10usr/10sys/116real ms.
Pass 2: analysis failed.  [man error::pass2]
Running rm -rf /tmp/stapdzKK7Y
Spawn waitpid result (0x0): 0
Removed temporary directory "/tmp/stapdzKK7Y"

Maybe version 2.7 is now to recent ? What version would you recommend for my system ?



> > Btw: is there a chance to monitor the variables of a blacklisted function
> ?
>
> If you use guru mode ('stap -g'), you can probe blacklisted functions.
> However, functions are usually blacklisted for a good reason...
>

I understand.

> --

Helmholtz Zentrum München
Deutsches Forschungszentrum für Gesundheit und Umwelt (GmbH)
Ingolstädter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir´in Bärbel Brumme-Bothe
Geschäftsführer: Prof. Dr. Günther Wess, Dr. Nikolaus Blum, Dr. Alfons Enhsen
Registergericht: Amtsgericht München HRB 6466
USt-IdNr: DE 129521671

Re: no access to local variables in function

[David Smith]

On 05/29/2015 07:26 AM, Lentes, Bernd wrote:
> Maybe version 2.7 is now to recent ? What version would you recommend
> for my system ?

Looking at the release announcements, release 2.4 is the last one that
was tested on a kernel less than 2.6.18. So, I'd go with 2.4 or earlier.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: no access to local variables in function

[Frank Ch. Eigler]

"Lentes, Bernd" <bernd.lentes@helmholtz-muenchen.de> writes:

> [...]
> idcc-devel:~/systemtap # stap -L 'kernel.function("tasklet_kill_immediate")'
> kernel.function("tasklet_kill_immediate@kernel/softirq.c:439")
>
> I don't see any local variable. What can I do ?

Local variables exist within functions, not at their entry (which is what
.function probes are for).  Use .statement probes for function-interior
probing (and thus access to variables there).

stap -L 'kernel.statement("tasklet_kill_immediate@kernel/softirq.c:*")'


- FChE