Fwd: please help

Fwd: please help

[siddharam suresh]

i want visualize what all the stuffs happening inside the Linux kernel
when this program is executed


#include<stdio.h>
void main()
{
printf("Test kernel");  <-- here i want know what are all the system
calls and the kernel data structure kernel is used by the kernel. Is
it possible to put break points here and can we see what are the
things happening inside the kernel when printf is executed

}


is it possible to put break points in side the my program using the
system tap i want to visualize what are system calls and kernel data
structure used by particular statement.

as above stated test.c program i what see what is happening inside the
kernel when printf("Test Kernel"); is executed .This because when the
my program become large it will show so many information which may not
be use full to me and very difficult to get the particular
information.Is there any facilities like step in and step out in
system tap



thank you in advance
waiting great response

Re: please help

[Peter Teoh]

first to use systemtap, u need to roughly know which kernel APIs u
want to trace.   to do that, u can use "strace" to trace the userspace
execution of the program, and from the system call output, (usually
:-)) add the sys_xxx to derive the kernel API version, eg, for
execve() it is sys_execve():

The following is your program:

/root>strace ./a.out
execve("./a.out", ["./a.out"], [/* 28 vars */]) = 0
brk(0)                                  = 0x804a000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7ef5000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=54870, ...}) = 0
mmap2(NULL, 54870, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ee7000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0`\1\000"...,
512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=1307104, ...}) = 0
mmap2(NULL, 1312164, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
3, 0) = 0xb7da6000
mmap2(0xb7ee1000, 12288, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13b) = 0xb7ee1000
mmap2(0xb7ee4000, 9636, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ee4000
close(3)                                = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7da5000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7da56c0,
limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb7ee1000, 4096, PROT_READ)   = 0
munmap(0xb7ee7000, 54870)               = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7ef4000
write(1, "Test kernel", 11Test kernel)             = 11
exit_group(11)                          = ?

And sys_execve() in arch/x86/kernel/process_32.c is your start (mine
kernel is 2.6.20).   Well, it will be a long long path towards full
understanding even of this simple program.   Alternatively, pick a
topic of interest, and understand in greater depth about it.   Eg,
tracing the filesystem VFS APIs etc.

On Sat, Oct 11, 2008 at 10:12 PM, siddharam suresh
<siddharam.s.t@gmail.com> wrote:
> i want visualize what all the stuffs happening inside the Linux kernel
> when this program is executed
>
>
> #include<stdio.h>
> void main()
> {
> printf("Test kernel");  <-- here i want know what are all the system
> calls and the kernel data structure kernel is used by the kernel. Is
> it possible to put break points here and can we see what are the
> things happening inside the kernel when printf is executed
>
> }
>
>
> is it possible to put break points in side the my program using the
> system tap i want to visualize what are system calls and kernel data
> structure used by particular statement.
>
> as above stated test.c program i what see what is happening inside the
> kernel when printf("Test Kernel"); is executed .This because when the
> my program become large it will show so many information which may not
> be use full to me and very difficult to get the particular
> information.Is there any facilities like step in and step out in
> system tap
>
>
>
> thank you in advance
> waiting great response
>



-- 
Regards,
Peter Teoh

Re: please help

[Peter Teoh]

Alternatively, as Frank earlier said in another email:

 probe process("a.out").function("*") { log ($$parms) }

On Tue, Oct 14, 2008 at 11:14 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
> first to use systemtap, u need to roughly know which kernel APIs u
> want to trace.   to do that, u can use "strace" to trace the userspace
> execution of the program, and from the system call output, (usually
> :-)) add the sys_xxx to derive the kernel API version, eg, for
> execve() it is sys_execve():
>
> The following is your program:
>
> /root>strace ./a.out
> execve("./a.out", ["./a.out"], [/* 28 vars */]) = 0
> brk(0)                                  = 0x804a000
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7ef5000
> access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY)      = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=54870, ...}) = 0
> mmap2(NULL, 54870, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ee7000
> close(3)                                = 0
> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
> open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0`\1\000"...,
> 512) = 512
> fstat64(3, {st_mode=S_IFREG|0644, st_size=1307104, ...}) = 0
> mmap2(NULL, 1312164, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
> 3, 0) = 0xb7da6000
> mmap2(0xb7ee1000, 12288, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13b) = 0xb7ee1000
> mmap2(0xb7ee4000, 9636, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ee4000
> close(3)                                = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7da5000
> set_thread_area({entry_number:-1 -> 6, base_addr:0xb7da56c0,
> limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
> limit_in_pages:1, seg_not_present:0, useable:1}) = 0
> mprotect(0xb7ee1000, 4096, PROT_READ)   = 0
> munmap(0xb7ee7000, 54870)               = 0
> fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = 0xb7ef4000
> write(1, "Test kernel", 11Test kernel)             = 11
> exit_group(11)                          = ?
>
> And sys_execve() in arch/x86/kernel/process_32.c is your start (mine
> kernel is 2.6.20).   Well, it will be a long long path towards full
> understanding even of this simple program.   Alternatively, pick a
> topic of interest, and understand in greater depth about it.   Eg,
> tracing the filesystem VFS APIs etc.
>
> On Sat, Oct 11, 2008 at 10:12 PM, siddharam suresh
> <siddharam.s.t@gmail.com> wrote:
>> i want visualize what all the stuffs happening inside the Linux kernel
>> when this program is executed
>>
>>
>> #include<stdio.h>
>> void main()
>> {
>> printf("Test kernel");  <-- here i want know what are all the system
>> calls and the kernel data structure kernel is used by the kernel. Is
>> it possible to put break points here and can we see what are the
>> things happening inside the kernel when printf is executed
>>
>> }
>>
>>
>> is it possible to put break points in side the my program using the
>> system tap i want to visualize what are system calls and kernel data
>> structure used by particular statement.
>>
>> as above stated test.c program i what see what is happening inside the
>> kernel when printf("Test Kernel"); is executed .This because when the
>> my program become large it will show so many information which may not
>> be use full to me and very difficult to get the particular
>> information.Is there any facilities like step in and step out in
>> system tap
>>
>>
>>
>> thank you in advance
>> waiting great response
>>
>
>
>
> --
> Regards,
> Peter Teoh
>



-- 
Regards,
Peter Teoh

Re: please help

[David Smith]

Peter Teoh wrote:
> Alternatively, as Frank earlier said in another email:
> 
>  probe process("a.out").function("*") { log ($$parms) }

Another alternative to think about is using LTTNG
(<http://ltt.polymtl.ca/>).  LTTNG ("Linux Trace Toolkit Next
Generation") is more of a tracing tool than systemtap is and might be
better suited to what you are looking for.

LTTNG and systemtap use some of the same kernel features under the hood
to do their work.

> On Tue, Oct 14, 2008 at 11:14 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
>> first to use systemtap, u need to roughly know which kernel APIs u
>> want to trace.   to do that, u can use "strace" to trace the userspace
>> execution of the program, and from the system call output, (usually
>> :-)) add the sys_xxx to derive the kernel API version, eg, for
>> execve() it is sys_execve():
>>
>> The following is your program:
>>
>> /root>strace ./a.out
>> execve("./a.out", ["./a.out"], [/* 28 vars */]) = 0
>> brk(0)                                  = 0x804a000
>> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
>> mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0xb7ef5000
>> access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
>> open("/etc/ld.so.cache", O_RDONLY)      = 3
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=54870, ...}) = 0
>> mmap2(NULL, 54870, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7ee7000
>> close(3)                                = 0
>> access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
>> open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
>> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0`\1\000"...,
>> 512) = 512
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=1307104, ...}) = 0
>> mmap2(NULL, 1312164, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE,
>> 3, 0) = 0xb7da6000
>> mmap2(0xb7ee1000, 12288, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13b) = 0xb7ee1000
>> mmap2(0xb7ee4000, 9636, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7ee4000
>> close(3)                                = 0
>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0xb7da5000
>> set_thread_area({entry_number:-1 -> 6, base_addr:0xb7da56c0,
>> limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
>> limit_in_pages:1, seg_not_present:0, useable:1}) = 0
>> mprotect(0xb7ee1000, 4096, PROT_READ)   = 0
>> munmap(0xb7ee7000, 54870)               = 0
>> fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
>> 0) = 0xb7ef4000
>> write(1, "Test kernel", 11Test kernel)             = 11
>> exit_group(11)                          = ?
>>
>> And sys_execve() in arch/x86/kernel/process_32.c is your start (mine
>> kernel is 2.6.20).   Well, it will be a long long path towards full
>> understanding even of this simple program.   Alternatively, pick a
>> topic of interest, and understand in greater depth about it.   Eg,
>> tracing the filesystem VFS APIs etc.
>>
>> On Sat, Oct 11, 2008 at 10:12 PM, siddharam suresh
>> <siddharam.s.t@gmail.com> wrote:
>>> i want visualize what all the stuffs happening inside the Linux kernel
>>> when this program is executed
>>>
>>>
>>> #include<stdio.h>
>>> void main()
>>> {
>>> printf("Test kernel");  <-- here i want know what are all the system
>>> calls and the kernel data structure kernel is used by the kernel. Is
>>> it possible to put break points here and can we see what are the
>>> things happening inside the kernel when printf is executed
>>>
>>> }
>>>
>>>
>>> is it possible to put break points in side the my program using the
>>> system tap i want to visualize what are system calls and kernel data
>>> structure used by particular statement.
>>>
>>> as above stated test.c program i what see what is happening inside the
>>> kernel when printf("Test Kernel"); is executed .This because when the
>>> my program become large it will show so many information which may not
>>> be use full to me and very difficult to get the particular
>>> information.Is there any facilities like step in and step out in
>>> system tap
>>>
>>>
>>>
>>> thank you in advance
>>> waiting great response
>>>
>>
>>
>> --
>> Regards,
>> Peter Teoh
>>
> 
> 
> 


-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Fwd: please help

[Frank Ch. Eigler]

"siddharam suresh" <siddharam.s.t@gmail.com> writes:

> i want visualize what all the stuffs happening inside the Linux kernel
> when this program is executed [...]

> is it possible to put break points in side the my program using the
> system tap

This part is fine.

> i want to visualize what are system calls and kernel data
> structure used by particular statement.

This part is too vague.  You can express some call graph tracing
reasonably easily in systemtap, but there are still problems with
kprobes placed in senstive spots in the kernel.  This script:

stap para-callgraph.stp \
     'kernel.function("*@fs/*")' \             <-- broad but not too broad
     'process("a.out").function("printf")' \
     -c a.out

gives me lots of data, far more though than you probably really want.
As to what data structures are used -- that's a whole different animal.


- FChE

Re: please help

[Breno Leitao]

Peter, 

Peter Teoh wrote:
> Alternatively, as Frank earlier said in another email:
> 
>  probe process("a.out").function("*") { log ($$parms) }

On PPC, I am getting the following error when running this code: 

[root@otitis systemtap]# stap userspace.stp  -v
Pass 1: parsed user script and 45 library script(s) in 320usr/0sys/326real ms.
semantic error: ELF machine ppc (code 20) mismatch with target ppc64 in '/root/systemtap/a.out'
semantic error: no match while resolving probe point process("a.out").function("*")
semantic error: no probes found
Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) in 0usr/0sys/7real ms.
Pass 2: analysis failed.  Try again with more '-v' (verbose) options.

Isn't the process tracing supported on the PPC platform ?

Re: please help

[David Smith]

Breno Leitao wrote:
> Peter, 
> 
> Peter Teoh wrote:
>> Alternatively, as Frank earlier said in another email:
>>
>>  probe process("a.out").function("*") { log ($$parms) }
> 
> On PPC, I am getting the following error when running this code: 
> 
> [root@otitis systemtap]# stap userspace.stp  -v
> Pass 1: parsed user script and 45 library script(s) in 320usr/0sys/326real ms.
> semantic error: ELF machine ppc (code 20) mismatch with target ppc64 in '/root/systemtap/a.out'
> semantic error: no match while resolving probe point process("a.out").function("*")
> semantic error: no probes found
> Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s) in 0usr/0sys/7real ms.
> Pass 2: analysis failed.  Try again with more '-v' (verbose) options.
> 
> Isn't the process tracing supported on the PPC platform ?

It appears that systemtap is complaining that you are trying to put
probes in a ppc (32-bit) executable while running a ppc64 (64-bit)
kernel.  Unfortunately, that is the normal case on a ppc64 box (if I
remember correctly).

Can you try compiling the a.out as a ppc64 executable and see if that works?

If I'm reading tapsets.cxx correctly, the same problem will happen when
trying to probe an x86 program on a system with a x86_64 kernel.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: please help

[Breno Leitao]

[-- Attachment #1: Type: text/plain, Size: 581 bytes --]

David Smith wrote:
 > It appears that systemtap is complaining that you are trying to put
> probes in a ppc (32-bit) executable while running a ppc64 (64-bit)
> kernel.  Unfortunately, that is the normal case on a ppc64 box (if I
> remember correctly).
> 
> Can you try compiling the a.out as a ppc64 executable and see if that works?
That is right David, the problem disappeared, but I got the following other one: 

[root@otitis systemtap]# stap userspace.stp  
Pass 4: compilation failed.  Try again with more '-v' (verbose) options.

See the details in the attachment.

Thanks

[-- Attachment #2: details.txt --]
[-- Type: text/plain, Size: 14549 bytes --]

[root@otitis systemtap]# stap userspace.stp  -vv
SystemTap translator/driver (version 0.7.1/0.133 git branch master, commit 180b066c)
Copyright (C) 2005-2008 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
Session arch: ppc64 release: 2.6.27-rc9
Created temporary directory "/tmp/stap196kZZ"
Searched '/usr/local/share/systemtap/tapset/ppc64/*.stp', found 2
Searched '/usr/local/share/systemtap/tapset/*.stp', found 43
Pass 1: parsed user script and 45 library script(s) in 320usr/0sys/327real ms.
probe main@/root/systemtap/foo.c:6 process=/root/systemtap/a.out reloc=.absolute section=.text pc=0x10000578
probe foo@/root/systemtap/foo.c:1 process=/root/systemtap/a.out reloc=.absolute section=.text pc=0x10000538
Pass 2: analyzed script: 2 probe(s), 1 function(s), 0 embed(s), 0 global(s) in 0usr/10sys/8real ms.
probe_1427 locks nothing
probe_1428 elided, duplicates probe_1427
dump_unwindsyms /root/systemtap/a.out index=0 base=0x10000000
Pass 3: translated to C into "/tmp/stap196kZZ/stap_5b42bca8044eb63cef313d7b4d5021cc_1361.c" in 30usr/100sys/132real ms.
Pass 4, preamble: (re)building SystemTap's version of uprobes.
Running make -C /usr/local/share/systemtap/runtime/uprobes >/dev/null
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:29:26: error: linux/utrace.h: No such file or directory
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:36:27: error: asm/tracehook.h: No such file or directory
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘utask_adjust_flags’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:352: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:360: error: implicit declaration of function ‘utrace_set_flags’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘clear_utrace_quiesce’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:366: error: ‘UTRACE_ACTION_QUIESCE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:366: error: (Each undeclared identifier is reported only once
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:366: error: for each function it appears in.)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:366: error: implicit declaration of function ‘UTRACE_EVENT’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:366: error: ‘QUIESCE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘quiesce_all_threads’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:430: error: ‘UTRACE_ACTION_QUIESCE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:431: error: ‘QUIESCE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_free_process’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:492: error: implicit declaration of function ‘utrace_detach’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_add_task’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:582: error: implicit declaration of function ‘utrace_attach’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:582: error: ‘UTRACE_ATTACH_CREATE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:583: warning: assignment makes pointer from integer without a cast
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:598: error: ‘SIGNAL’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:598: error: ‘SIGNAL_IGN’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:599: error: ‘SIGNAL_CORE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:599: error: ‘EXEC’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:600: error: ‘CLONE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:600: error: ‘EXIT’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_get_task’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:824: error: implicit declaration of function ‘find_task_by_pid’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:824: warning: assignment makes pointer from integer without a cast
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_delay_signal’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1626: error: ‘UTRACE_ACTION_HIDE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1626: error: ‘UTRACE_SIGNAL_IGN’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1627: error: ‘UTRACE_ACTION_SINGLESTEP’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1627: error: ‘UTRACE_ACTION_NEWSTATE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_report_signal’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: warning: type defaults to ‘int’ in declaration of ‘_________p1’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: warning: type defaults to ‘int’ in declaration of ‘type name’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1683: warning: cast to pointer from integer of different size
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1694: error: ‘UTRACE_ACTION_RESUME’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1767: error: ‘UTRACE_ACTION_HIDE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1767: error: ‘UTRACE_SIGNAL_IGN’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1768: error: ‘UTRACE_ACTION_SINGLESTEP’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1768: error: ‘UTRACE_ACTION_NEWSTATE’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1834: error: ‘UTRACE_ACTION_DETACH’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘utask_quiesce_pending_sigtrap’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1860: error: implicit declaration of function ‘utrace_native_view’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1860: warning: assignment makes pointer from integer without a cast
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1861: error: implicit declaration of function ‘utrace_regset’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1861: warning: assignment makes pointer from integer without a cast
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1865: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1866: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1867: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1870: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1873: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1873: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1873: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_report_quiesce’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: warning: type defaults to ‘int’ in declaration of ‘_________p1’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: warning: type defaults to ‘int’ in declaration of ‘type name’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1892: warning: cast to pointer from integer of different size
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1904: error: ‘UTRACE_ACTION_RESUME’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_report_exit’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: warning: type defaults to ‘int’ in declaration of ‘_________p1’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: warning: type defaults to ‘int’ in declaration of ‘type name’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:1993: warning: cast to pointer from integer of different size
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2051: error: ‘UTRACE_ACTION_DETACH’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_report_clone’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: warning: type defaults to ‘int’ in declaration of ‘_________p1’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: warning: type defaults to ‘int’ in declaration of ‘type name’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2172: warning: cast to pointer from integer of different size
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2235: error: ‘UTRACE_ACTION_RESUME’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: In function ‘uprobe_report_exec’:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: warning: type defaults to ‘int’ in declaration of ‘_________p1’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: error: dereferencing pointer to incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: warning: type defaults to ‘int’ in declaration of ‘type name’
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2262: warning: cast to pointer from integer of different size
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2278: error: ‘UTRACE_ACTION_DETACH’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2278: error: ‘UTRACE_ACTION_RESUME’ undeclared (first use in this function)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c: At top level:
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2281: error: variable ‘uprobe_utrace_ops’ has initializer but incomplete type
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2283: error: unknown field ‘report_quiesce’ specified in initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2283: warning: excess elements in struct initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2283: warning: (near initialization for ‘uprobe_utrace_ops’)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2284: error: unknown field ‘report_signal’ specified in initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2284: warning: excess elements in struct initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2284: warning: (near initialization for ‘uprobe_utrace_ops’)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2285: error: unknown field ‘report_exit’ specified in initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2285: warning: excess elements in struct initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2285: warning: (near initialization for ‘uprobe_utrace_ops’)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2286: error: unknown field ‘report_clone’ specified in initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2286: warning: excess elements in struct initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2286: warning: (near initialization for ‘uprobe_utrace_ops’)
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2287: error: unknown field ‘report_exec’ specified in initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2288: warning: excess elements in struct initializer
/usr/local/share/systemtap/runtime/uprobes/uprobes.c:2288: warning: (near initialization for ‘uprobe_utrace_ops’)
make[2]: *** [/usr/local/share/systemtap/runtime/uprobes/uprobes.o] Error 1
make[1]: *** [_module_/usr/local/share/systemtap/runtime/uprobes] Error 2
make: *** [default] Error 2
Uprobes (re)build failed.
Pass 4: compiled C into "stap_5b42bca8044eb63cef313d7b4d5021cc_1361.ko" in 410usr/310sys/714real ms.
Pass 4: compilation failed.  Try again with more '-v' (verbose) options.
Running rm -rf /tmp/stap196kZZ


[root@otitis systemtap]# file a.out 
a.out: ELF 64-bit MSB executable, cisco 7500, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped


[root@otitis systemtap]# cat userspace.stp 
probe process("a.out").function("*") { 
	printf("%s\n", probefunc())
}

Re: please help

[David Smith]

Breno Leitao wrote:
> David Smith wrote:
>  > It appears that systemtap is complaining that you are trying to put
>> probes in a ppc (32-bit) executable while running a ppc64 (64-bit)
>> kernel.  Unfortunately, that is the normal case on a ppc64 box (if I
>> remember correctly).
>>
>> Can you try compiling the a.out as a ppc64 executable and see if that works?
> That is right David, the problem disappeared, but I got the following other one: 
> 
> [root@otitis systemtap]# stap userspace.stp  
> Pass 4: compilation failed.  Try again with more '-v' (verbose) options.
> 
> See the details in the attachment.

From your attachment, your kernel is 2.6.27-rc9.  Unfortunately, uprobes
isn't ported to the new version of utrace.  Right now, uprobes only
works on kernels with the old version of utrace, such as the RHEL5 kernel.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: please help

[Frank Ch. Eigler]

David Smith <dsmith@redhat.com> writes:

> [...]
> It appears that systemtap is complaining that you are trying to put
> probes in a ppc (32-bit) executable while running a ppc64 (64-bit)
> kernel.  Unfortunately, that is the normal case on a ppc64 box (if I
> remember correctly).
>[...]

> If I'm reading tapsets.cxx correctly, the same problem will happen when
> trying to probe an x86 program on a system with a x86_64 kernel.

Right, this is a bug.  If uprobes can handle it, we should support
bi-architecture userspace probing, and relax the above check to assert
arch matching for the kernel side only.

- FChE

Re: please help

[Peter Teoh]

On Sat, Oct 11, 2008 at 10:12 PM, siddharam suresh
<siddharam.s.t@gmail.com> wrote:
> i want visualize what all the stuffs happening inside the Linux kernel
> when this program is executed
>
>
> #include<stdio.h>
> void main()
> {
> printf("Test kernel");  <-- here i want know what are all the system
> calls and the kernel data structure kernel is used by the kernel. Is
> it possible to put break points here and can we see what are the
> things happening inside the kernel when printf is executed
>
> }
>
>
> is it possible to put break points in side the my program using the
> system tap i want to visualize what are system calls and kernel data
> structure used by particular statement.
>

How about trying UML?

http://user-mode-linux.sourceforge.net/hacking.html

After you have "b start_kernel", if u enter "n" all the way, basically
u will see all the functions (or sub-functions) called.

For example, mine it gives:

(gdb) b start_kernel
Breakpoint 1 at 0x80493c8: file
/sda2/linux-source-2.6.20-2.6.20/init/main.c, line 490.
(gdb) cont
The program is not being run.
(gdb) run
Starting program: /sda2/linux-source-2.6.20-2.6.20/uml/vmlinux
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking for tmpfs mount on /dev/shm...OK
Checking PROT_EXEC mmap in /dev/shm/...OK
Checking for the skas3 patch in the host:
  - /proc/mm...not found
  - PTRACE_FAULTINFO...not found
  - PTRACE_LDT...not found
UML running in SKAS0 mode

Breakpoint 1, start_kernel () at
/sda2/linux-source-2.6.20-2.6.20/init/main.c:490
490             smp_setup_processor_id();
(gdb) n
499             local_irq_disable();
(gdb)
38              __asm__ __volatile__( LOCK_PREFIX
(gdb)
510             printk(KERN_NOTICE);
(gdb)
511             printk(linux_banner);
(gdb)
512             setup_arch(&command_line);
(gdb)
522             sched_init();
(gdb)
528             build_all_zonelists();
(gdb)
529             page_alloc_init();
(gdb)
530             printk(KERN_NOTICE "Kernel command line: %s\n",
saved_command_line);
(gdb)

531             parse_early_param();
(gdb)
532             parse_args("Booting kernel", command_line, __start___param,
(gdb)
535             if (!irqs_disabled()) {
(gdb)
540             sort_main_extable();
(gdb)
541             trap_init();

(gdb)
542             rcu_init();
(gdb)
543             init_IRQ();
(gdb)
544             pidhash_init();
(gdb)
545             init_timers();
(gdb)
546             hrtimers_init();
(gdb)
547             softirq_init();
(gdb)
548             timekeeping_init();
(gdb)
549             time_init();
(gdb)
550             profile_init();
(gdb)
551             if (!irqs_disabled())
(gdb)
554             local_irq_enable();
(gdb)
561             console_init();
(gdb)
562             if (panic_later)
(gdb)
582             vfs_caches_init_early();
(gdb)
584             mem_init();
(gdb)
start_idle_thread (stack=0x81dc000, switch_buf=0x81e147c) at
/sda2/linux-source-2.6.20-2.6.20/arch/um/os-Linux/skas/process.c:481
481             switch(n){
(gdb)
489                     (*cb_proc)(cb_arg);
(gdb)
490                     UML_LONGJMP(cb_back, 1);
(gdb)
initial_thread_cb_skas (proc=0x805739b <map_cb>, arg=0x0) at
/sda2/linux-source-2.6.20-2.6.20/arch/um/os-Linux/skas/process.c:515
515             unblock_signals();
(gdb)
520     }
(gdb)
517             cb_proc = NULL;
(gdb)
518             cb_arg = NULL;
(gdb)
519             cb_back = NULL;
(gdb)
520     }
(gdb)
initial_thread_cb (proc=0x805739b <map_cb>, arg=0x0) at
/sda2/linux-source-2.6.20-2.6.20/arch/um/kernel/process.c:188
188             kmalloc_ok = save_kmalloc_ok;
(gdb)
189     }
(gdb)
188             kmalloc_ok = save_kmalloc_ok;
(gdb)
189     }
(gdb)
mem_init () at /sda2/linux-source-2.6.20-2.6.20/arch/um/kernel/mem.c:77
77              free_bootmem(__pa(brk_end), uml_reserved - brk_end);


If u want u can zoom down to the specific functions and put a break
point there instead, and then print out any of the data u need, if the
symbols can be recognized in your vmlinux file (use the tab key in
gdb).

> as above stated test.c program i what see what is happening inside the
> kernel when printf("Test Kernel"); is executed .This because when the
> my program become large it will show so many information which may not
> be use full to me and very difficult to get the particular
> information.Is there any facilities like step in and step out in
> system tap
>
>
>
> thank you in advance
> waiting great response
>



-- 
Regards,
Peter Teoh