Script run makes the system to hang

Script run makes the system to hang

[beginner966]

Hi all,

I had written this script to gather the time spent by a task in the
runqueue. It runs fine for 1-2 ms or so(short time I mean). But whenever I
try to run it without an exit() in timer function, my system hangs.
Please suggest some help regarding this matter as to how to make this script
run for infinite amount of time and provide useful statistics.


#!/usr/bin/env stap  


/*
 * Global variable declaration
 */ 


global runqueuetime
global runnabletime
global cputime
global oncputime
global tstamp
global names
global start

probe timer.ms(11)
{
	exit();
}

probe kernel.function("finish_task_switch")
{
	if($prev->state==0 && $prev->tgid)
		runnabletime[$prev->tgid,$prev->pid]=gettimeofday_us();
}

probe kernel.mark("_ctx_switch_done")
{
	if($arg6)
	{
		if($arg3)
			names[$arg3]=$arg1;
		names[$arg6]=$arg4;
		if(!runnabletime[$arg6,$arg5])
			runnabletime[$arg6,$arg5]=start;
		ttime=gettimeofday_us();
		runqueuetime[$arg6,$arg5] += ttime - runnabletime[$arg6,$arg5];
		oncputime[$arg6,$arg5]=ttime;
	}
}

/*
 * Catch the transition from (task_struct *)p->state to TASK_RUNNING state;
Process added in runqueue or really running
 */


probe kernel.function("activate_task")
{
	if($p->tgid)
		runnabletime[$p->tgid,$p->pid]=gettimeofday_us();
}


/*
 * Catch the transition from TASK->RUNNING to (task_struct *)p->state;
Process removed from CPU, in runqueue or some other state
 */


probe kernel.function("deactivate_task") 
{
	if($p->tgid)
	{
		if(!oncputime[$p->tgid,$p->pid])
			oncputime[$p->tgid,$p->pid]=start;
		cputime[$p->tgid,$p->pid] += gettimeofday_us() -
oncputime[$p->tgid,$p->pid];
	}
}


/*
 * Process has died and perfom the cleanup; Transition: (task_struct
*)p->state to Cleaned
 */


probe kernel.function("release_task").return 
{
	printf("%-15s %-6d %-6d %-15d
%-15d\n",names[$p->tgid],$p->tgid,$p->pid,cputime[$p->tgid,$p->pid],runqueuetime[$p->tgid,$p->pid]);
	delete runqueuetime[$p->tgid,$p->pid];
	delete cputime[$p->tgid,$p->pid];
}


probe timer.ms(5)
{
	//start=gettimeofday_us();
	tstamp=5;
	printf("Cumulative statistics for processes on all CPUs for last %d msec :
\n",tstamp);
        printf("\n\n\t%-15s   %-6s   %-6s   %-15s  
%-15s\n","Process","Pid","Tid","Cpu_Time(us)","Runqueue_Time(us)");
        printf("\t%-15s   %-6s   %-6s   %-15s  
%-15s\n","-------","---","---","------------","-----------------");
	tstamp=0;
        foreach ([pid+,tid] in runqueuetime)
        {
                if(!pid)
                {
                        //continue;
                        printf("\t%-15s   %-6d   %-6d   %-15d  
%-15d\n","Swapper",pid,tid,cputime[pid,tid],runqueuetime[pid,tid]);
                }
                else
                {
                	printf("\t%-15s   %-6d   %-6d   %-15d  
%-15d\n",names[pid],pid,tid,cputime[pid,tid],runqueuetime[pid,tid]);
                }
		/*tstamp+=1;
		if(tstamp==10)
			break;*/
        }
	//delete cputime;
	//delete runqueuetime;
	//delete names;
	printf("\n\n");
}



/*
 * Take a process id from user and print its corresponding detail
 */


probe begin 
{
	start=gettimeofday_us();
	print("Collecting data for all processes ... Press Ctrl-C to display
results and exit\n\n\n");
}

/*
 * Perform clean up
 */

	
probe end
{
	printf("\n\n");
	delete cputime;
	delete runqueuetime;
}
-- 
View this message in context: http://www.nabble.com/Script-run-makes-the-system-to-hang-tp22271687p22271687.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.

Re: Script run makes the system to hang

[Frank Ch. Eigler]

beginner966 <nitin966@gmail.com> writes:

> I had written this script to gather the time spent by a task in the
> runqueue. It runs fine for 1-2 ms or so(short time I mean). But
> whenever I try to run it without an exit() in timer function, my
> system hangs. [...]

Can you collect the information suggested by the HowToReportBugs systemtap
wiki page and/or by running the "stap-report" script?


> Please suggest some help regarding this matter as to how to make this script
> run for infinite amount of time and provide useful statistics.
> probe timer.ms(11)
> {
> 	exit();
> }

(Please note that 11 milliseconds is a very short time indeed.)


> probe kernel.function("finish_task_switch") [...]

I wonder if utrace-based or scheduler.stp tapset-based probe points
may serve as well.


- FChE

Re: Script run makes the system to hang

[beginner966]

I am attaching the result of stap-report script.

http://www.nabble.com/file/p22308493/output output 
-- 
View this message in context: http://www.nabble.com/Script-run-makes-the-system-to-hang-tp22271687p22308493.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.

Re: Script run makes the system to hang

[beginner966]

I gathered the results of each pass.
They are as follows:

root@nitin-desktop:/media/disk/kernel/Desktop/SystemTapProject/scripts/projectscripts#
stap -vvg /home/nitin/Desktop/demo/runqueue_time.stp > nitin
SystemTap translator/driver (version 0.9/0.131 non-git sources)
Copyright (C) 2005-2009 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
Session arch: i686 release: 2.6.29-rc6-rt3
Created temporary directory "/tmp/stapnsNOYB"
Searched '/usr/local/share/systemtap/tapset/i686/*.stp', found 2
Searched '/usr/local/share/systemtap/tapset/*.stp', found 45
Pass 1: parsed user script and 47 library script(s) in 360usr/20sys/523real
ms.
probe finish_task_switch@kernel/sched.c:2850 kernel reloc=.dynamic
section=.text pc=0xc011ded2
probe activate_task@kernel/sched.c:1938 kernel reloc=.dynamic section=.text
pc=0xc011a9f7
probe deactivate_task@kernel/sched.c:1950 kernel reloc=.dynamic
section=.text pc=0xc011aa12
probe release_task@kernel/exit.c:176 kernel reloc=.dynamic section=.text
pc=0xc01257ec
probe release_task@kernel/exit.c:176 kernel reloc=.dynamic section=.text
pc=0xc01257ec
Pass 2: analyzed script: 10 probe(s), 14 function(s), 2 embed(s), 11
global(s) in 340usr/130sys/3487real ms.
Pass 3: using cached
/root/.systemtap/cache/29/stapconf_297666a7ea51531673cf30696f0e715a_117.h
probe_1389 locks nothing
probe_1390 locks runnabletime[w]
probe_1391 locks names[w] oncputime[w] runnabletime[rw] runqueuetime[rw]
probe_1392 locks runnabletime[w]
probe_1393 locks oncputime[rw] cputime[rw]
probe_1394 locks _dwarf_tvar_p_33_ctr[rw] _dwarf_tvar_p_33[rw]
_dwarf_tvar_p_32_ctr[rw] _dwarf_tvar_p_32[rw] names[r] cputime[rw]
runqueuetime[rw]
probe_1396 locks names[r] tstamp[rw] cputime[r] runqueuetime[rw]
probe_1399 locks _dwarf_tvar_p_33_ctr[rw] _dwarf_tvar_p_33[w]
_dwarf_tvar_p_32_ctr[rw] _dwarf_tvar_p_32[w]
dump_unwindsyms kernel index=0 base=0xc0100000
Found build-id in kernel, length 20, end at 0xc0451a1c
Pass 3: translated to C into
"/tmp/stapnsNOYB/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.c" in
910usr/50sys/1054real ms.
Running make -C "/lib/modules/2.6.29-rc6-rt3/build" M="/tmp/stapnsNOYB"
modules >/dev/null
Pass 4: compiled C into "stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.ko" in
4180usr/600sys/11940real ms.
Copying /tmp/stapnsNOYB/stapconf_297666a7ea51531673cf30696f0e715a_117.h to
/root/.systemtap/cache/29/stapconf_297666a7ea51531673cf30696f0e715a_117.h
Copying /tmp/stapnsNOYB/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.ko to
/root/.systemtap/cache/e3/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.ko
Copying /tmp/stapnsNOYB/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.c to
/root/.systemtap/cache/e3/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.c
Cache cleaning successful, removed entries:
/root/.systemtap/cache/68/stap_680091c4cfee0edc4e5b1689fe769a86_7352
Pass 5: starting run.
Running /usr/local/bin/staprun -v
/tmp/stapnsNOYB/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.ko
WARNING: Number of errors: 0, skipped probes: 3
stapio:cleanup_and_exit:336 detach=0
stapio:cleanup_and_exit:353 closing control channel
Pass 5: run completed in 10usr/20sys/374real ms.
Running rm -rf /tmp/stapnsNOYB
-- 
View this message in context: http://www.nabble.com/Script-run-makes-the-system-to-hang-tp22271687p22385551.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.

Re: Script run makes the system to hang

[Frank Ch. Eigler]

beginner966 <nitin966@gmail.com> writes:

> I gathered the results of each pass.
> [...]
> Pass 5: starting run.
> Running /usr/local/bin/staprun -v
> /tmp/stapnsNOYB/stap_e3664fa6333b76ce7d5e80e8de5bcdfb_8077.ko
> WARNING: Number of errors: 0, skipped probes: 3
> stapio:cleanup_and_exit:336 detach=0
> stapio:cleanup_and_exit:353 closing control channel
> Pass 5: run completed in 10usr/20sys/374real ms.
> [...]

This means that the script ran successfully, though without producing
output that you may have expected.  To explain the "skipped" probes,
rerun it with "stap -t ...".

- FChE

Re: Script run makes the system to hang

[beginner966]

Hi,

>This means that the script ran successfully, though without producing
>output that you may have expected.  To explain the "skipped" probes,
>rerun it with "stap -t ...".

Script ran successfully because I ran it only for 11 ms.
When I comment the timer.ms(11) probe, it hangs as usual.
It produced the desired output, however, it was redirected to a file while
running the command. I'v not been able to find a solution to this problem
till now.

Thanks.

-- 
View this message in context: http://www.nabble.com/Script-run-makes-the-system-to-hang-tp22271687p22389458.html
Sent from the Sourceware - systemtap mailing list archive at Nabble.com.