Using systemtap to look at futexs

Using systemtap to look at futexs

[William Cohen]

Monday I went to lunch with Gavin Romig-Koch. Gavin works in Red Hat 
support. While we were talking he mentioned trying to solve a customer 
problem that appeared to be caused by futex. An attempt was made to use 
GDB to track futex operations from userspace. However, that really 
slowed down the machine and never got the to reported problem. Gavin 
thought that SystemTap might be useful in diagnosing this problem. This 
morning I wrote a quick script to print out futex information while the 
system is running. I don't know whether this would identify the specific 
problem Gavin mentioned.

-Will

# futex.stp
#
# track which things are using futex

probe syscall.futex
{
	printf("pid=%d exec=%s sys_futex(%s)\n", pid(), execname(), argstr)
}

probe syscall.futex.return
{
	printf("pid=%d exec=%s sys_futex()=%s\n", pid(), execname(), retstr)
}

Re: Using systemtap to look at futexs

[Frank Ch. Eigler]

wcohen wrote:

> [...] probe syscall.futex
> {
> 	printf("pid=%d exec=%s sys_futex(%s)\n", pid(), execname(), argstr)
> }
> 
> probe syscall.futex.return
> {
> 	printf("pid=%d exec=%s sys_futex()=%s\n", pid(), execname(), retstr)
> } [...]

Given the sheer number of such scripts that are almost the same, how
about working out a generalization?  How about a tapset that includes
a common trace function (unless the LKET scripts already include
this).  Or a shell script to generate the systemtap scripts?  Or
another command-line-argument substitution mechanism?

- FChE

Re: Using systemtap to look at futexs

[Tim Bird]

Frank Ch. Eigler wrote:
> Hi -
> 
> On Tue, May 16, 2006 at 03:09:03PM -0700, Tim Bird wrote:
> 
>> [...]  Aside from improved ease-of-development, it could be
>> beneficial to have a standardized output format, such that the data
>> could be used with other trace presentation and analysis tools (like
>> lttv from LTT or kd from KFT).
> 
> One can write functions in the scripting language to format trace data
> in any reasonable format.  

Yes, but my point is that outputting a known, standardized format
(as opposed to "any reasonable format") could have side benefits.
If you're going to write helper functions for the output, a nice
side effect of the regularization of the handling would also be
a regularization of the formatting, with the possibility to make
the data more widely usable.

> (Plus one can write scripting language code
> to perform some of the analysis live that these trace tools do after
> the fact.)

Yes, some.  Is there a visualization tool for SystemTap data?
 -- Tim

P.S. Sorry, my original comment didn't go to the list because
I forgot to reply-all. I hope it's OK to put the thread back
on-list.

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
=============================

Re: Using systemtap to look at futexs

[Vara Prasad]

Tim Bird wrote:

>Frank Ch. Eigler wrote:
>  
>
> [...]
>
>
>Yes, some.  Is there a visualization tool for SystemTap data?
>  
>

We are developing a GUI interface to SystemTap. We have plans to emit 
xml data out that should make it easy for other visualization tools to 
present the data.

> -- Tim
>
>P.S. Sorry, my original comment didn't go to the list because
>I forgot to reply-all. I hope it's OK to put the thread back
>on-list.
>
>=============================
>Tim Bird
>Architecture Group Chair, CE Linux Forum
>Senior Staff Engineer, Sony Electronics
>=============================
>  
>

Re: Using systemtap to look at futexs

[Hien Nguyen]

Tim Bird wrote:

> Is there a visualization tool for SystemTap data?
> -- Tim
>  
>
We are working on the systemtap gui tool. There are two parts for this 
tool, the IDE, and data visualization. We are going to  release the 
first  code drop  for the IDE part in the near future and start working 
on the visualization part soon.

Hien.