RE: simple (dumb) script to track sizes of IOs

RE: simple (dumb) script to track sizes of IOs

[Zhang, Yanmin]

>>-----Original Message-----
>>From: systemtap-owner@sources.redhat.com
>>[mailto:systemtap-owner@sources.redhat.com] On Behalf Of Frank Ch. Eigler
>>Sent: 2005年10月9日 0:45
>>To: systemtap@sources.redhat.com
>>Subject: RE: simple (dumb) script to track sizes of IOs
>>
>>Hi -
>>
>>anmin.zhang@intel.com wrote:
>>
>>> Could a timer be added and the report be printed periodically? Users
>>> are used to see the data difference in timely style? The timer
>>> interval can be set as a parameter.
>>
>>Certainly.  Add the single line
>>
>>probe timer.jiffies(1000 /* tune as required */) { report() }
Thanks. I know how to add timer to a stap script. I just want to suggest the developer to change the script of tracking sizes of IO to report statistics periodically instead of in the end.

RE: simple (dumb) script to track sizes of IOs

[Frank Ch. Eigler]

Hi -

anmin.zhang@intel.com wrote:

> Could a timer be added and the report be printed periodically? Users
> are used to see the data difference in timely style? The timer
> interval can be set as a parameter.

Certainly.  Add the single line

probe timer.jiffies(1000 /* tune as required */) { report() }

- FChE

RE: simple (dumb) script to track sizes of IOs

[Zhang, Yanmin]

>>-----Original Message-----
>>From: systemtap-owner@sources.redhat.com
>>[mailto:systemtap-owner@sources.redhat.com] On Behalf Of Daniel P. Berrange
>>Sent: 2005年10月8日 6:42
>>To: Frank Ch. Eigler
>>Cc: Badari Pulavarty; systemtap@sources.redhat.com
>>Subject: Re: simple (dumb) script to track sizes of IOs
>>
>>On Fri, Oct 07, 2005 at 06:01:50PM -0400, Frank Ch. Eigler wrote:
>>> Hi -
>>>
>>> On Fri, Oct 07, 2005 at 02:18:37PM -0700, Badari Pulavarty wrote:
>>> > [...]
>>> > > > Does this functionality exists today ?
>>> > >
>>> > > Yes, it does.  Just to keep the parsing simple, I implemented a simple
>>> > > suffix syntax.  For example:
>>> > >
>>> > > foreach (io+ in iosizes) { ... }  // in increasing order of io index
>>> > > foreach (io in iosizes-) { ... }  // in decreasing order of iosizes[]
>>>
>>> By the way, if someone has better suggestions of a nice, compact,
>>> easy-to-read and -parse alternative, let me know.
>>
>>How about something either using 'asc' or 'desc' as the operators
>>
>>  foreach (io asc iosizes) { ... }
>>  foreach (io desc iosizes) { ... }
>>
>>Or using 'asc' 'desc' as modifiers for the 'in' operator
>>
>>  foreach (io in asc iosizes) { ... }
>>  foreach (io in desc iosizes) { ... }
>>
>>Regards,
>>Dan.
>>--
>>|=- Red Hat, Global Professional Services, London.  +44 (0)7977 267 243 -=|
>>|=-         Perl modules: http://search.cpan.org/~danberr/              -=|
>>|=-             Projects: http://freshmeat.net/~danielpb/
>>-=|
>>|=- GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

Could a timer be added and the report be printed periodically? Users are used to see the data difference in timely style? The timer interval can be set as a parameter.

Re: simple (dumb) script to track sizes of IOs

[Daniel P. Berrange]

On Fri, Oct 07, 2005 at 06:01:50PM -0400, Frank Ch. Eigler wrote:
> Hi -
> 
> On Fri, Oct 07, 2005 at 02:18:37PM -0700, Badari Pulavarty wrote:
> > [...]
> > > > Does this functionality exists today ?
> > > 
> > > Yes, it does.  Just to keep the parsing simple, I implemented a simple
> > > suffix syntax.  For example:
> > > 
> > > foreach (io+ in iosizes) { ... }  // in increasing order of io index
> > > foreach (io in iosizes-) { ... }  // in decreasing order of iosizes[]
> 
> By the way, if someone has better suggestions of a nice, compact,
> easy-to-read and -parse alternative, let me know.

How about something either using 'asc' or 'desc' as the operators

  foreach (io asc iosizes) { ... }
  foreach (io desc iosizes) { ... }

Or using 'asc' 'desc' as modifiers for the 'in' operator

  foreach (io in asc iosizes) { ... }
  foreach (io in desc iosizes) { ... }

Regards,
Dan.
-- 
|=- Red Hat, Global Professional Services, London.  +44 (0)7977 267 243 -=|
|=-         Perl modules: http://search.cpan.org/~danberr/              -=|
|=-             Projects: http://freshmeat.net/~danielpb/               -=|
|=- GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| 

Re: simple (dumb) script to track sizes of IOs

[Frank Ch. Eigler]

Hi -

On Fri, Oct 07, 2005 at 02:18:37PM -0700, Badari Pulavarty wrote:
> [...]
> > > Does this functionality exists today ?
> > 
> > Yes, it does.  Just to keep the parsing simple, I implemented a simple
> > suffix syntax.  For example:
> > 
> > foreach (io+ in iosizes) { ... }  // in increasing order of io index
> > foreach (io in iosizes-) { ... }  // in decreasing order of iosizes[]

By the way, if someone has better suggestions of a nice, compact,
easy-to-read and -parse alternative, let me know.

> BTW, don't you want to do
> foreach(io- in iosizes) { ..}
> to be consistent ?

Such permutations work too.

- FChE

Re: simple (dumb) script to track sizes of IOs

[Badari Pulavarty]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]

On Fri, 2005-10-07 at 15:17 -0400, Frank Ch. Eigler wrote:
> Badari Pulavarty <pbadari@us.ibm.com> writes:
> 
> > > > [...]
> > > > Is there a way, I can print in the order of IO sizes easily ?
> > > There will be a way of iterating in a sorted order, probably
> > > expressed syntactically like this: [...]
> 
> > > #   foreach (ascending(io) in iosizes) {
> 
> > Does this functionality exists today ?
> 
> Yes, it does.  Just to keep the parsing simple, I implemented a simple
> suffix syntax.  For example:
> 
> foreach (io+ in iosizes) { ... }  // in increasing order of io index
> foreach (io in iosizes-) { ... }  // in decreasing order of iosizes[]

Cool. It works & output looks much better !!

BTW, don't you want to do

foreach(io- in iosizes) { ..}

to be consistent ?

Thanks,
Badari

[root@elm3b23 systemtap.samples]# stap iosizes.stp
io sizes tracking, start time=1128719937
iosize  = 1k     iocount = 2
iosize  = 2k     iocount = 1
iosize  = 4k     iocount = 24
iosize  = 8k     iocount = 4
iosize  = 12k    iocount = 2
iosize  = 16k    iocount = 2
iosize  = 20k    iocount = 1
iosize  = 28k    iocount = 3
iosize  = 32k    iocount = 2
iosize  = 48k    iocount = 1
iosize  = 108k   iocount = 1
iosize  = 120k   iocount = 1
iosize  = 128k   iocount = 13886
iosize  = 208k   iocount = 1
iosize  = 228k   iocount = 1
iosize  = 276k   iocount = 1
iosize  = 332k   iocount = 1
iosize  = 388k   iocount = 1
iosize  = 424k   iocount = 1
iosize  = 440k   iocount = 2
iosize  = 512k   iocount = 1024
io sizes tracking, end time=1128719961




[-- Attachment #2: iosizes.stp --]
[-- Type: text/plain, Size: 662 bytes --]

#! stap
# Simple (dumb) script to figure out the sizes of IOs getting
# generated.
#
# TODO: 
#	- add per device counters
#	- hook it up in right places (to handle failures correctly)
#

global iosizes

probe kernel.function("blk_rq_map_sg") {
   iosizes [$rq->nr_sectors] ++
}

function _(n) { return string(n) } 

function report () {
  foreach (io+ in iosizes) {
	print("iosize  = " . _(io/2) .
	      "k\t iocount = " . _(iosizes[io]) .
	      "\n")
  }
  delete iosizes
}


probe begin {
  print ("io sizes tracking, start time=" . _(gettimeofday_s()) . "\n")
}
probe end {
  report()
  print ("io sizes tracking, end time=" . _(gettimeofday_s()) . "\n")
}

Re: simple (dumb) script to track sizes of IOs

[Frank Ch. Eigler]

Badari Pulavarty <pbadari@us.ibm.com> writes:

> > > [...]
> > > Is there a way, I can print in the order of IO sizes easily ?
> > There will be a way of iterating in a sorted order, probably
> > expressed syntactically like this: [...]

> > #   foreach (ascending(io) in iosizes) {

> Does this functionality exists today ?

Yes, it does.  Just to keep the parsing simple, I implemented a simple
suffix syntax.  For example:

foreach (io+ in iosizes) { ... }  // in increasing order of io index
foreach (io in iosizes-) { ... }  // in decreasing order of iosizes[]


- FChE

Re: simple (dumb) script to track sizes of IOs

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 472 bytes --]

Hi -

> > > [...]
> > > Is there a way, I can print in the order of IO sizes easily ?
> > There will be a way of iterating in a sorted order [...]
> Does this functionality exists today ?

Not yet.  Subscribe to bug #1366.

> [...] Is there some documentation on the allowed syntax and library
> functions ?

There are very few library functions, but they are documented in the
stapfuncs(5) man page.  General language syntax is outlined in the
stap(1) man page.


- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: simple (dumb) script to track sizes of IOs

[Badari Pulavarty]

Frank Ch. Eigler wrote:
> pbadari wrote:
> 
> 
>>Here is a simple systemtap script to track the sizes of IOs 
>>that are getting generated (to the driver).
> 
> 
> That's block i/o only right?

Yes.

> 
> 
>>[...]
>>Is there a way, I can print in the order of IO sizes easily ?
> 
> 
> There will be a way of iterating in a sorted order, probably
> expressed syntactically like this:
> 



> # function report () {
> #   foreach (ascending(io) in iosizes) {
> #         if (iosizes[io]) # ...
> # }
> 

Does this functionality exists today ?

> 
>>[...]# stap -g iosizes.stp [...]
> 
> 
> Your script does not need guru mode.  Merely reading $rq->nr_sectors
> is fine without that.  You also don't need to check "if (iosizes[io])"
> since zero-valued array elements do not appear in foreach()
> iterations.

Thanks. Is there some documentation on the allowed syntax and
library functions ?

Thanks,
Badari

Re: simple (dumb) script to track sizes of IOs

[Frank Ch. Eigler]

pbadari wrote:

> Here is a simple systemtap script to track the sizes of IOs 
> that are getting generated (to the driver).

That's block i/o only right?

> [...]
> Is there a way, I can print in the order of IO sizes easily ?

There will be a way of iterating in a sorted order, probably
expressed syntactically like this:

# function report () {
#   foreach (ascending(io) in iosizes) {
#         if (iosizes[io]) # ...
# }

> [...]# stap -g iosizes.stp [...]

Your script does not need guru mode.  Merely reading $rq->nr_sectors
is fine without that.  You also don't need to check "if (iosizes[io])"
since zero-valued array elements do not appear in foreach()
iterations.

- FChE

simple (dumb) script to track sizes of IOs

[Badari Pulavarty]

[-- Attachment #1: Type: text/plain, Size: 1194 bytes --]

Hi,

Here is a simple systemtap script to track the sizes of IOs 
that are getting generated (to the driver).

Is this useful for anyone ? Earlier (before systemtap), I did 
this in the kernel to figure out the IO pattern thats getting
generated from a database run (to check the IO merging and 
optimize). Since I was doing it in the kernel, I had better
places to add counters to avoid all the error handling and 
retry cases.

BTW, feel free to enhance it to complete TODO list.

Is there a way, I can print in the order of IO sizes easily ?

Thanks,
Badari

Here is the output:

[root@elm3b23 systemtap.samples]# stap -g iosizes.stp
io sizes tracking, start time=1128617047
iosize  = 124k   iocount = 1
iosize  = 92k    iocount = 1
iosize  = 4k     iocount = 11
iosize  = 8k     iocount = 2
iosize  = 32k    iocount = 2
iosize  = 48k    iocount = 1
iosize  = 172k   iocount = 1
iosize  = 12k    iocount = 1
iosize  = 364k   iocount = 1
iosize  = 2k     iocount = 1
iosize  = 1k     iocount = 1
iosize  = 512k   iocount = 3371
iosize  = 256k   iocount = 5505
iosize  = 72k    iocount = 1
iosize  = 128k   iocount = 6830
iosize  = 44k    iocount = 1
io sizes tracking, end time=1128617074




[-- Attachment #2: iosizes.stp --]
[-- Type: text/plain, Size: 727 bytes --]

#! stap
# Simple (dumb) script to figure out the sizes of IOs getting
# generated.
#
# TODO: 
#	- sort the output by iosizes for easy read
#	- add per device counters
#	- hook it up in right places (to handle failures correctly)
#

global iosizes

probe kernel.function("blk_rq_map_sg") {
   iosizes [$rq->nr_sectors] ++
}

function _(n) { return string(n) } 

function report () {
  foreach (io in iosizes) {
	if (iosizes[io])
		print("iosize  = " . _(io/2) .
		      "k\t iocount = " . _(iosizes[io]) .
		      "\n")
  }
  delete iosizes
}


probe begin {
  print ("io sizes tracking, start time=" . _(gettimeofday_s()) . "\n")
}
probe end {
  report()
  print ("io sizes tracking, end time=" . _(gettimeofday_s()) . "\n")
}