request to add additional runtime path

request to add additional runtime path

[Guanglei Li]

Hi,
  for tapsets, stap could use -I option to look for additional .stp 
script files, in addition to /usr/local/share/systemTap/tapsets.

  But for runtime, the -R option could specify only one place to look 
for runtime files.

  Suppose I have some runtimes which are applicable only to some 
specific user defined tapsets, so I don't want to put them into 
/usr/local/share/systemTap/runtime. But currently I have no way to 
tell stap to look into a user specific runtime dir in addition to the 
runtime dir of systemTap itself.

  So it's a good idea to let -R option have the same behavior as -I 
option? So that I can specify multiple runtime dirs. 

Re: request to add additional runtime path

[Frank Ch. Eigler]

"Guanglei Li" <guanglei@cn.ibm.com> writes:

> [...]  Suppose I have some runtimes which are applicable only to
> some specific user defined tapsets [...]

Can you describe the nature of this C code?  Is it not suitable for
inclusion directly into a %{ %} block in your custom tapset?

- FChE

Re: request to add additional runtime path

[Guanglei Li]

----- Original Message ----- 
> Can you describe the nature of this C code?  Is it not suitable for
> inclusion directly into a %{ %} block in your custom tapset?
>
> - FChE

One example is that I want to use some macros inside the embedded c 
functions in my stp files:
function foo1()
%{
    _stp_printf("%d, %d...", EVENTID_1, HOOKID_2, ...);
%}

such macros appear almost in every stp files, so it's better to 
include a .h file in each stp file:
%{
#include "macro_defs.h"
%}

But the problem is that if macro_defs.h is put into the place other 
than /usr/local/share/systemTap/runtime, stap can't find it.

Re: request to add additional runtime path

[Frank Ch. Eigler]

Hi -


> One example is that I want to use some macros inside the embedded c 
> functions in my stp files:
> function foo1()
> %{
>     _stp_printf("%d, %d...", EVENTID_1, HOOKID_2, ...);
> %}

Not that this directly relates to your issue, but why would you use
embedded-C for something like this?  Could you post a fuller example?

It is undesirable for embedded C code to make direct calls to the
runtime.  Among other reasons, this is since that API exists for the
convenience of the translator, and thus may change frequently


> [...] such macros appear almost in every stp files, so it's better
> to include a .h file in each stp file [...]

Assuming this were a good idea, it could be supported by adding the
tapset source directories to the C compiler's include path.


- FChE

Re: request to add additional runtime path

[Guanglei Li]

> Not that this directly relates to your issue, but why would you use
> embedded-C for something like this?  Could you post a fuller 
> example?
>

In fact, I am working on a kernel event trace tool using systemTap 
which targets for performance analysis. This tool will look at some 
interesting places inside the kernel and get the relative data for 
post analysis. The places it looks at are those such as scsi layers, 
io schedulers, syscalls etc.

So I need some embedded c functions which will get the information by 
access kernel function parameters or directly calling some kernel 
functions or some other ways.

Here's one of them, and I will send out the entire codes of this trace 
tool onto mailinglist some time later:

/* when a request is retrieved from request queue */
probe addevent.ioscheduler.elv_next_request
        =  kernel.function("elv_next_request")
{
        if(filter_by_pid() == 1 )
        {
                log_tracedata_common(5)
                log_ioscheduler_tracedata_extra_elv_next($q)
        }
}

%( kernel_v == "2.6.9" %?
function log_ioscheduler_tracedata_extra_elv_next(var:long)
%{

        struct request_queue *q;
        struct request *rq;

        q = (struct request_queue *)((long)THIS->var);

        /* If there is a request in the request queue:
                elevator name|major|minor|
           if there is no request in the request queue:
                elevator name|empty|
        */
        if(list_empty(&(q->queue_head)))
        {
                _stp_printf("%s|empty|", q->elevator.elevator_name);
        }
        else  {
                rq = list_entry_rq(q->queue_head.next);
                _stp_printf("%s|%ld|%ld", q->elevator.elevator_name, 
rq->rq_disk->major, rq->rq_disk->first_minor);
        }
%}

%:
...  /* to deal with some other kernel versions, omitted ... */
%)

/* data tracing filter by pid
   return:
        1 - if continue to log the raw data
        0 - return without logging the raw data
 */
function filter_by_pid:long()
%{
        struct task_struct *cur = current;

        if(cur->tgid != _stp_pid) /* skip stapd itself */
        {
                /* to trace a specific process if we explicitly 
specify
                   which process we want to trace by:
                    1. stap -c "process_to_trace" ...
                    2. stap -x pid_to_trace ...
                   else we will trace all the processes
                 */
                if( _stp_target != 0 && cur->tgid != _stp_target)
                {
                        THIS->__retvalue = 0;
                        return;
                }

                THIS->__retvalue = 1;
        }
        else  /*skip the events generated by stap itself*/
                THIS->__retvalue = 0;
        return;
%}

/* Log the data common to all events */
function log_tracedata_common(hookID:long)
%{
        struct timeval tv;
        struct task_struct *cur = current;

        /* second|usec|pid|ppid|tid|cpuid|hoodID */

        do_gettimeofday (&tv);
        _stp_printf("\n%ld|%ld|%ld|%ld|%ld|%u|%ld|", tv.tv_sec, 
tv.tv_usec, cur->tgid, \
                cur->parent->pid, cur->pid, 
cur->thread_info->cpu,THIS->hookID);
%}

Here you can see log_tracedata_common(5), 5 is the hookID. I think 
it's better to use IOSCHED_NEXT_REQ_HOOKID instead of 5. So that it's 
easier for maintenance.

> It is undesirable for embedded C code to make direct calls to the
> runtime.  Among other reasons, this is since that API exists for the
> convenience of the translator, and thus may change frequently
>
Currently I have no need to put .c files into runtime dir.
>
>> [...] such macros appear almost in every stp files, so it's better
>> to include a .h file in each stp file [...]
>
> Assuming this were a good idea, it could be supported by adding the
> tapset source directories to the C compiler's include path.
>
Yes, in fact at present what I want is to add -I includepath when 
making the module.
>
> - FChE
>
> 

Re: request to add additional runtime path

[Frank Ch. Eigler]

"Guanglei Li" <guanglei@cn.ibm.com> writes:

> [...] The places it looks at are those such as scsi layers, 
> io schedulers, syscalls etc.

Nice.


> /* when a request is retrieved from request queue */
> probe addevent.ioscheduler.elv_next_request
>         =  kernel.function("elv_next_request")
> [...]
>                 log_tracedata_common(5)
>                 log_ioscheduler_tracedata_extra_elv_next($q)
> [...]
> Here you can see log_tracedata_common(5), 5 is the hookID. I think 
> it's better to use IOSCHED_NEXT_REQ_HOOKID instead of 5. So that it's 
> easier for maintenance.

But in this case, the literal 5 appears in script.  You could define
script globals in a separate .stp file thusly:

  # tracedata_hookids.stp
  global IOSCHED_NEXT_REQ_HOOKID # , FOO_HOOKID, BAR_HOOKID

  probe begin {
    IOSCHED_NEXT_REQ_HOOKID = 5 
    # ...
  }

Then your original stp file could refer to the variable name directly.
No C-level shared headers are needed.  The translator will pull in the
tracedata_hookids.stp file automatically.


> function log_ioscheduler_tracedata_extra_elv_next(var:long)
> %{
> [...]
>                 rq = list_entry_rq(q->queue_head.next);
>                 _stp_printf("%s|%ld|%ld", q->elevator.elevator_name, 
> rq->rq_disk->major, rq->rq_disk->first_minor);
>         }
> %}

Instead of using stp_printf, and thus calling directly into the
runtime, you might consider instead making this a function that
returns a string, which the caller script routine could print.  That
string could be built with sprintf or a friend.


> function filter_by_pid:long()
> %{
> [...]
>         if(cur->tgid != _stp_pid) /* skip stapd itself */
> [...]
>                 if( _stp_target != 0 && cur->tgid != _stp_target)
> [...]

Consider instead a single new contextinfo.stp embedded-C function to
return "_stp_pid", and code the rest of this in script.


> /* Log the data common to all events */
> function log_tracedata_common(hookID:long)
> %{
>         struct timeval tv;
>         struct task_struct *cur = current;
>         /* second|usec|pid|ppid|tid|cpuid|hoodID */
>         do_gettimeofday (&tv);
>         _stp_printf("\n%ld|%ld|%ld|%ld|%ld|%u|%ld|", tv.tv_sec, 
> tv.tv_usec, cur->tgid, \
>                 cur->parent->pid, cur->pid, 
> cur->thread_info->cpu,THIS->hookID);
> %}

This can already be coded as a single script printf statement.


> > It is undesirable for embedded C code to make direct calls to the
> > runtime.  Among other reasons, this is since that API exists for the
> > convenience of the translator, and thus may change frequently
>
> Currently I have no need to put .c files into runtime dir. [...]

I didn't mean putting new .c files there.  I meant *making calls*
directly into the runtime.  While _stp_printf is relatively tame,
there are many functions there that are not.


- FChE