[Bug translator/2115] New: support some function calls on maps

[Bug translator/2115] New: support some function calls on maps

[hunt at redhat dot com]

Systemtap does not recognize maps as a type so they cannot be passed to
functions. However, there are a number of functions that need to operate on maps
that must be recognized by the translator.  Using foreach as the method of doing
all map operations is too limiting.

The needed functions are:
sort, printa, normalize, and clear

These would need to simply call the runtime with the proper parameters.

-- 
           Summary: support some function calls on maps
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: translator
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: hunt at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2115] support some function calls on maps

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2006-01-06 18:17 -------
"clear" == "delete MAP"
"sort" == "sorted foreach"
"normalize" == what's that?
"printa" == an unfinished extension of the print statement


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

Re: [Bug translator/2115] support some function calls on maps

[Vara Prasad]

fche at redhat dot com wrote:

>------- Additional Comments From fche at redhat dot com  2006-01-06 18:17 -------
>"clear" == "delete MAP"
>"sort" == "sorted foreach"
>  
>
Are you referring to the foreach extension you proposed in this msg 
http://sourceware.org/ml/systemtap/2005-q4/msg00488.html. If that is the 
case i personally don't like extending foreach in every possible way and 
if we do this as you indicated we will make it look like common LISP .

>"normalize" == what's that?
>  
>
Normalize the data in the stat over a constant. For example i am 
observing performance of write systemcall and i know if the system is 
working normally what the value should be i would like to normalize the 
data collected over that value so that i know what the deviation from 
the normal is. Does this make sense?

>"printa" == an unfinished extension of the print statement
>  
>
I am not sure i understand what you mean by this. I personally like the 
idea of a print statement for aggregations separate from print statement 
for scalar variables as the contents of both are quiet different. I 
would like to see printfa() function which gives me an ability to 
specify the format for printing aggregates. I think Martin proposed 
printa() few weeks back, i was hoping we are going to get that implemented.

>
>  
>

[Bug translator/2115] support some function calls on maps

[prasadav at us dot ibm dot com]

------- Additional Comments From prasadav at us dot ibm dot com  2006-01-06 18:34 -------
Subject: Re:  support some function calls on maps

fche at redhat dot com wrote:

>------- Additional Comments From fche at redhat dot com  2006-01-06 18:17 -------
>"clear" == "delete MAP"
>"sort" == "sorted foreach"
>  
>
Are you referring to the foreach extension you proposed in this msg 
http://sourceware.org/ml/systemtap/2005-q4/msg00488.html. If that is the 
case i personally don't like extending foreach in every possible way and 
if we do this as you indicated we will make it look like common LISP .

>"normalize" == what's that?
>  
>
Normalize the data in the stat over a constant. For example i am 
observing performance of write systemcall and i know if the system is 
working normally what the value should be i would like to normalize the 
data collected over that value so that i know what the deviation from 
the normal is. Does this make sense?

>"printa" == an unfinished extension of the print statement
>  
>
I am not sure i understand what you mean by this. I personally like the 
idea of a print statement for aggregations separate from print statement 
for scalar variables as the contents of both are quiet different. I 
would like to see printfa() function which gives me an ability to 
specify the format for printing aggregates. I think Martin proposed 
printa() few weeks back, i was hoping we are going to get that implemented.

>
>  
>




-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2115] support some function calls on maps

[hunt at redhat dot com]

------- Additional Comments From hunt at redhat dot com  2006-01-06 18:35 -------
(In reply to comment #1)
> "clear" == "delete MAP"
OK. I keep forgetting delete = clear

> "sort" == "sorted foreach"
> "normalize" == what's that?
It does what dtrace normalize does. Not in the runtime yet, but trivial.

> "printa" == an unfinished extension of the print statement

The problem is that our map manipulation is awkward, at best.  The foreach
syntax is getting very strange.  and adding a map print to the print() functions
is of little use if we cannot sort (and possibly normalize) the data first.





-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2115] support some function calls on maps

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2006-01-09 18:25 -------
Re "normalize", this seems to be a bit of a kludge in dtrace that comes from
their inability to read the stats arrays at runtime.  Systemtap users could
explicitly perform an equivalent scaling operation at print time, though more
clumsily for the histograms.  Or the translator could provide an implicit
scaling facility (perhaps via an auxiliary scalar variable for each array),
which again is used only for printing.  Unless I'm mistaken, the runtime need
not deal with this at all.

Re "printa", we've been over this several times, and the existing
sorted-foreach/print pair was not worse than the proposed sort/print sequence.

This leaves only iteration-limited sort/print, possibly combined with deletion,
which is already being discussed in bug #2051.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2115] support some function calls on maps

[hunt at redhat dot com]

------- Additional Comments From hunt at redhat dot com  2006-01-16 09:20 -------
(In reply to comment #4)

> Re "printa", we've been over this several times, and the existing
> sorted-foreach/print pair was not worse than the proposed sort/print sequence.

I only agreed that the script complexity was not too much worse. Because you can
extend the foreach to do anything you want:

foreach ([a,b,c+,d] in foo limit 10  mormalize 1024) {
  printf("foo[%d,%d,%d,%d]: count:%d  sum:%d  avg:%d  min:%d  max:%d\n", 
      a, b, c, d, @count(foo[a,b,c,d]), 
      @sum(foo[a,b,c,d]), @avg(foo[a,b,c,d]),
      @min(foo[a,b,c,d]), @max(foo[a,b,c,d]))
  print(@hist_log(foo[a,b,c,d]))
}
 - compared to something like -
sortn(foo, KEY3, 10)
normalize(foo, 1024)
printa(foo, 10, "foo[%1,%2,%3,%4]: count:%C  sum:%S  avg:%A  min:%m  max:%M\n%H)

When comparing lines of code generated, which increases compile time and module
size, there is no comparison. The second approach is just a a few calls to the
runtime. 

When comparing executing time, the best case showed the foreach taking 50% more
time and that was for plain maps. Stats are worse.  Worse case is much, much
worse. The reason should be obvious if you look at the code generated. Foreach
iterates through the map nodes getting keys. Then it uses those keys over and
over again to look up the node it already had to get stats pointer to get count,
then sum, etc. Worse case is keys are 5 strings so each of those is copied
twice, hashed, etc. 

Of course execution time is really only a concern when using timers because we
are holding a lock.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2115

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2115] support some function calls on maps

[fche at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=2115

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |fche at redhat dot com
         Resolution|---                         |WONTFIX

--- Comment #6 from Frank Ch. Eigler <fche at redhat dot com> ---
Macros may express some of this stuff tighter;
for efficient use of stats values, see bug #6746.

-- 
You are receiving this mail because:
You are the assignee for the bug.