[Bug translator/2307] New: binary tracing

[Bug translator/2307] New: binary tracing

[fche at redhat dot com]

See thread:

http://sources.redhat.com/ml/systemtap/2006-q1/msg00253.html

-- 
           Summary: binary tracing
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: translator
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: fche at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[guanglei at cn dot ibm dot com]

------- Additional Comments From guanglei at cn dot ibm dot com  2006-02-17 06:53 -------
The following are my thoughts about binary trace interface. In a summary, I
prefer Martin Hunt's ideas.

The idea Tom proposed will bring the programming convenience and performance
improvement. But it has some problems:
 1. required to introduce struct into systemtap script language. 
 2. Inefficiency when refer to several fields of a struct target in different
assignments. e.g:
     ...
     trace_scsi.hostno = $cmd->device->host->host_no
     trace_scsi.channel = $cmd->device->channel
     trace_scsi.lun = $cmd->device->lun
       ...
     each reference to $target will generate a serial of function
calls(function__tvar_get_cmd, fetch_register, deref etc). This is why I don't
use printf directly in LKET in most cases, but use _stp_printf() in embedded c code.

 3. Another issue is that what will happen if some fields of a struct is a
string? e.g:
     trace_process_fork.processname = str_processname
    How should systemtap generate the corresponding struct? What's the size of
the string field of that struct?


So after thinking about this topic, I prefer what Martin Hunt proposed:
  (http://sources.redhat.com/ml/systemtap/2006-q1/msg00284.html)
  
It looks like a more generic interface, and it allows mixture of data
types(binary & string). We could implement it in the runtime lib, naming it
_stp_bin_printf(const char *fmt, ...). We could introduce a new function:
bin_vsnprintf based on the existing vsnprintf to support _stp_bin_printf, which
will directly assign the integer value instead of converting the integer into a
ASCII string. Of course we should modify stpd to let it support binary data
transfer, but I think BTI should already done about this, maybe only a slight
change is required.

Jose also implemented a feature which will use /proc to control whether the
probe handler should start/stop printing data. So we could also introduce
_lket_printf(const char *fmt, ...) using bin_vsnprintf which will check the
/proc to see if it should print data now and will also do some common work like
log_tracedata_common(). But I think start/stop tracing by /proc is useful and
should be incorporated into systemtap.

By having _stp_bin_printf/bin_vsnprintf/_lket_printf, we can introduce some
wrapper functions, such as lket_trace(). One idea is to introduce lket_print
into translator, just like print, so that we could using lket_print("%d%d%s%d",
int1,int2,str1,int3) in script directly.

I am not sure if I missed some points.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[hiramatu at sdl dot hitachi dot co dot jp]

------- Additional Comments From hiramatu at sdl dot hitachi dot co dot jp  2006-02-17 10:54 -------
(In reply to comment #1)
> The following are my thoughts about binary trace interface. In a summary, I
> prefer Martin Hunt's ideas.

Basically, I agree with his ideas too.
I suggest introducing the compressed format when the arguments of the same type
continue.
For example:

printb("%d%d%d%d", long1, long2, long3, long4);

This expression will be compressed as below;

printb("%d4", long1, long2, long3, long4);

This compressed format is simple and reduces transporting data size.

By the way, I also hope the name of the interface is more neutral as printf ;-).


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[guanglei at cn dot ibm dot com]

------- Additional Comments From guanglei at cn dot ibm dot com  2006-02-17 15:32 -------
> printb("%d4", long1, long2, long3, long4);
> 
> This compressed format is simple and reduces transporting data size.
> 
In fact, this is almost the same as what I did in
LKET(log_tracedata_common_compact & log_tracedata_common_compact_more in
logtrace.stp). It should be useful for someone, although not much useful for
LKET maybe.

> By the way, I also hope the name of the interface is more neutral as printf ;-).

How about _stp_bin_printf/bin_vsnprintf corresponding to current
_stp_printf/vsnprintf? Any good suggestions about their names?

_lket_printf is intended for LKET only. 

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[jrs at us dot ibm dot com]

------- Additional Comments From jrs at us dot ibm dot com  2006-02-19 05:01 -------
Before we go deciding on a name we need to first decide on a general concept of
the interface that's going to be implemented.  I also like Martin's method for
implementing binary tracing since I find it very natural for me.  Since we have
a couple of people voting for this interface, is Martin's method the way to go?

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2006-02-20 03:00 -------
It seems to me that an approach along these lines will not satisfy optimized
tracing along IBM's original request (re. struct support in the language), nor
permit much or any translator assistance to provide metadata to a binary
decoder.  I consider it a technically weak solution.

On the other hand, just extending printf with binary formatting directives
should be easy, and not cause adverse interference with other future designs. 
So, if people want to experiment with this part, go ahead.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |NEW
   Last reconfirmed|0000-00-00 00:00:00         |2006-02-20 03:00:04
               date|                            |


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[hunt at redhat dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         AssignedTo|systemtap at sources dot    |hunt at redhat dot com
                   |redhat dot com              |


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug translator/2307] binary tracing

[hunt at redhat dot com]

------- Additional Comments From hunt at redhat dot com  2006-02-21 08:43 -------
Both proposals require some changes to the transport interface to improve
efficiency. It will also improve performance of printf to have our own optimized
copy of vsnprintf() instead of using the kernel version.  Adding binary support
to that will be trivial. So I can add binary support to printf and if we change
our minds later and want to try the other way, we haven't really wasted any time.

I'll get started on it.



-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED


http://sourceware.org/bugzilla/show_bug.cgi?id=2307

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.