Re: [pcp] suitability of PCP for event tracing

[nathans@aconex.com]

----- "Ken McDonell" <kenj@internode.on.net> wrote:

> Typed strings is a concept that probably makes sense independent of
> the
> event records stuff, e.g. a simple metric that returns a
> PM_TYPE_STRING
> value where the producer and consumer agree the value is encoded in a
> particular way, e.g. JSON.

*nod*, very good point.

> One option would be to add more values to the sem (semantics) field
> of the pmDesc, e.g. PM_SEM_XML, PM_SEM_JSON, ... these additonal
> semantic
> settings would need to imply PM_SEM_DISCRETE or PM_SEM_INSTANT.  This
> would
>       * make them available for all metrics (not just the parameter(s)
>         for an event record), and
>       * involve NO PDU, API or archive changes
>       * seem more natural as it is part of the semantic metadata for
>         the metric, rather than something to be encoded in the vtype
>         header
>         of a pmValueBlock (which is where the PM_TYPE_* data for the
>         composite data types are carried around in each fetch result)
> 
> Thoughts?

Yep, that does sound like a much better approach.

> > | typedef struct {
> > |     struct timeval	er_timestamp;
> > |     int		er_nparams;	/* number of er_param[] entries */
> > |     pmEventParameter	er_param[1];
> > | } pmEventRecord;
> > 
> 
> I considered er_flags but this seems to be dependent on prior
> agreement of the producer and consumer, 

...?  To my mind, its the opposite.  The idea is that these flags
encapsulate the generic characteristics of traces, which allows a
generic client to make more sense of them.

For example, pmchart could do a decent job of displaying traces with
these flags, but it has no chance without them (cannot tell what is
a start event, whats an end event, nor how to build a hierarchy ...
for an arbitrary trace).  Thinking specifically of the gantt chart
(e.g. http://www.bootchart.org/images/bootchart.png) class of trace
visualisation here, if that helps clarify wtf I'm on about.  That is
feasible (and useful!) for arbitrary traces via er_flags, I think.

>  Also, there is no obvious place to store the
> value when the packed array of event records is expanded (it cannot
> go
> in the pmResult, so needs another allocation and return parameter for
> pmUnpackEventRecords (as you indicated later).

So, this seems to be more of an issue, yes it complicates the API a
bit ... but its the difference between something bland and something
alot more useful IMO.

> I'd prefer to see event "flags" or event "type" encoded in a specific
> parameter of the event record.  This also needs prior agreement by
> the
> consumer and the producer, enables all of the same functionality that
> er_flags would, and requires no additional API or data structure
> changes.

The problem there, as you say, is that it means client tools must all
be custom written (prior agreement)... theres really not much useful/
generic that pmchart could do with this.

> Not so ... the number of metrics per PMDA is 2^10 (items) x (2^12-1)
> (clusters) because I've only pinched one value for the cluster field.

Ah, my mistake - thats great!

> I suspect we need to wait for someone to use all this new functionality
> in anger before we will be in a position to consider tweaks to what
> has been done so far.

*nod*.  Will get a code review in too when time permits (not for a week
or two though).

cheers.

-- 
Nathan