Re: [pcp] suitability of PCP for event tracing

["Frank Ch. Eigler" <fche@redhat.com>]

Hi -

Thanks, Nathan, Ken, Greg, Mark, for clarifying the status quo and
some of the history.

We understand that the two problem domains are traditionally handled
with the event-tracing -vs- stats-monitoring distinction.  We're trying
to see where best to focus efforts to make some small steps to bridge
the two, where plenty of compromises are possible.  We'd prefer to
help build on an existing project with a nice community than to do new
stuff.

For the poll-based data gathering issue, a couple of approaches came up:

(1) bypassing pmcd and generating an pmarchive file directly from
    trace data This appears to imply continuing the archive-vs-live
    dichotomy that makes it difficult for clients to process both
    recent and current data seamlessly together.  Since using such
    files would probably also need a custom client, then we'd not be
    using much of the pcp infrastructure, only as a passive data
    encoding layer.  This may not be worthwhile.

(2) protocol extensions for live-push on pmda and pmcd-client interfaces
    This clearly larger effort is only worth undertaking with the
    community's sympathy and assistance.  It might have some
    interesting integration possibilities with the other tools,
    espectially pmie (the inference engine).

For the static-pmns issue, the possibility of dynamic instance
domains, metric subspaces is probably sufficient, if the event
parameters are limited to only 1-2 degrees of freedom.  (In contrast,
imagine browsing a trace of NFS or kernel VFS operations; these have
~5 parameters.)

For the scalar-payloads issue, the BLOB/STRING metric types are indeed
available but are opaque to other tools, so don't compose well.  Would
you consider one additional data type, something like a JSON[1]
string?  It would be self-describing, with pmie and general processing
opportunities, though those numbers would lack the PMDA_PMUNITS
dimensioning.

For the filtering issue, pmStore() is an interesting possibility,
allowing the PMDAs to bear the brunt.  OTOH, if pmcd evolved into a
data-push-capable widget, it could serve as a filtering proxy,
requiring separate API or interpretation of the pmStore data.

For the web-based frontend issue, yeah, javascript+svg+etc. sounds
most promising, especially if it can be made to speak the native wire
protocol to pmdc.  This would seem to argue for a stateful
archive-serving pmdc, or perhaps a archive-serving proxy, as in Greg's
old project.


Is this sounding reasonable?


- FChE

[1] http://en.wikipedia.org/wiki/JSON