Re: [pcp] systemtap/pcp integration

[Nathan Scott <nathans@redhat.com>]

Hey David,

----- Original Message -----
> On 07/21/2014 08:32 PM, Nathan Scott wrote:
> > Hi David,
> > 
> > ----- Original Message -----
> >> [...]
> >> Note that systemtap will create a file called 'mmv' in
> >> /proc/systemtap/{MODULE_NAME}. I've just been using pcp's 'mmvdump'
> >> utility to dump the contents of the /proc/systemtap/{MODULE_NAME}/mmv
> >> file. Currently the pcp mmv pmda only looks in one place for mmv files,
> >> but it might be possible to create a symbolic link to systemtap's mmv
> >> file to make it happy.
> > 
> > (OOC, what's {MODULE_NAME} in this context?)
> 
> As Frank mentioned, {MODULE_NAME} is the name of the module, usually
> autogenerated.

Got it.  So, next I'm wondering... what is it doing here, in this interface
between systemtap/pcp?  (interfaces being "forever" and all that, the simpler
the better).  From a PCP POV it (appears to) add no value, so would seem to
be something we can remove/simplify?  OTOH maybe not -maybe you can use it as
follows:

In MMV (and the existing pmdammv, in particular), the basename of these files
is used to form the first component of the metric namespace - that is, unless
the caller (i.e. you, in your kernel code) sets MMV_FLAG_NOPREFIX.  In that
case, its ignored  (see also mmv_stats_init(3), "name" parameter).

So, perhaps for the stap case you could always set that flag, and use those
module names as the <xxx> in a /proc/mmv/<xxx> -alike scheme.  That'd be a
fairly simple mechanism that'd work directly with the patch I sent earlier,
I think (they'd need to be files, not directories though - else some slightly
more complex code is going to be needed).

> > A symlink would sorta work but it feels like a bit of a workaround - the
> > PMDA is written to be able to detect arrival/departure of new MMV files
> > based on changes in a directory (and the location of that directory is
> > parameterised via /etc/pcp.conf variables).  I'd not recommend trying to
> > find it within a stap script ... I imagine it will be cleaner if we go
> > for separate user/kernel locations for MMV files.
> 
> The symlink feels like a bit of a workaround, because it *is* one. Long

:)

> term we certainly want a better way to find the systemtap MMV files.
> 
> > Attached patch (lightly tested) implements the PCP side of things with
> > that in mind - with this patch (and making the kernel code manage the
> > lifecycle of separate /proc/mmv/* entries), things should begin to work
> > out-of-the-box (the MMV PMDA is already default-enabled in the default
> > pmcd config file, so everything else is in place for you).
> 
> This code sounds like a step in the right direction.

Well, hopefully something to experiment with anyway; it might get you close
to a complete end-to-end working scenario.

> [...] 
> I think MMV and a JSON fetcher/parser aren't mutually exclusive. I'd say
> they are complementary.

Using memory-mappings to share between user/kernel (or user/user as existing
PCP MMV code does) requires fixed locations in the mapping - we're accessing
it via pointers on both sides of the fence.  Entirely-string representations
(which is what I guess the intention with a JSON interface would be?) would
lose that property, since the strings (and the values in particular) do not
have fixed offsets within the mapping file anymore.

I guess you'd have to then completely start over again (for ... reasons?) and
implement it more like a traditional /proc interface (via seq_file), where
userspace sampling is via open/read/close, and not the mapped memory model
where the kernel is actively writing while userspace is actively reading.

I haven't seen any other need for a generic JSON interface, so can't really
comment as to its suitability for this kind of thing.  There is one existing
JSON PMDA - the elasticsearch PMDA uses JSON encodings fairly extensively -
but it's very specific & not a generic representation like you're after here.
Maybe you can use it to help gauge levels of complexity to expect though.

cheers.

--
Nathan