From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 26834 invoked by alias); 23 Jul 2014 10:29:36 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 26780 invoked by uid 89); 23 Jul 2014 10:29:36 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-4.5 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx6-phx2.redhat.com Received: from mx6-phx2.redhat.com (HELO mx6-phx2.redhat.com) (209.132.183.39) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 23 Jul 2014 10:29:34 +0000 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s6NATSwX031387; Wed, 23 Jul 2014 06:29:28 -0400 Date: Wed, 23 Jul 2014 10:29:00 -0000 From: Nathan Scott Reply-To: Nathan Scott To: David Smith Cc: Systemtap List , pcp Message-ID: <181878137.16180790.1406111368167.JavaMail.zimbra@redhat.com> In-Reply-To: <53CE7A2E.6010306@redhat.com> References: <53C83CB9.3020808@redhat.com> <861139755.14608867.1405992742567.JavaMail.zimbra@redhat.com> <53CE7A2E.6010306@redhat.com> Subject: Re: [pcp] systemtap/pcp integration MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2014-q3/txt/msg00076.txt.bz2 Hey David, ----- Original Message ----- > On 07/21/2014 08:32 PM, Nathan Scott wrote: > > Hi David, > > > > ----- Original Message ----- > >> [...] > >> Note that systemtap will create a file called 'mmv' in > >> /proc/systemtap/{MODULE_NAME}. I've just been using pcp's 'mmvdump' > >> utility to dump the contents of the /proc/systemtap/{MODULE_NAME}/mmv > >> file. Currently the pcp mmv pmda only looks in one place for mmv files, > >> but it might be possible to create a symbolic link to systemtap's mmv > >> file to make it happy. > > > > (OOC, what's {MODULE_NAME} in this context?) > > As Frank mentioned, {MODULE_NAME} is the name of the module, usually > autogenerated. Got it. So, next I'm wondering... what is it doing here, in this interface between systemtap/pcp? (interfaces being "forever" and all that, the simpler the better). From a PCP POV it (appears to) add no value, so would seem to be something we can remove/simplify? OTOH maybe not -maybe you can use it as follows: In MMV (and the existing pmdammv, in particular), the basename of these files is used to form the first component of the metric namespace - that is, unless the caller (i.e. you, in your kernel code) sets MMV_FLAG_NOPREFIX. In that case, its ignored (see also mmv_stats_init(3), "name" parameter). So, perhaps for the stap case you could always set that flag, and use those module names as the in a /proc/mmv/ -alike scheme. That'd be a fairly simple mechanism that'd work directly with the patch I sent earlier, I think (they'd need to be files, not directories though - else some slightly more complex code is going to be needed). > > A symlink would sorta work but it feels like a bit of a workaround - the > > PMDA is written to be able to detect arrival/departure of new MMV files > > based on changes in a directory (and the location of that directory is > > parameterised via /etc/pcp.conf variables). I'd not recommend trying to > > find it within a stap script ... I imagine it will be cleaner if we go > > for separate user/kernel locations for MMV files. > > The symlink feels like a bit of a workaround, because it *is* one. Long :) > term we certainly want a better way to find the systemtap MMV files. > > > Attached patch (lightly tested) implements the PCP side of things with > > that in mind - with this patch (and making the kernel code manage the > > lifecycle of separate /proc/mmv/* entries), things should begin to work > > out-of-the-box (the MMV PMDA is already default-enabled in the default > > pmcd config file, so everything else is in place for you). > > This code sounds like a step in the right direction. Well, hopefully something to experiment with anyway; it might get you close to a complete end-to-end working scenario. > [...] > I think MMV and a JSON fetcher/parser aren't mutually exclusive. I'd say > they are complementary. Using memory-mappings to share between user/kernel (or user/user as existing PCP MMV code does) requires fixed locations in the mapping - we're accessing it via pointers on both sides of the fence. Entirely-string representations (which is what I guess the intention with a JSON interface would be?) would lose that property, since the strings (and the values in particular) do not have fixed offsets within the mapping file anymore. I guess you'd have to then completely start over again (for ... reasons?) and implement it more like a traditional /proc interface (via seq_file), where userspace sampling is via open/read/close, and not the mapped memory model where the kernel is actively writing while userspace is actively reading. I haven't seen any other need for a generic JSON interface, so can't really comment as to its suitability for this kind of thing. There is one existing JSON PMDA - the elasticsearch PMDA uses JSON encodings fairly extensively - but it's very specific & not a generic representation like you're after here. Maybe you can use it to help gauge levels of complexity to expect though. cheers. -- Nathan