From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 26319 invoked by alias); 18 Jul 2014 18:27:57 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 26308 invoked by uid 89); 18 Jul 2014 18:27:56 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.7 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Fri, 18 Jul 2014 18:27:55 +0000 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s6IIRqdG007325 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 18 Jul 2014 14:27:52 -0400 Received: from fche.csb (vpn-61-251.rdu2.redhat.com [10.10.61.251]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s6IIRqRN022102; Fri, 18 Jul 2014 14:27:52 -0400 Received: by fche.csb (Postfix, from userid 2569) id 5CF8858390; Fri, 18 Jul 2014 14:27:51 -0400 (EDT) Date: Fri, 18 Jul 2014 18:27:00 -0000 From: "Frank Ch. Eigler" To: David Smith Cc: Systemtap List , pcp Subject: Re: systemtap/pcp integration Message-ID: <20140718182751.GE20905@redhat.com> References: <53C83CB9.3020808@redhat.com> <53C94A6F.4080808@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53C94A6F.4080808@redhat.com> User-Agent: Mutt/1.4.2.2i X-SW-Source: 2014-q3/txt/msg00041.txt.bz2 Hi - dsmith wrote: > [...] > > Overall, are you happy with the general approach of reusing the exact > > MMV format (and thus the PMDA)? > > [...] > However, as I've worked with the MMV format I've come to realize its > limitations. As Nathan has pointed out in another email, the MMV format > is designed to only support exporting values, and isn't suited for more > event-like tracing. As far as the more technical side of things goes, > some of the internal offset logic might be done better/differently. An application of pmda/mmv & pmda/logger to the same stap module could perhaps accomplish both goals (assuming we consider the pcp events overengineered to the extent that supplying timestamped strings is sufficient). Have you considered an alternative unified design? This reminds me of another PCP PMDA we've mentioned in the past: a JSON fetcher/parser. We'll need something like this for a variety of non-systemtap purposes too (interop with JSON-producing tools). What if stap were to produce pcp metrics in the form of /proc/systemtap/* JSON files that the PMDA would read on demand? (The cost of the parsing overhead may be low enough not to worry about it.) A separate generated JSON file could provide metadata. That format could be rich enough to contain events too (mapped from arrays of string). > > At one point I suggested reworking the earlier prototype so that the > > bulk of the MMV format's emulation would be based on tapset script > > code (and possibly more declarative / dynamic / safe) rather than C. > > Have you come to any conclusions about the propriety of that? > > I've been focused on other things, like reworking the allocation logic. > As you describe it above, I'm not sure I see where you are headed. To spell it out, the idea was to encode the mmv format logic (including metadata management) within a stap tapset script instead of as C in the runtime. Then the C runtime would need to do nothing but provide a memory-mapped-byte-array kind of abstraction, and a way for the script code to read/write it (maybe a variant of sprintf("%b...")?). > [...] > > How much post-initialization change can the MMV format tolerate, as > > regarding indom contents or metric availability? I assume such > > metadata changes are synchronized with the PMDA via the generation > > numbers. Moving around contents of the mmap region as in > > __stp_mmv_alloc_data_item sounds like it leaves the data inconsistent > > during the process; does it need similar protection? > > The MMV format doesn't support any post-initialization changes - once > you call "start" the file format can't change without removing and > recreating the file. (The reader knows "start" has been called based on > when the generation numbers match.) [...] (I think generation numbers can be changed during the run, to trigger a pmda/mmv reload, but don't know how thoroughly that works.) - FChE