From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 13880 invoked by alias); 31 Aug 2010 19:49:50 -0000 Received: (qmail 13873 invoked by uid 22791); 31 Aug 2010 19:49:49 -0000 X-SWARE-Spam-Status: No, hits=-4.2 required=5.0 tests=AWL,BAYES_50,RCVD_IN_DNSWL_HI,SPF_HELO_PASS,TW_PM,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Tue, 31 Aug 2010 19:49:44 +0000 Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o7VJnhiB003674 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 31 Aug 2010 15:49:43 -0400 Received: from fche.csb (vpn-235-228.phx2.redhat.com [10.3.235.228]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o7VJng8i002178; Tue, 31 Aug 2010 15:49:42 -0400 Received: by fche.csb (Postfix, from userid 2569) id 01C0558119; Tue, 31 Aug 2010 15:49:41 -0400 (EDT) Date: Tue, 31 Aug 2010 19:49:00 -0000 From: "Frank Ch. Eigler" To: pcp@oss.sgi.com, systemtap@sources.redhat.com Subject: Re: [pcp] suitability of PCP for event tracing Message-ID: <20100831194941.GC5762@redhat.com> References: <20100827153906.GD3185@redhat.com> <4C7A7DFE.2040606@internode.on.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C7A7DFE.2040606@internode.on.net> User-Agent: Mutt/1.4.2.2i Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2010-q3/txt/msg00331.txt.bz2 Hi - Thanks, Nathan, Ken, Greg, Mark, for clarifying the status quo and some of the history. We understand that the two problem domains are traditionally handled with the event-tracing -vs- stats-monitoring distinction. We're trying to see where best to focus efforts to make some small steps to bridge the two, where plenty of compromises are possible. We'd prefer to help build on an existing project with a nice community than to do new stuff. For the poll-based data gathering issue, a couple of approaches came up: (1) bypassing pmcd and generating an pmarchive file directly from trace data This appears to imply continuing the archive-vs-live dichotomy that makes it difficult for clients to process both recent and current data seamlessly together. Since using such files would probably also need a custom client, then we'd not be using much of the pcp infrastructure, only as a passive data encoding layer. This may not be worthwhile. (2) protocol extensions for live-push on pmda and pmcd-client interfaces This clearly larger effort is only worth undertaking with the community's sympathy and assistance. It might have some interesting integration possibilities with the other tools, espectially pmie (the inference engine). For the static-pmns issue, the possibility of dynamic instance domains, metric subspaces is probably sufficient, if the event parameters are limited to only 1-2 degrees of freedom. (In contrast, imagine browsing a trace of NFS or kernel VFS operations; these have ~5 parameters.) For the scalar-payloads issue, the BLOB/STRING metric types are indeed available but are opaque to other tools, so don't compose well. Would you consider one additional data type, something like a JSON[1] string? It would be self-describing, with pmie and general processing opportunities, though those numbers would lack the PMDA_PMUNITS dimensioning. For the filtering issue, pmStore() is an interesting possibility, allowing the PMDAs to bear the brunt. OTOH, if pmcd evolved into a data-push-capable widget, it could serve as a filtering proxy, requiring separate API or interpretation of the pmStore data. For the web-based frontend issue, yeah, javascript+svg+etc. sounds most promising, especially if it can be made to speak the native wire protocol to pmdc. This would seem to argue for a stateful archive-serving pmdc, or perhaps a archive-serving proxy, as in Greg's old project. Is this sounding reasonable? - FChE [1] http://en.wikipedia.org/wiki/JSON