From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 31076 invoked by alias); 5 Aug 2010 11:48:38 -0000 Received: (qmail 31061 invoked by uid 22791); 5 Aug 2010 11:48:35 -0000 X-SWARE-Spam-Status: No, hits=-0.7 required=5.0 tests=AWL,BAYES_00,TVD_RCVD_SPACE_BRACKET X-Spam-Check-By: sourceware.org Received: from mail7.hitachi.co.jp (HELO mail7.hitachi.co.jp) (133.145.228.42) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 05 Aug 2010 11:48:29 +0000 Received: from mlsv5.hitachi.co.jp (unknown [133.144.234.166]) by mail7.hitachi.co.jp (Postfix) with ESMTP id 2C9E237AC4; Thu, 5 Aug 2010 20:48:27 +0900 (JST) Received: from mfilter1.hitachi.co.jp by mlsv5.hitachi.co.jp (8.13.1/8.13.1) id o75BmRLq014778; Thu, 5 Aug 2010 20:48:27 +0900 Received: from hitachi.com (mfbcchk2.hitachi.co.jp [10.201.6.151]) by mfilter1.hitachi.co.jp (Switch-3.3.2/Switch-3.3.2) with ESMTP id o75AaVEY009773; Thu, 5 Aug 2010 20:48:26 +0900 Received: from vshuts3.hitachi.co.jp ([vshuts3.hitachi.co.jp [10.201.6.72]]) by mfbcchk2.hitachi.co.jp with RELAY id o75BmPBw018514 ; Thu, 5 Aug 2010 20:48:26 +0900 Received: from hsdlgw92.sdl.hitachi.co.jp (unknown [133.144.7.20]) by vshuts3.hitachi.co.jp (Symantec Mail Security) with ESMTP id 39BD2774276; Thu, 5 Aug 2010 20:48:23 +0900 (JST) Received: from vgate2.sdl.hitachi.co.jp by hsdlgw92.sdl.hitachi.co.jp (8.13.1/3.7W06092911) id o75BmMgj031149; Thu, 5 Aug 2010 20:48:22 +0900 Received: from sdl99w.sdl.hitachi.co.jp ([133.144.14.250]) by vgate2.sdl.hitachi.co.jp (SAVSMTP 3.1.1.32) with SMTP id M2010080520482230641 ; Thu, 05 Aug 2010 20:48:22 +0900 Received: from [127.0.0.1] (localhost.localdomain [127.0.0.1]) by sdl99w.sdl.hitachi.co.jp (Postfix) with ESMTP id 2C24112550F; Thu, 5 Aug 2010 20:48:20 +0900 (JST) Message-ID: <4C5AA503.8070305@hitachi.com> Date: Thu, 05 Aug 2010 11:48:00 -0000 From: Masami Hiramatsu User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: "Frank Ch. Eigler" Cc: systemtap@sources.redhat.com, Satoshi Oshima Subject: Re: [RFC] SystemTap future direction References: <4C58F852.7030103@hitachi.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-FMFTCR: RANGEC X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2010-q3/txt/msg00188.txt.bz2 Frank Ch. Eigler wrote: > masami.hiramatsu.pt wrote: > >> As you may know (of course I Cc'd discussion on LKML), Ingo and >> Christoph said that (at least) uprobes (but also kprobes) should not >> support out-of-tree module. > > This is unfortunate, but is in a way useful evidence about what sorts > of effects we can expect from further outreach efforts. You and > Srikar have spent at least a year working on mini-stap functionality > for the benefit of the perf side of the house, and yet the amount of > goodwill offered in return is ... where? Well..., there may be still less use-cases they know. >> This means that if we succeed to merge uprobes into kernel, >> SystemTap can't use uprobes itself. > > Well, there are always various social and technical measures to work > around gratuitious obstacles. > >> Even worse, if someone tries to remove kprobes' module support, that >> could shake the foundation of SystemTap. > > It is hard to imagine someone deliberately hurting linux users that way. > I hope so. and now I know we already have many users, use-cases. I think the next step is let them know the usefulness of systemtap, with actual use-case examples. > >> At least, to add support kmodules to uprobes, I think we have two >> options, one is pushing systemtap itself and useful scripts into >> kernel tree, or the other is finding very useful use-case of *probes >> which requires out-of-tree module. [...] > > Or #3, coming up with one more substantial in-tree uprobes example > than the one hch instructed srikar to drop. OK. >> [...] >> I'd like to suggest some directions here; >> >> - Merge runtime and module-source generator into linux kernel. >> This will requires rewriting whole of systemtap code from C++ to >> C or other LL (perl or python) > > More concretely, to rewrite and LKML-code-standarize the lot, but > retain current architecture? Do you sense that there's any interest > in this sort of solution by Linus? Not sure. If the on-line scripting (which requires module-generator, or in-kernel interpreter) is so useful as you think, you can convince him to support it in-kernel. Again, we already have several users who are using systemtap (with Dtrace tracepoints) agressively. I think that means there are real demands of application tracing. (I also think we still need to discuss what is the best implemantation for that.) >> - Port SystemTap on the perf/ftrace and extend perf/ftrace to support >> extend handlers which provided by modules. > > More concretely, to make a version of systemtap that instead of > generating stand-alone kernel modules that operate independently of > perf/etc., that they be bound to perf event sources & infrastructure? Right. > But retain the power of our system by still executing arbitrary > generated code from those callbacks? Do you sense that there's any > interest in this sort of solution by the perf people? I think so, if we can indicate the flexibility of on-line scripting and its power, we may convince them. > Now if we're talking about a module-encased bytecode interpreter / JIT > rich enough to encompass our runtime/language features, I have some > interest in this sort of solution, whether coupled or decoupled from > perf. But this is a large amount of effort. But we're tempted. Yeah, me too :) Actually, Ingo had once suggested to build an interpreter in kernel. >> - Port SystemTap on the perf/ftrace but drop embedded-C support. >> This will enhance perf/ftrace to support enough flexible data >> filter/modifier (including fault injection feature). In this case, >> SystemTap scripts will handle the data in user-space (not on-line). > > I get the sense the perf people believe they are on this course > already, without needing any help. Yeah, I just would like to help systemtap users to run their stap scripts even if perf people choose this way. >> - Or, just do nothing and wait for kernel maintainers choking >> our necks... > > I don't think the situation is in fact deteriorating. We're shipping > decent releases, growing our user base, within and without the kernel > developer community, and still have plenty of major feature areas to > work on. We have not seen regressive LKML obstructions, though > admittedly that is a low standard when it comes to serving the > community. Maybe I'm a paranoid. However, I can't see the things getting better too, at least in the kernel mailing list. perf and ftrace are becoming to de-facto standard tracing tools among the linux kernel community. Indeed systemtap has growing user base and major features. And then, why NOT appeal that? I can't see any systemtap developers in the collaboration summit this year, and on LinuxCon attendee list. Recently I feel that we are losing presence among the linux kernel community, even though systemtap can run ONLY on the linux. Anyway, I plan to hold a tracing panel discussion with systemtap users and perf/ftrace developers in LinuxCon Japan and will try to clarify what features users need. I hope that helps things going forward. Thank you, -- Masami HIRAMATSU 2nd Research Dept. Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com