From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 8839 invoked by alias); 10 Jul 2007 14:39:12 -0000 Received: (qmail 8832 invoked by uid 22791); 10 Jul 2007 14:39:11 -0000 X-Spam-Status: No, hits=-2.4 required=5.0 tests=AWL,BAYES_00,DK_POLICY_SIGNSOME,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (66.187.233.31) by sourceware.org (qpsmtpd/0.31) with ESMTP; Tue, 10 Jul 2007 14:39:09 +0000 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.13.1/8.13.1) with ESMTP id l6AEd6Hr003143; Tue, 10 Jul 2007 10:39:06 -0400 Received: from pobox.toronto.redhat.com (pobox.toronto.redhat.com [172.16.14.4]) by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id l6AEd6Fu018968; Tue, 10 Jul 2007 10:39:06 -0400 Received: from touchme.toronto.redhat.com (IDENT:postfix@touchme.toronto.redhat.com [172.16.14.9]) by pobox.toronto.redhat.com (8.12.11.20060308/8.12.11) with ESMTP id l6AEd5cb012135; Tue, 10 Jul 2007 10:39:05 -0400 Received: from ton.toronto.redhat.com (ton.toronto.redhat.com [172.16.14.15]) by touchme.toronto.redhat.com (Postfix) with ESMTP id C0BA4800002; Tue, 10 Jul 2007 10:39:05 -0400 (EDT) Received: from ton.toronto.redhat.com (localhost.localdomain [127.0.0.1]) by ton.toronto.redhat.com (8.13.1/8.13.1) with ESMTP id l6AEd5Ud001122; Tue, 10 Jul 2007 10:39:05 -0400 Received: (from fche@localhost) by ton.toronto.redhat.com (8.13.1/8.13.1/Submit) id l6AEd5pp001119; Tue, 10 Jul 2007 10:39:05 -0400 X-Authentication-Warning: ton.toronto.redhat.com: fche set sender to fche@redhat.com using -f To: Dave Nomura Cc: systemtap@sourceware.org, Maynard Johnson , James Keniston Subject: Re: proposed instruction trace support in SystemTap References: <4689826A.9040902@us.ibm.com> <4693CC29.8090700@us.ibm.com> From: fche@redhat.com (Frank Ch. Eigler) Date: Tue, 10 Jul 2007 14:39:00 -0000 In-Reply-To: <4693CC29.8090700@us.ibm.com> Message-ID: User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2007-q3/txt/msg00060.txt.bz2 Dave Nomura writes: > [...] The single_step and branch_step syntax identifies the handler > code and needs to be set up before the user probe code that turns > on//off instruction tracing. That goes without saying. Whatever setup need be done must be done before explicit on/off operations become available to script code. > I am assuming that since you have access to the pid() function in > your *.stp script, that the user program is invoked after all of the > probes have been processed, so I'm not sure I understand how the > actual instruction tracing events would get lost. Consider the scenario where tracing cannot be toggled on and off atomically, and yet the probe handle requesting this toggling prefers not to block. So we may have an enqueued "turn tracing back on" request that may take a little while to satisfy. During this time, we may lose events. True, this might not apply to uprobes if we choose to take advantage of its handlers' ability to sleep. But it would apply to other types of probes. > > Is the "raw trace data" a well-defined thing? Why would this sort of > > hard-coded data set be desirable, as opposed to letting the programmer > > write something explicit like: > > printf("%2b%8b%4b", cpu(), get_ticks(), pc()) > > (Of course this can be hidden in a function of his own, or in an > > inspectable tapset.) > > > The user could do the simple printf that you suggest. The proposed > callout to itrace_output() would only by used if you wanted more > detailed information (like timestamp) as required by a tool like > qtrace [...] It sounds like what we may need is a collection of functions that print tracing data in formats compatible with specific post-processing tools. It's not "raw trace data" in the itrace sense - it is trace data in the *qtrace* sense - and that of other tools. > > Is this sort of thing done/doable in kernel space also, or just on > > user-space threads? Is there an existing kernel API for management of > > these registers/fields? > [...] There is basically one single instruction trap handler that > the stap translator will generate with logic to figure out what > handler code to run [...] The "existing kernel API" is the key issue here. How exactly does one activate single-stepping traps on each of the interesting architectures, and on multiple different kernel generations (RHEL)? How does one hook into the handling system correctly (avoiding interference to other consumers of trap data like gdb, uprobes)? These questions need answers in order for systemtap to generate code to implement this. > One issue that Jim Keniston identified is that we would want some > way to not trace any of the nstructions in the kernel code > associated with stap probes, trap handler, etc. [...] It's a complicated area with potential to easily bring a system down. Let's stay away from it until we get some more experience with user-space. - FChE