From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 29494 invoked by alias); 6 Jun 2006 19:08:21 -0000 Received: (qmail 29486 invoked by uid 22791); 6 Jun 2006 19:08:19 -0000 X-Spam-Status: No, hits=-3.1 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (66.187.233.31) by sourceware.org (qpsmtpd/0.31) with ESMTP; Tue, 06 Jun 2006 19:07:43 +0000 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.12.11.20060308/8.12.11) with ESMTP id k56J7eGD029425 for ; Tue, 6 Jun 2006 15:07:40 -0400 Received: from pobox.toronto.redhat.com (pobox.toronto.redhat.com [172.16.14.4]) by int-mx1.corp.redhat.com (8.12.11.20060308/8.12.11) with ESMTP id k56J7diX001083 for ; Tue, 6 Jun 2006 15:07:40 -0400 Received: from touchme.toronto.redhat.com (IDENT:postfix@touchme.toronto.redhat.com [172.16.14.9]) by pobox.toronto.redhat.com (8.12.8/8.12.8) with ESMTP id k56J7d5C009275 for ; Tue, 6 Jun 2006 15:07:39 -0400 Received: from ton.toronto.redhat.com (ton.toronto.redhat.com [172.16.14.15]) by touchme.toronto.redhat.com (Postfix) with ESMTP id 9F34D80004C for ; Tue, 6 Jun 2006 15:07:39 -0400 (EDT) Received: from ton.toronto.redhat.com (localhost.localdomain [127.0.0.1]) by ton.toronto.redhat.com (8.13.1/8.13.1) with ESMTP id k56J7d43002513 for ; Tue, 6 Jun 2006 15:07:39 -0400 Received: (from fche@localhost) by ton.toronto.redhat.com (8.13.1/8.13.1/Submit) id k56J7dXL002510; Tue, 6 Jun 2006 15:07:39 -0400 X-Authentication-Warning: ton.toronto.redhat.com: fche set sender to fche@redhat.com using -f To: systemtap@sources.redhat.com Subject: user-space probes -- plan B from outer space From: fche@redhat.com (Frank Ch. Eigler) Date: Tue, 06 Jun 2006 19:08:00 -0000 Message-ID: User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2006-q2/txt/msg00570.txt.bz2 Hi - Here is an outline of how systemtap might support user-space probes, even in the absence of kernel-based user kprobes. This is a "plan B" only, a desperate stopgap until lkml sees the light. Maybe "plan Z" is more appropriate, considering the limitations I'm about to outline. The idea is to support limited systemtap scripts that refer only to user-space probe targets such as existing processes. These scripts would be translated to a user-space probe program instead of a kernel probe module. Probes would be specified with a probe point syntax such as: user.process(233).statement(0xfeedface) user("fche").process("/bin/vi").function("*init*") Instead of kprobes of a probe module, this probe program would use ptrace to insert breakpoints into any target processes, perhaps using code from RDA or GDB. Given the process-id or process name, systemtap should be able to locate the necessary debugging information at translation time. When probes are hit, the probe process would run the compiled probe handlers in much the same way as now. Access to $target vars should be possible. The runtime code would have to have a new variant to use some user-level facility (plain pipes?) to communicate with the front-end. Q: Wouldn't this be slow? A: Oh yes, quite. Several ptrace context-switch round-trips per probe hit. Lots more if we want to pull out target-side state like $variables or stack backtraces. Q: What about concurrency? A: You mean like probes concurrently hit in several target processes, like SMP kprobes? If there was any indication that this was worthwhile, then we could make the systemtap-generated probe process be multi-threaded (one probe thread per target thread). Q: Any other limitations? A: Because of ptrace, any process can be supervised by only one process at a time. So if you run systemtap on a user process, you won't be able to run gdb or another systemtap session on it. Q: What about probing the kernel and user space together? A: Maybe this scheme would work if kernel-space systemtap probes run concurrently, and arrange to share systemtap globals with userspace somehow (mmap?). Shared variables like this would likely cause many more locking timeouts (=> skipped probes) than now. There are also additional security concerns. Q: What about probing shared libraries? A: Because of the way ptrace works, we'd have to turn these into process-level probes, including probes that just sit around monitoring the threads and all their children to dlopen/mmap the named libraries. Q: Is it worth it to try? Is there a better way? A: You tell me. - FChE