From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 7269 invoked by alias); 19 Oct 2006 22:56:46 -0000 Received: (qmail 7262 invoked by uid 22791); 19 Oct 2006 22:56:46 -0000 X-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,BAYES_00 X-Spam-Check-By: sourceware.org Received: from mga03.intel.com (HELO mga03.intel.com) (143.182.124.21) by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 19 Oct 2006 22:56:43 +0000 Received: from azsmga001.ch.intel.com ([10.2.17.19]) by mga03.intel.com with ESMTP; 19 Oct 2006 15:56:41 -0700 Received: from scsmsx331.sc.intel.com (HELO scsmsx331.amr.corp.intel.com) ([10.3.90.4]) by azsmga001.ch.intel.com with ESMTP; 19 Oct 2006 15:56:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: i="4.09,330,1157353200"; d="scan'208"; a="133448747:sNHT19994709" Received: from scsmsx413.amr.corp.intel.com ([10.3.90.32]) by scsmsx331.amr.corp.intel.com with Microsoft SMTPSVC(6.0.3790.211); Thu, 19 Oct 2006 15:56:40 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: user mode backtrace Date: Thu, 19 Oct 2006 22:56:00 -0000 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: user mode backtrace Thread-Index: Acbzv8wujovj0MvLTEa9L6r0cxVxLgABa1BA From: "Stone, Joshua I" To: Cc: "SystemTap" X-OriginalArrivalTime: 19 Oct 2006 22:56:40.0953 (UTC) FILETIME=[D71FAE90:01C6F3D1] X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2006-q4/txt/msg00198.txt.bz2 On Thursday, October 19, 2006 1:47 PM, David Boreham wrote: > I'd like to get a stack trace for the process that made the > system call I'm probing (I'm looking at filesystem access > typically, so reads/writes/syncs etc). The systemtap backtrace > function appears to only get the kernel mode stack which > is not much use to me. I was wondering if anyone had > discovered a good solution to this problem already ? > I was thinking perhaps I could invoke pstack (gdb) > on the current pid/tid. But I'm worried that doing so > might deadlock since the process is inside a system > call. >=20 > I'm looking at a very large application that beats up on > the filesystem, in case you're wondering why I want to do > this. It's so large that nobody is quite sure what code > access which files, when and why. >=20 > Thanks. Deadlock issues aside, there's not really a way for you to invoke a process (like pstack) from within a SystemTap script. You could run a separate user program or script to do this for you though, and then you just need to coordinate with SystemTap. Such a method might look like this: ----------------------------------------------- /* Main test driver */ ----------------------------------------------- pid =3D fork_ptraceme_exec("myapp"); // start the app paused stappid =3D fork_exec("stap myscript.stp -x " + pid); // start systemtap ptrace(DETACH, pid, ...); // let the app run while(pid =3D=3D waitpid(pid, stat, 0)) { if (WIFSTOPPED(stat)) { // app is stopped system("pstack " + pid); // dump the stack kill(pid, SIGCONT); // continue the app } else if (WIFEXITED(stat)) { break; } } kill(stappid, SIGINT); // tell systemtap to stop waitpid(stappid, ...); ----------------------------------------------- ----------------------------------------------- /* SystemTap script: myscript.stp */ ----------------------------------------------- probe syscall.read { if (target() !=3D tid()) next; /* log some stuff: filename, etc. */ } probe syscall.read.return { if (target() !=3D tid()) next; send_stop() // do this on return to avoid EINTR } function send_stop %{ send_sig(SIGSTOP, current, 1); %} ----------------------------------------------- This is all pretty rough, and I haven't actually tried it, so who knows if it will actually work. Of course at the end of the day, this is just a convoluted strace with a stack printout. You could probably do the same thing by hacking gdb's backtrace function into strace. But this SystemTap method would also let you do probe other things besides just system calls... If anyone gets this working I would LOVE to hear about it... :) Josh