From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 40687 invoked by alias); 11 Aug 2016 16:18:52 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 40672 invoked by uid 89); 11 Aug 2016 16:18:51 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.4 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=periodic, 997, H*MI:sk:42a5bad, H*f:sk:42a5bad X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 11 Aug 2016 16:18:48 +0000 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6AEB583F45; Thu, 11 Aug 2016 16:18:46 +0000 (UTC) Received: from fche.csb (vpn-55-67.rdu2.redhat.com [10.10.55.67]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u7BGIjtw029506; Thu, 11 Aug 2016 12:18:46 -0400 Received: by fche.csb (Postfix, from userid 2569) id 4814E589C6; Thu, 11 Aug 2016 12:18:45 -0400 (EDT) Date: Thu, 11 Aug 2016 16:18:00 -0000 From: "Frank Ch. Eigler" To: Avi Kivity Cc: Mark Wielaard , David Smith , systemtap@sourceware.org Subject: Re: Some newbie questions Message-ID: <20160811161845.GC12190@redhat.com> References: <20160809112501.GK2508@stream> <91592a2a-7373-6a27-413c-cf4aca70cb76@scylladb.com> <20160810164742.GF13748@redhat.com> <382185d7-86d6-45d0-1855-d7e3c1639083@scylladb.com> <42a5bad6-c094-cb7a-6f89-a592367ca30b@scylladb.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42a5bad6-c094-cb7a-6f89-a592367ca30b@scylladb.com> User-Agent: Mutt/1.4.2.2i X-SW-Source: 2016-q3/txt/msg00158.txt.bz2 Hi - avi wrote: > [...] > >>(Have you tried a stap script that merely traps all the same the > >>function calls, and has empty probe handlers?) > >I can try it. > > It was actually pretty bad with empty handlers: > > # > 66.25% scylla [kernel.kallsyms] [k] > _raw_spin_lock > | > ---_raw_spin_lock > | > |--49.95%-- 0x62ab > | syscall_trace_leave > [...] > |--49.46%-- 0x619b > | syscall_trace_enter_phase2 > [...] > > I don't have any system call probes. Just two empty static probes, and > a timer.profile handler. It would sure be nice if perf gave you an explanation of those 0x62ab bits. Maybe use dwarf unwind based callgraph collection? The stap runtime does sometimes create probes for internal purposes, such as trapping exec/mmap operations (so probes on new programs can be activated). It is conceivable that this is the source of the spinlock activity, but such a high rate doesn't make sense. > Will timer.profile work with dyninst? Profile timers, not yet, but normal periodic ones like timer.hz(997) yes. Unfortunately backtracing is also not yet implemented there. - FChE