From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28148 invoked by alias); 2 Oct 2006 15:44:15 -0000 Received: (qmail 28136 invoked by uid 22791); 2 Oct 2006 15:44:15 -0000 X-Spam-Status: No, hits=-2.4 required=5.0 tests=AWL,BAYES_00,FORGED_RCVD_HELO X-Spam-Check-By: sourceware.org Received: from tomts43-srv.bellnexxia.net (HELO tomts43-srv.bellnexxia.net) (209.226.175.110) by sourceware.org (qpsmtpd/0.31) with ESMTP; Mon, 02 Oct 2006 15:44:11 +0000 Received: from krystal.dyndns.org ([65.95.39.115]) by tomts43-srv.bellnexxia.net (InterMail vM.5.01.06.13 201-253-122-130-113-20050324) with ESMTP id <20061002154407.KNJ19825.tomts43-srv.bellnexxia.net@krystal.dyndns.org> for ; Mon, 2 Oct 2006 11:44:07 -0400 Received: from localhost (localhost [127.0.0.1]) (uid 1000) by krystal.dyndns.org with local; Mon, 02 Oct 2006 11:38:49 -0400 id 001E685C.45213289.00003E74 Date: Mon, 02 Oct 2006 15:44:00 -0000 From: Mathieu Desnoyers To: "Jose R. Santos" Cc: Martin Bligh , "Frank Ch. Eigler" , Masami Hiramatsu , prasanna@in.ibm.com, Andrew Morton , Ingo Molnar , Paul Mundt , linux-kernel , Jes Sorensen , Tom Zanussi , Richard J Moore , Michel Dagenais , Christoph Hellwig , Greg Kroah-Hartman , Thomas Gleixner , William Cohen , ltt-dev@shafik.org, systemtap@sources.redhat.com, Alan Cox , Jeremy Fitzhardinge , Karim Yaghmour , Pavel Machek , Joe Perches , "Randy.Dunlap" Subject: Re: Performance analysis of Linux Kernel Markers 0.20 for 2.6.17 Message-ID: <20061002153849.GA19568@Krystal> References: <20060930180157.GA25761@Krystal> <45212F1E.3080409@us.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline In-Reply-To: <45212F1E.3080409@us.ibm.com> X-Editor: vi X-Info: http://krystal.dyndns.org:8080 X-Operating-System: Linux/2.4.32-grsec (i686) X-Uptime: 11:33:01 up 40 days, 12:41, 4 users, load average: 0.82, 0.71, 0.54 User-Agent: Mutt/1.5.13 (2006-08-11) X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2006-q4/txt/msg00009.txt.bz2 Hi Jose, * Jose R. Santos (jrs@us.ibm.com) wrote: > > The problem now is how do we define "high event rate". This is > something that is highly dependent on the workload being run as well as > the system configuration for such workload. There are a lot of places > in the kernel that can be turned into high event rates with with the > right workload even though the may not represent 99% of most user cases. > > I would guess that anything above 500 event/s per-CPU on several > realistic workloads is a good place to start. > Yes, it seems like a good starting point. But besides the event rate, just having the most widely used events marked in the code should also be the target. The markup mechanism serves two purposes : 1 - identify important events in a way that follows code change. 2 - speed up instrumentation. > > >On the macro-benchmark side, no significant difference in performance has > >been > >found between the vanilla kernel and a kernel "marked" with the standard > >LTTng > >instrumentation. > > > > Out of curiosity, how many cycles does it take to process a complete > LTTng event up until the point were it has been completely stored into > the trace buffer. Since this should take a lot more than 55.74 cycles, > it would be interesting to know at what event rate would a static marker > stop showing as big of a performance advantage compared to dynamic probing. > In my OLS paper, I pointed out that, in its current state, LTTng took about 278 cycles on the same Pentium 4. I think I could lower that by implementing per-cpu atomic operations (removing the LOCK prefix, as the data is not shared between the CPUs; the atomic operations are only useful to protect from higher priority execution contexts on the same CPU). Regards, Mathieu OpenPGP public key: http://krystal.dyndns.org:8080/key/compudj.gpg Key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68