Tapset for probing IRQs, workqueues, etc

Tapset for probing IRQs, workqueues, etc

[Prerna Saxena]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

Hi,

Attached is a tracepoint based tapset for probing IRQ handlers, 
softirqs, and workqueue operations.
Of these, softirqs and irq handler based probes cannot have kprobe 
equivalents, 'cos of the kprobe blacklist. And the position of 
tracepoints for workqueues makes it difficult to define a kprobe 
fallback for most of these probes.

Looking fwd to feedback to improve it..

-- 
Prerna Saxena

Linux Technology Centre,
IBM Systems and Technology Lab,
Bangalore, India

[-- Attachment #2: irq-patch --]
[-- Type: text/plain, Size: 7198 bytes --]

Index: stap-git-Oct-01/tapset/irq.stp
===================================================================
--- /dev/null
+++ stap-git-Oct-01/tapset/irq.stp
@@ -0,0 +1,160 @@
+/*
+ * Tracepoint based tapset for IRQs, Workqueues, etc
+ *
+ */
+// Probes for workqueues.
+
+/* probe workqueue.create : probes creation of a new workqueue
+ *	Variables :
+ *		wq_thread : task_struct of the workqueue thread.
+ *		cpu	  : cpu for which the worker thread is created.
+ */
+probe workqueue.create = kernel.trace("workqueue_creation")
+{
+	wq_thread = $wq_thread
+	cpu = $cpu
+}
+
+/* probe workqueue.insert : probes queuing of work on a workqueue
+ *	Variables :
+ *		wq_thread : 	task_struct of the workqueue thread.
+ *		work	  :	work_struct* being queued.
+ *		work_func :	pointer to handler func.
+ */
+probe workqueue.insert = kernel.trace("workqueue_insertion")
+{
+	wq_thread = $wq_thread
+	work = $work
+	work_func = @cast(work, "work_struct", "kernel")->func
+}
+
+/* probe workqueue.execute : probes execution of deferred work.
+ *	Variables :
+ *		wq_thread : 	task_struct of the workqueue thread.
+ *		work	  :	work_struct* being executed.
+ *		work_func :	pointer to handler func.
+ */
+probe workqueue.execute = kernel.trace("workqueue_execution")
+{
+	wq_thread = $wq_thread
+	work = $work
+	work_func = @cast(work, "work_struct", "kernel")->func
+}
+
+/* probe workqueue.destroy : probes destruction of each worker thread of each cpu for a workqueue.
+ *	Variables :
+ *		wq_thread : 	task_struct of the workqueue thread.
+ */
+probe workqueue.destroy = kernel.trace("workqueue_destruction")
+{
+	wq_thread = $wq_thread
+}
+
+// Probes for IRQ handlers.
+
+/* probe irq_handler.entry : Fires prior to execution of interrupt handler.
+ * 	variables :
+ *		irq 	: irq number.
+ * 		action	: struct irqaction* for this interrupt num.
+ *		handler	: interrupt handler function.
+ *		flags	:
+ *			IRQF_DISABLED [0x00000020]	: keep irqs disabled when calling action handler.
+ *			IRQF_SAMPLE_RANDOM [0x00000040]	: irq is used to feed the random generator
+ *			IRQF_SHARED [0x00000080]	: allow sharing the irq among several devices
+ *			IRQF_PROBE_SHARED [0x00000100]	: set by callers when they expect sharing mismatches to occur
+ *			IRQF_TIMER [0x00000200]		: Flag to mark this interrupt as timer interrupt
+ *			IRQF_PERCPU [0x00000400]	: Interrupt is per cpu.
+ *			IRQF_NOBALANCING [0x00000800]	: Flag to exclude this interrupt from irq balancing
+ *			IRQF_IRQPOLL [0x00001000]	: Interrupt is used for polling.
+ *			IRQF_ONESHOT [0x00002000]	: Interrupt is not reenabled after the hardirq handler finished.
+ *		dev_name: name of device.
+ *		dev_id	: Cookie to identify device.
+ *		next_irqaction	: pointer to next irqaction for shared interrupts.
+ *		dir	: pointer to the proc/irq/NN/name entry
+ *		thread_fn: interupt handler function for threaded interrupts.
+ *		thread	: thread pointer for threaded interrupts.
+ *		thread_flags: Flags related to thread.
+ */
+probe irq_handler.entry = kernel.trace("irq_handler_entry")
+{
+	irq = $irq
+	action = $action
+	handler = @cast($action, "irqaction", "kernel")->handler
+	flags = @cast($action, "irqaction", "kernel")->flags
+	dev_name = @cast($action, "irqaction", "kernel")->name
+	dev_id = @cast($action, "irqaction", "kernel")->dev_id
+	next_irqaction = @cast($action, "irqaction", "kernel")->next
+	dir = @cast($action, "irqaction", "kernel")->dir
+	thread_fn = @cast($action, "irqaction", "kernel")->thread_fn
+	thread = @cast($action, "irqaction", "kernel")->thread
+	thread_flags = @cast($action, "irqaction", "kernel")->thread_flags
+}
+
+/* probe irq_handler.exit : Fires just after execution of interrupt handler.
+ * 	variables :
+ *		irq 	: interrupt number
+ * 		action	: struct irqaction*
+ * 		ret	: return value of the handler
+ *		handler	: interrupt handler function that was executed.
+ *		flags	:
+ *			IRQF_DISABLED	[0x00000020]	: keep irqs disabled when calling action handler.
+ *			IRQF_SAMPLE_RANDOM [0x00000040]	: irq is used to feed the random generator
+ *			IRQF_SHARED [0x00000080]	: allow sharing the irq among several devices
+ *			IRQF_PROBE_SHARED [0x00000100]	: set by callers when they expect sharing mismatches to occur
+ *			IRQF_TIMER [0x00000200]		: Flag to mark this interrupt as timer interrupt
+ *			IRQF_PERCPU [0x00000400]	: Interrupt is per cpu.
+ *			IRQF_NOBALANCING [0x00000800]	: Flag to exclude this interrupt from irq balancing
+ *			IRQF_IRQPOLL [0x00001000]	: Interrupt is used for polling.
+ *			IRQF_ONESHOT [0x00002000]	: Interrupt is not reenabled after the hardirq handler finished.
+ *		dev_name: name of device.
+ *		dev_id	: Cookie to identify device.
+ *		next_irqaction	: pointer to next irqaction for shared interrupts.
+ *		dir	: pointer to the proc/irq/NN/name entry
+ *		thread_fn: interupt handler function for threaded interrupts.
+ *		thread	: thread pointer for threaded interrupts.
+ *		thread_flags: Flags related to thread.
+ */
+probe irq_handler.exit = kernel.trace("irq_handler_exit")
+{
+	irq = $irq
+	action = $action
+	ret = $ret
+	handler = @cast($action, "irqaction", "kernel")->handler
+	flags = @cast($action, "irqaction", "kernel")->flags
+	dev_name = @cast($action, "irqaction", "kernel")->name
+	dev_id = @cast($action, "irqaction", "kernel")->dev_id
+	next_irqaction = @cast($action, "irqaction", "kernel")->next
+	dir = @cast($action, "irqaction", "kernel")->dir
+	thread_fn = @cast($action, "irqaction", "kernel")->thread_fn
+	thread = @cast($action, "irqaction", "kernel")->thread
+	thread_flags = @cast($action, "irqaction", "kernel")->thread_flags
+}
+
+// Softirq based probes.
+/* probe softirq.entry 	: triggered just before executing handler for a pending
+ *			  softirq
+ *	variables :
+ *		h	: struct softirq_action* for current pending softirq.
+ *		vec	: softirq_action vector.
+ *		action	: pointer to softirq handler just about to execute.
+ */
+probe softirq.entry = kernel.trace("softirq_entry")
+{
+	h = $h
+	vec = $vec
+	action = @cast($h, "softirq_action", "kernel")->action
+}
+
+/* probe softirq.exit 	: triggered just after executing handler for a pending
+ *			  softirq.
+ *	variables :
+ *		h	: struct softirq_action* for just executed softirq.
+ *		vec	: softirq_action vector.
+ *		action	: pointer to softirq handler that just finished execution.
+ */
+probe softirq.exit = kernel.trace("softirq_exit")
+{
+	h = $h
+	vec = $vec
+	action = @cast($h, "softirq_action", "kernel")->action
+}
Index: stap-git-Oct-01/tapset/aux_syscalls.stp
===================================================================
--- stap-git-Oct-01.orig/tapset/aux_syscalls.stp
+++ stap-git-Oct-01/tapset/aux_syscalls.stp
@@ -1831,3 +1831,24 @@ function _struct_sigaction32_u:string(ua
     }
 #endif
 %}
+
+%{
+#include <linux/interrupt.h>
+static const _stp_val_array const _stp_irq_list[] = {
+	V(IRQF_DISABLED),
+	V(IRQF_SAMPLE_RANDOM),
+	V(IRQF_SHARED),
+	V(IRQF_PROBE_SHARED),
+	V(IRQF_TIMER),
+	V(IRQF_PERCPU),
+	V(IRQF_NOBALANCING),
+	V(IRQF_IRQPOLL),
+	V(IRQF_ONESHOT),
+	{0, NULL}
+};
+%}
+
+function _irqflags_str:string(f:long)
+%{ /* pure */
+	_stp_lookup_or_str(_stp_irq_list, THIS->f, THIS->__retvalue, MAXSTRINGLEN);
+%}

Re: Tapset for probing IRQs, workqueues, etc

[David Smith]

On 10/23/2009 12:36 PM, Prerna Saxena wrote:
> Hi,
> 
> Attached is a tracepoint based tapset for probing IRQ handlers,
> softirqs, and workqueue operations.
> Of these, softirqs and irq handler based probes cannot have kprobe
> equivalents, 'cos of the kprobe blacklist. And the position of
> tracepoints for workqueues makes it difficult to define a kprobe
> fallback for most of these probes.
> 
> Looking fwd to feedback to improve it..

I took a short look at your tapset.  In general, it looks fine.

I do have one question.  You added a function called '_irqflags_str' to
aux_syscalls.stp, but I couldn't find a caller of that new function in
your tapset.  It looks like you might have meant to call it for the
'flags' variables in irq_handler.entry and irq_handler.exit.

It would be really nice if your new tapset automatically generated its
own documentation.  I don't think it would be much additional work over
the comments you have now.  See the "Documentation" section of the
"Tapset Developer's Guide"
<http://sources.redhat.com/git/?p=systemtap.git;a=blob_plain;f=tapset/DEVGUIDE>
for more details.

It would also be great if your tapset had a simple testcase that made
sure everything compiled.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Tapset for probing IRQs, workqueues, etc

[Josh Stone]

Hi Prerna,

On 10/23/2009 10:36 AM, Prerna Saxena wrote:
> +	work = $work
> +	work_func = @cast(work, "work_struct", "kernel")->func

Why do you need @cast in this tapset?  It should work just fine to say
"work_funk = $work->func".  Apart from brevity, this is also better for
not requiring the kernel debuginfo packages.

Josh

Re: Tapset for probing IRQs, workqueues, etc

[Prerna Saxena]

[-- Attachment #1: Type: text/plain, Size: 2016 bytes --]

Hi ,
Posting a new version of the IRQ tapset.

On 10/27/2009 01:42 AM, David Smith wrote:

>
> I took a short look at your tapset.  In general, it looks fine.
>
> I do have one question.  You added a function called '_irqflags_str' to
> aux_syscalls.stp, but I couldn't find a caller of that new function in
> your tapset.  It looks like you might have meant to call it for the
> 'flags' variables in irq_handler.entry and irq_handler.exit.

I have not included the flags string on purpose for this probe point, 
cos I think that one might need to look up the flags in specific 
debugging scenarios only. It would not be required in most commonly used 
cases, such as the attached example. However, I have documented the 
function both inline in the tapset documentation and also in the man 
pages, so that a script developer wanting to display a formatted string 
can call the function when needed.

> It would be really nice if your new tapset automatically generated its
> own documentation.  I don't think it would be much additional work over
> the comments you have now.  See the "Documentation" section of the
> "Tapset Developer's Guide"
> <http://sources.redhat.com/git/?p=systemtap.git;a=blob_plain;f=tapset/DEVGUIDE>
> for more details.
>

Thanks for the suggestion, I've tried to make the tapset follow the 
guidelines specified above.

> It would also be great if your tapset had a simple testcase that made
> sure everything compiled.
>

The present version includes :
1. IRQ tapset
2. testcase for checking if probes resolve & build fine.
3. Example script for profiling interrupts per device in a given time 
interval.
4. man pages for the tapset.
5. Updates to tapsets.tmpl so that documentation is auto-generated for 
the tapset.

Josh,
    Thanks for pointing me to the redundancy of using @cast. I've 
replaced it with direct resolution.

Hope I've covered things needed.. Looking fwd to suggestions !

-- 
Prerna Saxena

Linux Technology Centre,
IBM Systems and Technology Lab,
Bangalore, India

[-- Attachment #2: irq-patch --]
[-- Type: text/plain, Size: 14244 bytes --]

Index: stap-git-Oct-01/tapset/irq.stp
===================================================================
--- /dev/null
+++ stap-git-Oct-01/tapset/irq.stp
@@ -0,0 +1,172 @@
+/*
+ *      Copyright (C) 2009 IBM Corp.
+ *      This file is part of systemtap, and is free software.  You can
+ *      redistribute it and/or modify it under the terms of the GNU General
+ *      Public License (GPL); either version 2, or (at your option) any
+ *      later version.
+ *
+ *      Version 1.0     prerna@linux.vnet.ibm.com     2009-10-28
+ *
+ * Tracepoint based tapset for IRQs, Workqueues, etc
+ *
+ */
+// Probes for workqueues.
+
+/**
+ * probe workqueue.create : probes creation of a new workqueue
+ * @wq_thread : task_struct of the workqueue thread.
+ * @cpu	  : cpu for which the worker thread is created.
+ */
+probe workqueue.create = kernel.trace("workqueue_creation")
+{
+	wq_thread = $wq_thread
+	cpu = $cpu
+}
+
+/**
+ * probe workqueue.insert : probes queuing of work on a workqueue
+ * @wq_thread : task_struct of the workqueue thread.
+ * @work : work_struct* being queued.
+ * @work_func :	pointer to handler func.
+ */
+probe workqueue.insert = kernel.trace("workqueue_insertion")
+{
+	wq_thread = $wq_thread
+	work = $work
+	work_func = $work->func
+}
+
+/**
+ * probe workqueue.execute : probes execution of deferred work.
+ * @wq_thread :	task_struct of the workqueue thread.
+ * @work : work_struct* being executed.
+ * @work_func :	pointer to handler func.
+ */
+probe workqueue.execute = kernel.trace("workqueue_execution")
+{
+	wq_thread = $wq_thread
+	work = $work
+	work_func = $work->func
+}
+
+/**
+ * probe workqueue.destroy : probes destruction of each worker thread of each cpu for a workqueue.
+ * @wq_thread : task_struct of the workqueue thread.
+ */
+probe workqueue.destroy = kernel.trace("workqueue_destruction")
+{
+	wq_thread = $wq_thread
+}
+
+// Probes for IRQ handlers.
+
+/**
+ * probe irq_handler.entry : Fires prior to execution of interrupt handler.
+ * @irq	: irq number.
+ * @action : struct irqaction* for this interrupt num.
+ * @handler : interrupt handler function.
+ * @flags : Flags for IRQ handler
+ *			IRQF_DISABLED [0x00000020]	: keep irqs disabled when calling action handler.
+ *			IRQF_SAMPLE_RANDOM [0x00000040]	: irq is used to feed the random generator
+ *			IRQF_SHARED [0x00000080]	: allow sharing the irq among several devices
+ *			IRQF_PROBE_SHARED [0x00000100]	: set by callers when they expect sharing mismatches to occur
+ *			IRQF_TIMER [0x00000200]		: Flag to mark this interrupt as timer interrupt
+ *			IRQF_PERCPU [0x00000400]	: Interrupt is per cpu.
+ *			IRQF_NOBALANCING [0x00000800]	: Flag to exclude this interrupt from irq balancing
+ *			IRQF_IRQPOLL [0x00001000]	: Interrupt is used for polling.
+ *			IRQF_ONESHOT [0x00002000]	: Interrupt is not reenabled after the hardirq handler finished.
+ *			( Use function irqflags_str() to obtain a symbolic
+ * 			  string representation of IRQ flags )
+ * @dev_name : name of device.
+ * @dev_id : Cookie to identify device.
+ * @next_irqaction : pointer to next irqaction for shared interrupts.
+ * @dir	: pointer to the proc/irq/NN/name entry
+ * @thread_fn : interupt handler function for threaded interrupts.
+ * @thread : thread pointer for threaded interrupts.
+ * @thread_flags : Flags related to thread.
+ */
+probe irq_handler.entry = kernel.trace("irq_handler_entry")
+{
+	irq = $irq
+	action = $action
+	handler = $action->handler
+	flags = $action->flags
+	dev_name = $action->name
+	dev_id = $action->dev_id
+	next_irqaction = $action->next
+	dir = $action->dir
+	thread_fn = $action->thread_fn
+	thread = $action->thread
+	thread_flags = $action->thread_flags
+}
+
+/**
+ * probe irq_handler.exit : Fires just after execution of interrupt handler.
+ * @irq : interrupt number
+ * @action : struct irqaction*
+ * @ret	: return value of the handler
+ * @handler : interrupt handler function that was executed.
+ * @flags : flags for IRQ handler
+ *			IRQF_DISABLED	[0x00000020]	: keep irqs disabled when calling action handler.
+ *			IRQF_SAMPLE_RANDOM [0x00000040]	: irq is used to feed the random generator
+ *			IRQF_SHARED [0x00000080]	: allow sharing the irq among several devices
+ *			IRQF_PROBE_SHARED [0x00000100]	: set by callers when they expect sharing mismatches to occur
+ *			IRQF_TIMER [0x00000200]		: Flag to mark this interrupt as timer interrupt
+ *			IRQF_PERCPU [0x00000400]	: Interrupt is per cpu.
+ *			IRQF_NOBALANCING [0x00000800]	: Flag to exclude this interrupt from irq balancing
+ *			IRQF_IRQPOLL [0x00001000]	: Interrupt is used for polling.
+ *			IRQF_ONESHOT [0x00002000]	: Interrupt is not reenabled after the hardirq handler finished.
+ *			( Use function irqflags_str() to obtain a symbolic
+ * 			  string representation of IRQ flags )
+ * @dev_name : name of device.
+ * @dev_id : Cookie to identify device.
+ * @next_irqaction : pointer to next irqaction for shared interrupts.
+ * @dir	: pointer to the proc/irq/NN/name entry
+ * @thread_fn : interupt handler function for threaded interrupts.
+ * @thread : thread pointer for threaded interrupts.
+ * @thread_flags : Flags related to thread.
+ */
+probe irq_handler.exit = kernel.trace("irq_handler_exit")
+{
+	irq = $irq
+	action = $action
+	ret = $ret
+	handler = $action->handler
+	flags = $action->flags
+	dev_name = $action->name
+	dev_id = $action->dev_id
+	next_irqaction = $action->next
+	dir = $action->dir
+	thread_fn = $action->thread_fn
+	thread = $action->thread
+	thread_flags = $action->thread_flags
+}
+
+// Softirq based probes.
+/**
+ * probe softirq.entry 	: triggered just before executing handler
+ *			  for a pending softirq.
+ * @h : struct softirq_action* for current pending softirq.
+ * @vec	: softirq_action vector.
+ * @action : pointer to softirq handler just about to execute.
+ */
+probe softirq.entry = kernel.trace("softirq_entry")
+{
+	h = $h
+	vec = $vec
+	action = $h->action
+}
+
+/**
+ * probe softirq.exit 	: triggered just after executing handler for a pending
+ *			  softirq.
+ * @h : struct softirq_action* for just executed softirq.
+ * @vec	: softirq_action vector.
+ * @action : pointer to softirq handler that just finished execution.
+ */
+probe softirq.exit = kernel.trace("softirq_exit")
+{
+	h = $h
+	vec = $vec
+	action = $h->action
+}
Index: stap-git-Oct-01/tapset/aux_syscalls.stp
===================================================================
--- stap-git-Oct-01.orig/tapset/aux_syscalls.stp
+++ stap-git-Oct-01/tapset/aux_syscalls.stp
@@ -1831,3 +1831,29 @@ function _struct_sigaction32_u:string(ua
     }
 #endif
 %}
+
+/*
+ * function irqflags_str :
+ *	Return the symbolic string representation of the IRQ flags.
+ */
+
+%{
+#include <linux/interrupt.h>
+static const _stp_val_array const _stp_irq_list[] = {
+	V(IRQF_DISABLED),
+	V(IRQF_SAMPLE_RANDOM),
+	V(IRQF_SHARED),
+	V(IRQF_PROBE_SHARED),
+	V(IRQF_TIMER),
+	V(IRQF_PERCPU),
+	V(IRQF_NOBALANCING),
+	V(IRQF_IRQPOLL),
+	V(IRQF_ONESHOT),
+	{0, NULL}
+};
+%}
+
+function irqflags_str:string(f:long)
+%{ /* pure */
+	_stp_lookup_or_str(_stp_irq_list, THIS->f, THIS->__retvalue, MAXSTRINGLEN);
+%}
Index: stap-git-Oct-01/man/stapprobes.irq.3stap.in
===================================================================
--- /dev/null
+++ stap-git-Oct-01/man/stapprobes.irq.3stap.in
@@ -0,0 +1,151 @@
+.\" -*- nroff -*-
+.TH STAPPROBES.SNMP 3stap @DATE@ "IBM"
+.SH NAME
+stapprobes.irq \- Systemtap probes for IRQ, workqueue,etc
+
+.\" macros
+.de SAMPLE
+.br
+.RS
+.nf
+.nh
+..
+.de ESAMPLE
+.hy
+.fi
+.RE
+..
+
+.SH DESCRIPTION
+
+Probe points for probing irq handler execution, softirqs, workqueues,etc
+
+.P
+.TP
+.B workqueue.create
+probes creation of a new workqueue
+
+.B Arguments:
+.I wq_thread
+   task_struct of the workqueue thread.
+.I cpu
+   cpu for which the worker thread is created.
+
+.P
+.TP
+.B workqueue.insert
+probes queuing of work on a workqueue.
+
+.B Arguments:
+.I wq_thread
+   task_struct of the workqueue thread.
+.I work
+   work_struct* being executed.
+.I work_func
+   pointer to handler func.
+
+.P
+.TP
+.B workqueue.execute
+probes execution of deferred work.
+
+.B Arguments:
+.I wq_thread
+   task_struct of the workqueue thread.
+.I work
+   work_struct* being executed.
+.I work_func
+   pointer to handler func.
+
+.P
+.TP
+.B workqueue.destroy
+probes destruction of each worker thread of each cpu for a workqueue.
+
+.B Arguments:
+.I wq_thread
+  task_struct of the workqueue thread.
+
+.P
+.TP
+.B irq_handler.entry
+Fires prior to execution of interrupt handler.
+.B Arguments:
+.I irq
+   irq number
+.I action
+   struct irqaction* for this interrupt number
+.I handler
+   interrupt handler function
+.I flags
+   flags for this irq. A formatted string of flags can be obtained by passing this argument to the irqflags_str() function.
+.I dev_name
+   name of device
+.I dev_id
+   cookie to identify device
+.I next_irqaction
+   pointer to next irqaction for shared interrupts
+.I dir
+   pointer to the /proc/irq/NN/name entry
+.I thread_fn
+   interrupt handler function for threaded interrupts
+.I thread
+   thread pointer for threaded interrupts
+.I thread_flags
+   flags related to thread
+
+.P
+.TP
+.B irq_handler.exit
+Fires post execution of interrupt handler.
+.B Arguments:
+.I irq
+   irq number
+.I action
+   struct irqaction* for this interrupt number
+.I ret
+   return value from interrupt handler that just executed.
+.I handler
+   interrupt handler function
+.I flags
+   flags for this irq. A formatted string of flags can be obtained by passing this argument to the irqflags_str() function.
+.I dev_name
+   name of device
+.I dev_id
+   cookie to identify device
+.I next_irqaction
+   pointer to next irqaction for shared interrupts
+.I dir
+   pointer to the /proc/irq/NN/name entry
+.I thread_fn
+   interrupt handler function for threaded interrupts
+.I thread
+   thread pointer for threaded interrupts
+.I thread_flags
+   flags related to thread
+
+.P
+.TP
+.B softirq.entry
+triggered just before executing handler for a pending softirq
+.I h
+   struct softirq* for current pending softirq.
+.I vec
+   softirq_action vector
+.I action
+   pointer to softirq handler just about to execute.
+
+.P
+.TP
+.B softirq.exit
+triggered just after executing handler for a pending softirq
+.I h
+   struct softirq* for just executed softirq.
+.I vec
+   softirq_action vector
+.I action
+   pointer to softirq handler that just finished execution.
+
+.SH SEE ALSO
+.IR stap (1),
+.IR stapprobes (3stap)
Index: stap-git-Oct-01/testsuite/buildok/irq.stp
===================================================================
--- /dev/null
+++ stap-git-Oct-01/testsuite/buildok/irq.stp
@@ -0,0 +1,7 @@
+#! stap -p4
+
+// Tests if all probes in irq tapset are resolvable
+
+probe workqueue.* {}
+probe irq_handler.* {}
+probe softirq.* {}
Index: stap-git-Oct-01/testsuite/systemtap.examples/interrupt/interrupts-by-dev.stp
===================================================================
--- /dev/null
+++ stap-git-Oct-01/testsuite/systemtap.examples/interrupt/interrupts-by-dev.stp
@@ -0,0 +1,31 @@
+#! /usr/bin/env stap
+/*
+ *      Copyright (C) 2009 IBM Corp.
+ *      This file is part of systemtap, and is free software.  You can
+ *      redistribute it and/or modify it under the terms of the GNU General
+ *      Public License (GPL); either version 2, or (at your option) any
+ *      later version.
+ *
+ *      Version 1.0     prerna@linux.vnet.ibm.com     2009-10-28
+ *
+ *     Name:
+ *     interrupts-by-dev.stp
+ *
+ *     Description:
+ *     Script to profile interrupts received by each device per 100 ms.
+ *
+ *
+ */
+
+global devices
+
+probe irq_handler.entry {
+        devices[dev_name]++;
+}
+
+probe timer.ms(100) {
+        printf("\t  DEVICE \t NUMBER OF INTERRUPTS \n");
+        foreach ( devname in devices )
+                printf(" %20s :  %5d\n",kernel_string(devname),devices[devname]);
+        delete devices
+}
Index: stap-git-Oct-01/doc/SystemTap_Tapset_Reference/tapsets.tmpl
===================================================================
--- stap-git-Oct-01.orig/doc/SystemTap_Tapset_Reference/tapsets.tmpl
+++ stap-git-Oct-01/doc/SystemTap_Tapset_Reference/tapsets.tmpl
@@ -155,6 +155,15 @@
 !Itapset/ioscheduler.stp
   </chapter>
 
+  <chapter id="irq.stp">
+    <title>IRQ & Workqueue Tapset</title>
+    <para>
+      This family of probe points is used to probe IRQs, Workqueues, etc
+      It contains the following probe points:
+    </para>
+!Itapset/irq.stp
+  </chapter>
+
   <chapter id="scsi.stp">
     <title>SCSI Tapset</title>
     <para>
Index: stap-git-Oct-01/configure.ac
===================================================================
--- stap-git-Oct-01.orig/configure.ac
+++ stap-git-Oct-01/configure.ac
@@ -625,7 +625,7 @@ dnl Don't use this directly (when not gi
 AC_DEFINE_UNQUOTED(STAP_PREFIX, "$prefix", [configure prefix location])
 
 AC_CONFIG_HEADERS([config.h:config.in])
-AC_CONFIG_FILES(Makefile doc/Makefile doc/SystemTap_Tapset_Reference/Makefile grapher/Makefile stap.1 stapprobes.3stap stapfuncs.3stap stapvars.3stap stapex.3stap staprun.8 stap-server.8 man/stapprobes.iosched.3stap man/stapprobes.netdev.3stap man/stapprobes.nfs.3stap man/stapprobes.nfsd.3stap man/stapprobes.pagefault.3stap man/stapprobes.kprocess.3stap man/stapprobes.rpc.3stap man/stapprobes.scsi.3stap man/stapprobes.signal.3stap man/stapprobes.socket.3stap man/stapprobes.tcp.3stap man/stapprobes.udp.3stap man/stapprobes.snmp.3stap initscript/systemtap)
+AC_CONFIG_FILES(Makefile doc/Makefile doc/SystemTap_Tapset_Reference/Makefile grapher/Makefile stap.1 stapprobes.3stap stapfuncs.3stap stapvars.3stap stapex.3stap staprun.8 stap-server.8 man/stapprobes.iosched.3stap man/stapprobes.irq.3stap  man/stapprobes.netdev.3stap man/stapprobes.nfs.3stap man/stapprobes.nfsd.3stap man/stapprobes.pagefault.3stap man/stapprobes.kprocess.3stap man/stapprobes.rpc.3stap man/stapprobes.scsi.3stap man/stapprobes.signal.3stap man/stapprobes.socket.3stap man/stapprobes.tcp.3stap man/stapprobes.udp.3stap man/stapprobes.snmp.3stap initscript/systemtap)
 AC_CONFIG_SUBDIRS(testsuite)
 if test $enable_translator == "yes"; then
 	AC_CONFIG_FILES([run-stap], [chmod +x run-stap])

Re: Tapset for probing IRQs, workqueues, etc

[Josh Stone]

On 10/29/2009 08:55 AM, Prerna Saxena wrote:
> Hi ,
> Posting a new version of the IRQ tapset.
> 
> On 10/27/2009 01:42 AM, David Smith wrote:
> 
>>
>> I took a short look at your tapset.  In general, it looks fine.
>>
>> I do have one question.  You added a function called '_irqflags_str' to
>> aux_syscalls.stp, but I couldn't find a caller of that new function in
>> your tapset.  It looks like you might have meant to call it for the
>> 'flags' variables in irq_handler.entry and irq_handler.exit.
> 
> I have not included the flags string on purpose for this probe point, 
> cos I think that one might need to look up the flags in specific 
> debugging scenarios only. It would not be required in most commonly used 
> cases, such as the attached example. However, I have documented the 
> function both inline in the tapset documentation and also in the man 
> pages, so that a script developer wanting to display a formatted string 
> can call the function when needed.

It wouldn't hurt to have the tapset provide it as something like
flags_str.  That makes it more obvious for the user, and our optimizer
should remove it automatically if it's not used.


Josh

Re: Tapset for probing IRQs, workqueues, etc

[Prerna Saxena]

On 10/30/2009 04:44 AM, Josh Stone wrote:
> On 10/29/2009 08:55 AM, Prerna Saxena wrote:
>> Hi ,
>> Posting a new version of the IRQ tapset.

>> I have not included the flags string on purpose for this probe point,
>> cos I think that one might need to look up the flags in specific
>> debugging scenarios only. It would not be required in most commonly used
>> cases, such as the attached example. However, I have documented the
>> function both inline in the tapset documentation and also in the man
>> pages, so that a script developer wanting to display a formatted string
>> can call the function when needed.
>
> It wouldn't hurt to have the tapset provide it as something like
> flags_str.  That makes it more obvious for the user, and our optimizer
> should remove it automatically if it's not used.
>

Thanks!
Fixed in commit a476086d1ef0448c86e466f5955f78d282b473fd for new IRQ tapset.

-- 
Prerna Saxena

Linux Technology Centre,
IBM Systems and Technology Lab,
Bangalore, India

Re: Tapset for probing IRQs, workqueues, etc

[Mark Wielaard]

Hi Prerna,

On Mon, 2009-11-09 at 11:43 +0530, Prerna Saxena wrote:
> On 10/30/2009 04:44 AM, Josh Stone wrote:
> > On 10/29/2009 08:55 AM, Prerna Saxena wrote:
> >> I have not included the flags string on purpose for this probe point,
> >> cos I think that one might need to look up the flags in specific
> >> debugging scenarios only. It would not be required in most commonly used
> >> cases, such as the attached example. However, I have documented the
> >> function both inline in the tapset documentation and also in the man
> >> pages, so that a script developer wanting to display a formatted string
> >> can call the function when needed.
> >
> > It wouldn't hurt to have the tapset provide it as something like
> > flags_str.  That makes it more obvious for the user, and our optimizer
> > should remove it automatically if it's not used.
> >
> Thanks!
> Fixed in commit a476086d1ef0448c86e466f5955f78d282b473fd for new IRQ tapset.

That commit calls irqflag_str() in the tapset/irq.stp, but didn't add
the function itself. I assume you have it locally in aux_syscalls.stp.
Could you commit that also?

Thanks,

Mark

Re: Tapset for probing IRQs, workqueues, etc

[Prerna Saxena]

On 11/13/2009 06:26 PM, Mark Wielaard wrote:
> Hi Prerna,
>
> That commit calls irqflag_str() in the tapset/irq.stp, but didn't add
> the function itself. I assume you have it locally in aux_syscalls.stp.
> Could you commit that also?
>
> Thanks,
>
> Mark
>

Hi Mark,
Thanks for pointing it out..I guess my patch hadnt applied cleanly !
I've added this via commit dc5a95301448c26bc016ebc618a8038e531d85f0

Thanks,

-- 
Prerna Saxena

Linux Technology Centre,
IBM Systems and Technology Lab,
Bangalore, India

Re: Tapset for probing IRQs, workqueues, etc

[Mark Wielaard]

On Mon, 2009-11-16 at 11:31 +0530, Prerna Saxena wrote:
> Thanks for pointing it out..

No problem. You added a testcase, it failed. So easy to detect :)

> I guess my patch hadnt applied cleanly !
> I've added this via commit dc5a95301448c26bc016ebc618a8038e531d85f0

Thanks. I added one small workaround for kernel < 2.6.32 (which isn't
actually released yet) which don't have IRQF_ONESHOT. With that
everything passes also on my systems.

Cheers,

Mark

From 7b76473cbad92366721defee51d93396cb101134 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mjw@redhat.com>
Date: Mon, 16 Nov 2009 10:33:20 +0100
Subject: [PATCH] Define IRQF_ONESHOT for older kernels.

IRQF_ONESHOT was only introduced in 2.6.32. It doesn't hurt defining it
if it isn't already there. It will just never show up on older kernels.

* tapset/aux_syscalls.stp: Define IRQF_ONESHOT if not already defined.
---
 tapset/aux_syscalls.stp |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/tapset/aux_syscalls.stp b/tapset/aux_syscalls.stp
index d544472..5435d3a 100644
--- a/tapset/aux_syscalls.stp
+++ b/tapset/aux_syscalls.stp
@@ -1900,6 +1900,9 @@ function _struct_sigaction32_u:string(uaddr:long)
 
 %{
 #include <linux/interrupt.h>
+#ifndef IRQF_ONESHOT
+#define IRQF_ONESHOT 0x00002000
+#endif
 static const _stp_val_array const _stp_irq_list[] = {
        V(IRQF_DISABLED),
        V(IRQF_SAMPLE_RANDOM),
-- 
1.6.2.5