printk in kernel.function("__switch_to") probe crashes machine

printk in kernel.function("__switch_to") probe crashes machine

[Dave Nomura]

3. I have written a kernel function probe in __switch_to and find that I 
can define it as:
probe kernel.function("__switch_to")
{
  print("calling ITrace_switch\n");
  // ITrace_switch()
}
If I run this script I get:
Running sudo /usr/bin/staprun -u dcn -d 3614 -c 
"/home/dcn/itrace/ITrace_test/hanoi/hanoi64 -2" /tmp/stapAiagsM/pitrace.ko
start itrace
calling ITrace_switch
....
calling ITrace_switch
end itrace
and this works just as expected.
However, if I uncomment ITrace_switch() call in my __switch_to probe, 
and define it as:
function ITrace_switch()
%{
  printk("ITrace_switch called\n");
%}
this crashes the machine.

I'm a SystemTap novice, and kernel novice so I'm not sure if there is 
some obvious reason why this "printk" should cause the system to crash.  
There are no messages on the console.

If there is a simple explanation I guess it might help me avoid doing 
this sort of thing in the future.  For now, I'll just avoid using printk 
in this scenario.

Here is the machine info:
Linux elm3b158 2.6.16.43-0.5-ppc64 #1 SMP Mon Mar 19 08:28:24 UTC 2007 
ppc64 ppc64 ppc64 GNU/Linux
SUSE Linux Enterprise Server 10 SP1 RC1 (ppc)

Re: printk in kernel.function("__switch_to") probe crashes machine

[Jim Keniston]

On Wed, 2007-04-25 at 16:17 -0700, Dave Nomura wrote:
> 3. I have written a kernel function probe in __switch_to and find that I 
> can define it as:
> probe kernel.function("__switch_to")
> {
>   print("calling ITrace_switch\n");
>   // ITrace_switch()
> }
> If I run this script I get:
> Running sudo /usr/bin/staprun -u dcn -d 3614 -c 
> "/home/dcn/itrace/ITrace_test/hanoi/hanoi64 -2" /tmp/stapAiagsM/pitrace.ko
> start itrace
> calling ITrace_switch
> ....
> calling ITrace_switch
> end itrace
> and this works just as expected.
> However, if I uncomment ITrace_switch() call in my __switch_to probe, 
> and define it as:
> function ITrace_switch()
> %{
>   printk("ITrace_switch called\n");
> %}
> this crashes the machine.
> 
> I'm a SystemTap novice, and kernel novice so I'm not sure if there is 
> some obvious reason why this "printk" should cause the system to crash.  
> There are no messages on the console.
> 
> If there is a simple explanation I guess it might help me avoid doing 
> this sort of thing in the future.  For now, I'll just avoid using printk 
> in this scenario.

We advise against using printk in SystemTap handlers.  We know of some
functions that, if kprobed with handlers that call printk, can deadlock.
See bugzilla #3541.  I wouldn't think that __switch_to() would be in
that group (since printk() doesn't block and so shouldn't call
__switch_to()), but I could be wrong.

Jim

> 
> Here is the machine info:
> Linux elm3b158 2.6.16.43-0.5-ppc64 #1 SMP Mon Mar 19 08:28:24 UTC 2007 
> ppc64 ppc64 ppc64 GNU/Linux
> SUSE Linux Enterprise Server 10 SP1 RC1 (ppc)
> 

Re: printk in kernel.function("__switch_to") probe crashes machine

[Martin Hunt]

On Wed, 2007-04-25 at 16:17 -0700, Dave Nomura wrote:

> function ITrace_switch()
> %{
>   printk("ITrace_switch called\n");
> %}
> this crashes the machine.

A quick look at the source for __switch_to shows this comment

/* never put a printk in __switch_to... printk() calls wake_up*()
indirectly */

With systemtap it is always safe to use the builtin IO; print, printf,
etc. Using printk is not advised (and not permitted except when safety
is disabled with "-g") because it is not always safe.

Martin