Re: Rapidly running systemtap causing hangs or oops

[Josh Stone <jistone@redhat.com>]

On 06/22/2011 04:00 PM, Richard W.M. Jones wrote:
> Me again.  I can get something involving systemtap, ext2, the loop
> device, Linux 3.0 to oops very easily.  I'm not quite sure exactly
> what factor causes it, but here's an easy reproducer:
> 
> $ mkdir /tmp/mnt
> 
> $ truncate -s 1G /tmp/fs
> $ mkfs.ext2 -F /tmp/fs
> 
> $ cat > /tmp/test.sh 
> #!/bin/sh -
> echo mount
> mount -o loop /tmp/fs /tmp/mnt
> echo unmount
> umount /tmp/mnt
> 
> $ chmod +x /tmp/test.sh
> 
> $ while sudo stap -e 'probe module("ext2").statement ("*@*.c:*") { printf ("%s\n", pp()); }' -c /tmp/test.sh ; do : ; done
> 
> The final command usually either hangs the machine, or produces a long
> oops like the one attached, after just a few iterations.  It takes
> just a few seconds on my VM to get a hang or oops.

Can you try running stap with "-D STP_ALIBI"?  This alibi mode compiles
out most of stap's code, so each probe handler is reduced to just an
atomic increment, then a final hit count is reported on exit.

Another test might be to move the loop inside test.sh, so stap is left
running the whole time, and we might tell if the issue is timed around
stap's probe registration or unregistration.

> [  342.037017]  [<ffffffff8100b0ce>] show_registers+0xbd/0x206
> [  342.037017]  [<ffffffff814f6cba>] ? atomic_notifier_call_chain+0x14/0x16
> [  342.037017]  [<ffffffff814f4941>] __die+0x97/0xd8
> [  342.037017]  [<ffffffff8100be1c>] die+0x47/0x63
> [  342.037017]  [<ffffffff81009d79>] do_double_fault+0x65/0x67
> [  342.037017]  [<ffffffff814fb1aa>] double_fault+0x2a/0x30
> [  342.037017]  [<ffffffffa00ca6a6>] ? ext2_get_inode+0x6d/0x130 [ext2]

Is the Oops always this minimal?  Does it always (questionably) point to
the same ext2_get_inode location?

I'll play with this tomorrow and see if I can reproduce it myself...

Josh