[Bug kprobes/9740] New: syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[Bug kprobes/9740] New: syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

I've run stap -ve 'global pids; probe syscall.* {pids[pid()]<<<1}' on
2.6.29-0.31.rc1.git2.fc11.i686 and got below double fault.
---
PANIC: double fault, gdt at c34d3000 [255 bytes]
double fault, tss at c3619d80
eip = c06e4b9d, esp = f1a48f98
eax = f1a490d0, ebx = f8c5b1f0, ecx = 0000007b, edx = 00000000
esi = c0537f0e, edi = 00000000
---

I could reproduce this fault and it always happened the 2nd time of staprun. So,
how to reproduce is;
1. run stap -ve 'global pids; probe syscall.* {pids[pid()]<<<1}'
2. stop it (if you see a privilege error, don't care or remove stap module by
staprun -d)
3. run above command again.

And "c06e4b8f T do_page_fault".

c06e4b8f <do_page_fault>:
c06e4b8f:       55                      push   %ebp
c06e4b90:       89 e5                   mov    %esp,%ebp
c06e4b92:       57                      push   %edi
c06e4b93:       89 d7                   mov    %edx,%edi
c06e4b95:       56                      push   %esi
c06e4b96:       53                      push   %ebx
c06e4b97:       81 ec 24 01 00 00       sub    $0x124,%esp
c06e4b9d:       89 85 d0 fe ff ff       mov    %eax,-0x130(%ebp)

-- 
           Summary: syscall.* probe causes kernel panic(double fault) on
                    rawhide kernel/i686
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: kprobes
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: mhiramat at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/9740] syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

------- Additional Comments From mhiramat at redhat dot com  2009-01-13 21:47 -------
I disabled unregister_kprobes() and tested, but same double fault happened.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/9740] syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

------- Additional Comments From mhiramat at redhat dot com  2009-01-13 22:30 -------
This might be related to synchronize_sched() changes.
Here, the syscall2.ko is the pre-compiled script of 'global pids; probe
syscall.* {pids[pid()]<<<1}' *without* batch unregister.

$ time staprun -d syscall2.ko

real	0m0.005s
user	0m0.000s
sys	0m0.003s

This machine has 4 cpu and syscall2.ko has 318 probes. this means when removing
syscall2.ko, kprobes may call synchronize_sched() 318 times. Usually, it needs
more than 100ms because it doesn't use unregister_kprobes().


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/9740] syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

------- Additional Comments From mhiramat at redhat dot com  2009-01-13 23:07 -------
Here, I checked unloading time on 2.6.28-mm1 on 2CPU machine.

$ time staprun -d syscall2.ko 

real	0m0.340s
user	0m0.000s
sys	0m0.014s

As you can see, real took over 340ms because synchronize_sched() waits scheduler
synchronization.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/9740] syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

------- Additional Comments From mhiramat at redhat dot com  2009-01-13 23:21 -------
even if I use a vanilla 2.6.28-rc1 kernel(use same kconfig), double fault
occurred...

$ time staprun -d syscall2.ko 

real	0m0.006s
user	0m0.000s
sys	0m0.003s


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug kprobes/9740] syscall.* probe causes kernel panic(double fault) on rawhide kernel/i686

[mhiramat at redhat dot com]

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ananth at in dot ibm dot com
         AssignedTo|systemtap at sources dot    |mhiramat at redhat dot com
                   |redhat dot com              |
             Status|NEW                         |ASSIGNED


http://sourceware.org/bugzilla/show_bug.cgi?id=9740

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.