From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 25089 invoked by alias); 11 Dec 2007 13:52:07 -0000 Received: (qmail 25079 invoked by uid 22791); 11 Dec 2007 13:52:06 -0000 X-Spam-Status: No, hits=-2.0 required=5.0 tests=AWL,BAYES_00,DK_POLICY_SIGNSOME,DNS_FROM_RFC_ABUSE,SPF_PASS,TW_II,TW_JB,TW_PK,TW_UH X-Spam-Check-By: sourceware.org Received: from E23SMTP02.au.ibm.com (HELO e23smtp02.au.ibm.com) (202.81.18.163) by sourceware.org (qpsmtpd/0.31) with ESMTP; Tue, 11 Dec 2007 13:51:59 +0000 Received: from d23relay03.au.ibm.com (d23relay03.au.ibm.com [202.81.18.234]) by e23smtp02.au.ibm.com (8.13.1/8.13.1) with ESMTP id lBBDplR7032464 for ; Wed, 12 Dec 2007 00:51:47 +1100 Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay03.au.ibm.com (8.13.8/8.13.8/NCO v8.7) with ESMTP id lBBDn5ln3404030 for ; Wed, 12 Dec 2007 00:49:05 +1100 Received: from d23av04.au.ibm.com (loopback [127.0.0.1]) by d23av04.au.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id lBBDmm7j019010 for ; Wed, 12 Dec 2007 00:48:48 +1100 Received: from [9.124.35.50] (srinivasa-009124035050.in.ibm.com [9.124.35.50]) by d23av04.au.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id lBBDmZno018715; Wed, 12 Dec 2007 00:48:36 +1100 Message-ID: <475E952F.90403@in.ibm.com> Date: Tue, 11 Dec 2007 13:52:00 -0000 From: Srinivasa Ds User-Agent: Thunderbird 2.0.0.6 (X11/20071022) MIME-Version: 1.0 To: Masami Hiramatsu CC: ananth@in.ibm.com, Jim Keniston , Roland McGrath , Arjan van de Ven , prasanna@in.ibm.com, anil.s.keshavamurthy@intel.com, davem@davemloft.net, systemtap-ml Subject: Re: [RFC -mm][PATCH 5/6] prepare kprobes code for x86 unification References: <475DC362.9000707@redhat.com> In-Reply-To: <475DC362.9000707@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2007-q4/txt/msg00470.txt.bz2 Masami Hiramatsu wrote: > This patch cleanup kprobes code on x86 for unification. > This patch is based on Arjan's previous work. > > - Remove spurious whitespace changes > - Add harmless includes > - Make the 32/64 files more identical > - Generalize structure fields' and local variable name. > - Wrap accessing to stack address by macros. > - Modify bitmap making macro. > - Merge fixup code into is_riprel() and change its name to fix_riprel(). > - Set MAX_INSN_SIZE to 16 on both arch. > Hi Masami I was testing your patch on x86_64 by executing systemtap tests. I got this oops message. Unable to handle kernel paging request at ffffffff8086ccb3 RIP: [] arch_prepare_kprobe+0x22/0x217 PGD 203067 PUD 207063 PMD 7e086163 PTE 86c000 Oops: 0000 [1] SMP last sysfs file: /sys/module/stap_60ea9007c2ab1a78963339fffdc0a88e_356908/sections/.bss CPU 1 Modules linked in: stap_60ea9007c2ab1a78963339fffdc0a88e_356908 systemtap_test_module1 systemtap_test_module2 ipv6 autofs4 hidp rfcomm l2cap bluetooth sunrpc dm_multipath video output sbs sbshc battery acpi_memhotplug ac power_supply lp sg tg3 button i2c_i801 ide_cd parport_pc shpchp parport serio_raw cdrom i2c_core e752x_edac floppy edac_core pcspkr dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata aic79xx scsi_transport_spi sd_mod scsi_mod ext3 jbd ehci_hcd ohci_hcd uhci_hcd Pid: 3171, comm: stapio Tainted: GF 2.6.24-rc4-mm1 #2 RIP: 0010:[] [] arch_prepare_kprobe+0x22/0x217 RSP: 0018:ffff81003b6d7e48 EFLAGS: 00010282 RAX: ffffffff8086ccb3 RBX: ffffffff884371d0 RCX: ffffffff88224f20 RDX: 0000000000000f20 RSI: 6600000000000000 RDI: ffffffff884371d0 RBP: ffffffff884371d0 R08: ffff810035daf000 R09: ffff81007f834000 R10: ffffffff8024bf9c R11: 0000000000000000 R12: 00000000000036b0 R13: 0000000000000000 R14: ffffffff8840e3b2 R15: 0000000000000000 FS: 00002b2e14748b00(0000) GS:ffff81007fbac840(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffff8086ccb3 CR3: 000000007c5a3000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Process stapio (pid: 3171, threadinfo ffff81003b6d6000, task ffff8100187bc340) Stack: 0000000000000000 ffffffff80474c5c 0000000000000000 ffffffff884371d0 0000000000000000 00000000000036b0 00000000000001d2 ffffffff8840485b 00000000000036b0 00000000000a7fc4 ffff81003b6d7ee8 0000000000000008 Call Trace: [] __register_kprobe+0x1f0/0x2e8 [] :stap_60ea9007c2ab1a78963339fffdc0a88e_356908:systemtap_module_init+0x202/0x45f [] :stap_60ea9007c2ab1a78963339fffdc0a88e_356908:probe_start+0x9/0x12 [] :stap_60ea9007c2ab1a78963339fffdc0a88e_356908:_stp_handle_start+0x21/0x7c [] :stap_60ea9007c2ab1a78963339fffdc0a88e_356908:_stp_ctl_write_cmd+0x72/0xc3 [] audit_syscall_entry+0x141/0x174 [] vfs_write+0xc6/0x14f [] sys_write+0x45/0x6e [] tracesys+0xdc/0xe1 Code: 48 8b 10 48 89 11 48 8b 40 08 48 89 41 08 48 8b 53 70 8a 02 RIP [] arch_prepare_kprobe+0x22/0x217 RSP CR2: ffffffff8086ccb3 On debugging it further I found that, current.stp is causing oops(easily reproducible too) Current.stp probes all functions in kernel/sched.c and some module functions. 743 arch_copy_kprobe(): 744 /root/linux-2.6.24-rc4/arch/x86/kernel/kprobes.c:312 745 6bf: 48 8b 43 30 mov 0x30(%rbx),%rax 746 6c3: 48 8b 10 mov (%rax),%rdx <<<<<<<>>>> 747 6c6: 48 89 11 mov %rdx,(%rcx) 748 6c9: 48 8b 40 08 mov 0x8(%rax),%rax 749 6cd: 48 89 41 08 mov %rax,0x8(%rcx) ============================================== 310 static void __kprobes arch_copy_kprobe(struct kprobe *p) 311 { 312 memcpy(p->ainsn.insn, p->addr, MAX_INSN_SIZE * sizeof(kprobe_opcode_t)); <> 313 fix_riprel(p); 314 if (can_boost(p->addr)) { That means on accessing rax(p->addr), kernel has crashed. RAX=p->addr= ffffffff8086ccb3 cat /proc/kallsyms | grep ffffffff8086ccb3 ffffffff8086ccb3 t init_sched_debug_procfs Since "init_sched_debug_procfs" is a __init function and deallocation of memory of __init function may be causing the problem.I tried probing this function directly, but didn't see any oops. Thanks Srinivasa DS