From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 27396 invoked by alias); 7 Dec 2011 17:40:58 -0000 Received: (qmail 27386 invoked by uid 22791); 7 Dec 2011 17:40:57 -0000 X-SWARE-Spam-Status: No, hits=-6.7 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,SPF_HELO_PASS,TW_GF,TW_II,TW_KD,TW_PC X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 07 Dec 2011 17:40:43 +0000 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pB7Hef36004170 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 7 Dec 2011 12:40:41 -0500 Received: from fche.csb (vpn-8-102.rdu.redhat.com [10.11.8.102]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id pB7HefKv023351; Wed, 7 Dec 2011 12:40:41 -0500 Received: by fche.csb (Postfix, from userid 2569) id 74E2E5815E; Wed, 7 Dec 2011 12:40:40 -0500 (EST) To: Cc: Subject: Re: Kernel panics during execution of timeouts.stp example when booted with kgdboc=kms,kbd References: <9C7BACB01B839A499792154E76C9ADE5563DC350EF@MX19A.corp.emc.com> From: fche@redhat.com (Frank Ch. Eigler) Date: Wed, 07 Dec 2011 20:11:00 -0000 In-Reply-To: <9C7BACB01B839A499792154E76C9ADE5563DC350EF@MX19A.corp.emc.com> (dmitry krivenok's message of "Tue, 6 Dec 2011 03:39:31 -0500") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2011-q4/txt/msg00314.txt.bz2 writes: > I'm new to Systemtap, but already found it very useful and even > solved some real problems using it. Great! > The problem is that kernel panics if you run it with > "kgdboc=kms,kbd" parameter and then run standard timeout.stp example > as follows: [...] I can reproduce this crash even on plain kvm, in this case running systemtap: 1.6/0.152, 2.6.41.1-1.fc15.x86_64, 2 VCPU. Entering kdb (current=0xffff88007a4c1730, pid 1667) on processor 0 Oops: (null) due to oops @ 0xffffffff81076c33 CPU 0 Modules linked in: stap_7cd19ef8989e5e76ef083a66b68e4d39__1667 iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 netconsole configfs nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ppdev 8139too parport_pc parport 8139cp i2c_piix4 i2c_core mii ipv6 [last unloaded: scsi_wait_scan] Pid: 1667, comm: stapio Not tainted 2.6.41.1-1.fc15.x86_64 #1 Bochs Bochs RIP: 0010:[] [] sys_nanosleep+0x1/0x6a RSP: 0018:ffff880079367f80 EFLAGS: 00000297 RAX: 0000000000000023 RBX: ffffffffffffffff RCX: 00002ba8344af9d0 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00007fff249c12a0 RBP: 00002ba82cda7010 R08: 00002ba8344af700 R09: 00002ba8344af700 R10: 00002ba8344af9d0 R11: 0000000000000293 R12: 0000000000008002 R13: 00002ba82cdad178 R14: 00002ba82cda7008 R15: 000000000000000b FS: 00002ba82d585120(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00002ba82d2c3920 CR3: 0000000037635000 CR4: 00000000000006f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process stapio (pid: 1667, threadinfo ffff880079366000, task ffff88007a4c1730) <0>Stack: ffffffff814a3102 0000000000000293 00002ba8344af9d0 00002ba8344af700 00002ba8344af700 0000000000000023 00002ba82cfdd1e5 0000000000000000 0000000000000000 00007fff249c12a0 0000000000000023 00002ba82d2956ad <0>Call Trace: Without the kgdboc=kms,kbd boot options, the test case works fine. This leads me to believe that there is a conflict between the kernel's kretprobes and kgdb systems, and likely not a bug in systemtap proper. (I haven't been able to trigger the problem with a couple of 'perf probe' attempts though.) However, checking the hack associated with http://sourceware.org/PR13193, disabling kprobes optimization makes this test case work for me. Please try # echo 0 > /proc/sys/debug/kprobes-optimization and rerun your tests. The next release of systemtap will probably include code to do the same thing. :-( - FChE