From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 5575 invoked by alias); 4 Mar 2010 17:46:06 -0000 Received: (qmail 5197 invoked by uid 48); 4 Mar 2010 17:45:51 -0000 Date: Thu, 04 Mar 2010 17:46:00 -0000 Message-ID: <20100304174551.5196.qmail@sourceware.org> From: "srikar at linux dot vnet dot ibm dot com" To: systemtap@sources.redhat.com In-Reply-To: <20100204125941.11249.mjw@redhat.com> References: <20100204125941.11249.mjw@redhat.com> Reply-To: sourceware-bugzilla@sourceware.org Subject: [Bug runtime/11249] Tracking executable plus library fails on i386 X-Bugzilla-Reason: AssignedTo Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2010-q1/txt/msg00590.txt.bz2 ------- Additional Comments From srikar at linux dot vnet dot ibm dot com 2010-03-04 17:45 ------- The instruction in question is "call <__i686.get_pc_thunk.cx>" For __getpagesize, this is the first instruction of the function. Functions that exihibit similar problem are gnu_get_libc_release, gnu_get_libc_version,__errno_location, __gconv_get_modules_db, __gconv_get_alias_db etc. The interesting thing being in all these functions "call <__i686.get_pc_thunk.cx>" is the first instruction. On Subsequent analysis, We make a copy of this particular instruction to the ssol vma (in one of the slots of ssol vma). When we singlestep on that slot, we get a general protection fault. The simplest way to reproduce this is $ cat /tmp/a.c #include #include #include int main(int argc, char *argv[]) { printf("GNU libc version: %s\n", gnu_get_libc_version()); printf("GNU libc release: %s\n", gnu_get_libc_release()); exit(EXIT_SUCCESS); } /* END */ $ $ make /tmp/a $ stap -v -v -e 'probe process("/lib/libc.so.6").function("gnu_get_libc_version").call { log(pp()) }' -c /tmp/a $ $ Running a instrumented uprobes for the above program shows --> func=register_uprobe <-- func=register_uprobe return=0 --> func=uprobe_report_signal active_ppt=(null) utask->state=UTASK_RUNNING regs->ip=00788b61 signo=5 <-- func=uprobe_report_signal active_ppt=f4c11680 utask->state=UTASK_SINGLESTEP regs->ip=bfe03020 return=19 --> func=uprobe_report_signal active_ppt=f4c11680 utask->state=UTASK_SINGLESTEP regs->ip=bfe03020 signo=11 <-- func=uprobe_report_signal active_ppt=f4c11680 utask->state=UTASK_SINGLESTEP regs->ip=bfe03020 return=51 So it shows that we hit a probepoint, which we singlestepped. However uprobes signal callback was called because of SIGSEGV after it we requested for singlestep and the ip(bfe03020) still points to the slot in ssol vma. If we have a WARN_ON(ppt) in uprobe_report_exit(), the trace looks something like this. [] warn_slowpath+0x7c/0xa4 [] ? __put_cred+0x1f/0x21 [] ? put_cred+0x21/0x23 [] ? do_coredump+0x67a/0x74d [] uprobe_report_exit+0x3a/0x15b [uprobes] [] utrace_report_exit+0x84/0xcc [] do_exit+0x7e/0x705 [] ? utrace_get_signal+0x563/0x595 [] do_group_exit+0x74/0x9b [] get_signal_to_deliver+0x33c/0x363 [] ? do_general_protection+0x0/0x22c [] do_notify_resume+0x6e/0x62c [] ? task_rq_unlock+0xf/0x11 [] ? try_to_wake_up+0x226/0x231 [] ? complete_signal+0x119/0x120 [] ? send_signal+0x1a8/0x1bd [] ? force_sig_info+0xa2/0xac [] ? do_general_protection+0x0/0x22c [] work_notifysig+0x13/0x1a I was using 2.6.29.4-167.fc11.i686.PAE in my analysis. Whats interesting is if we run a 2.6.33-rc4 git kernel from branch utrace-gdbstub-uprobes from location git://web.elastic.org/~fche/utrace-ext.git on the same machine, the problem disappears. Though it needs commits 21e8e579ef10942bf2db3e1514026a6d132b1502 and 4c5ce7a55108edb5203b3d69949f09c2284f1963 from systemtap git tree to be reverted to make systemtap work. I dont see any difference in the ssol vma area is setup in uprobes embedded in SystemTap and the one in branch utrace-gdbstub-uprobes branch that could be causing this. -- What |Removed |Added ---------------------------------------------------------------------------- CC| |jkenisto at us dot ibm dot | |com http://sourceware.org/bugzilla/show_bug.cgi?id=11249 ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.