From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 21334 invoked by alias); 9 Jun 2007 22:04:10 -0000 Received: (qmail 21327 invoked by uid 22791); 9 Jun 2007 22:04:09 -0000 X-Spam-Status: No, hits=-1.1 required=5.0 tests=AWL,BAYES_05,DK_POLICY_SIGNSOME,DNS_FROM_RFC_ABUSE,SPF_PASS X-Spam-Check-By: sourceware.org Received: from e4.ny.us.ibm.com (HELO e4.ny.us.ibm.com) (32.97.182.144) by sourceware.org (qpsmtpd/0.31) with ESMTP; Sat, 09 Jun 2007 22:04:06 +0000 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e4.ny.us.ibm.com (8.13.8/8.13.8) with ESMTP id l59M443c002195 for ; Sat, 9 Jun 2007 18:04:04 -0400 Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v8.3) with ESMTP id l59M44nG357872 for ; Sat, 9 Jun 2007 18:04:04 -0400 Received: from d01av02.pok.ibm.com (loopback [127.0.0.1]) by d01av02.pok.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id l59M44WP009221 for ; Sat, 9 Jun 2007 18:04:04 -0400 Received: from [9.47.18.79] (dyn9047018079.beaverton.ibm.com [9.47.18.79]) by d01av02.pok.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id l59M43UR009209 for ; Sat, 9 Jun 2007 18:04:04 -0400 Subject: [PATCH] uprobes: fix SMP multithreading race From: Jim Keniston To: systemtap Content-Type: multipart/mixed; boundary="=-2sx3oZGeMgg1IU1+lmAJ" Date: Sat, 09 Jun 2007 22:04:00 -0000 Message-Id: <1181423053.3661.3.camel@ibm-ni9dztukfq8.beaverton.ibm.com> Mime-Version: 1.0 X-Mailer: Evolution 2.8.3 (2.8.3-2.fc6) X-Virus-Checked: Checked by ClamAV on sourceware.org X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2007-q2/txt/msg00545.txt.bz2 --=-2sx3oZGeMgg1IU1+lmAJ Content-Type: text/plain Content-Transfer-Encoding: 7bit Content-length: 1041 The enclosed patch fixes a problem seen on SMP systems when multiple threads of a multithreaded app are pounding the probed instruction when the uprobe is registered. This patch applies atop the most recent uprobes patch set, which I posted May 25 (May 26 UTC): http://sources.redhat.com/ml/systemtap/2007-q2/msg00399.html The problem was intermittent, but I was able to tickle it pretty reliably using the enclosed script. In the test suite, run $ probe7-thread 1000000000 # 1 billion and after that's started, run the script # ./poundt The bug causes probe7-thread to die with a SIGSEGV. The fix allows probe7-thread to run to completion. Jim Keniston ----- poundt ----- #!/bin/bash vaddr=`objdump -d probe7-thread | awk '$2==":" {print $1}'` pid=`ps -e -o fname,pid | awk '$1=="probe7-t" {print $2}'` kill -cont $pid # While probed process runs... while (ps -p $pid > /dev/null) do sleep 0.2 insmod probe5.ko verbose=0 vaddr=0x$vaddr pid=$pid if [ $? -ne 0 ] then exit 1 fi sleep 0.2 rmmod probe5 done exit 0 ----- --=-2sx3oZGeMgg1IU1+lmAJ Content-Disposition: attachment; filename=uprobes-smp-pounder-fix.patch Content-Type: text/x-patch; name=uprobes-smp-pounder-fix.patch; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-length: 2297 Fixes a problem seen on SMP systems when multiple threads of a multithreaded app are pounding the probed instruction when the uprobe is registered. uprobe_verify_ssol() was releasing the second thread through before SSOL setup was complete. --- include/linux/uprobes.h | 3 +++ kernel/uprobes.c | 8 +++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff -puN include/linux/uprobes.h~uprobes-smp-pounder-fix include/linux/uprobes.h --- linux-2.6.21-rc6/include/linux/uprobes.h~uprobes-smp-pounder-fix 2007-06-09 11:21:05.000000000 -0700 +++ linux-2.6.21-rc6-jimk/include/linux/uprobes.h 2007-06-09 11:23:33.000000000 -0700 @@ -169,6 +169,9 @@ struct uprobe_ssol_area { /* Ensures 2 threads don't try to set up the vma simultaneously. */ struct mutex setup_mutex; + + /* 1 = we've at least tried. IS_ERR(insn_area) if we failed. */ + int initialized; }; /* diff -puN kernel/uprobes.c~uprobes-smp-pounder-fix kernel/uprobes.c --- linux-2.6.21-rc6/kernel/uprobes.c~uprobes-smp-pounder-fix 2007-06-09 11:21:05.000000000 -0700 +++ linux-2.6.21-rc6-jimk/kernel/uprobes.c 2007-06-09 11:23:51.000000000 -0700 @@ -285,9 +285,9 @@ static void rouse_all_threads(struct upr if (utask->quiescing) { utask->quiescing = 0; if (utask->state == UPTASK_QUIESCENT) { - clear_utrace_quiesce(utask); utask->state = UPTASK_RUNNING; uproc->n_quiescent_threads--; + clear_utrace_quiesce(utask); } } } @@ -533,6 +533,7 @@ static struct uprobe_process *uprobe_mk_ uproc->uretprobe_trampoline_addr = NULL; uproc->ssol_area.insn_area = NULL; + uproc->ssol_area.initialized = 0; mutex_init(&uproc->ssol_area.setup_mutex); #ifdef CONFIG_UPROBES_SSOL uproc->sstep_out_of_line = 1; @@ -1020,13 +1021,14 @@ static __user uprobe_opcode_t { struct uprobe_ssol_area *area = &uproc->ssol_area; - if (unlikely(!area->insn_area)) { + if (unlikely(!area->initialized)) { /* First time through for this probed process */ static DEFINE_MUTEX(ssol_setup_mutex); mutex_lock(&ssol_setup_mutex); - if (likely(!area->insn_area)) + if (likely(!area->initialized)) /* Nobody snuck in and set things up ahead of us. */ uprobe_init_ssol(uproc); + area->initialized = 1; mutex_unlock(&ssol_setup_mutex); } return area->insn_area; _ --=-2sx3oZGeMgg1IU1+lmAJ--