From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 32133 invoked by alias); 3 Feb 2006 22:00:16 -0000 Received: (qmail 32125 invoked by uid 22791); 3 Feb 2006 22:00:15 -0000 X-Spam-Status: No, hits=-2.1 required=5.0 tests=AWL,BAYES_00,DNS_FROM_RFC_ABUSE,SPF_PASS X-Spam-Check-By: sourceware.org Received: from e32.co.us.ibm.com (HELO e32.co.us.ibm.com) (32.97.110.150) by sourceware.org (qpsmtpd/0.31) with ESMTP; Fri, 03 Feb 2006 22:00:14 +0000 Received: from d03relay04.boulder.ibm.com (d03relay04.boulder.ibm.com [9.17.195.106]) by e32.co.us.ibm.com (8.12.11/8.12.11) with ESMTP id k13M0D7c012758 for ; Fri, 3 Feb 2006 17:00:13 -0500 Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by d03relay04.boulder.ibm.com (8.12.10/NCO/VERS6.8) with ESMTP id k13M2aoY169778 for ; Fri, 3 Feb 2006 15:02:36 -0700 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.12.11/8.13.3) with ESMTP id k13M0Cbh000841 for ; Fri, 3 Feb 2006 15:00:12 -0700 Received: from [127.0.0.1] (sig-9-65-58-31.mts.ibm.com [9.65.58.31]) by d03av02.boulder.ibm.com (8.12.11/8.12.11) with ESMTP id k13M07Xp000479; Fri, 3 Feb 2006 15:00:09 -0700 Message-ID: <43E3D262.7020405@us.ibm.com> Date: Fri, 03 Feb 2006 22:00:00 -0000 From: Vara Prasad User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Stone, Joshua I" CC: "Frank Ch. Eigler" , systemtap@sources.redhat.com Subject: Re: user kprobes vs debuggers References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2006-q1/txt/msg00383.txt.bz2 Stone, Joshua I wrote: >Vara Prasad wrote: > > >>Let us say if there is no user space probes involved debugger >>remembers the original instruction before replacing it is with >>breakpoint instruction. It single steps the original instruction that >>is stored in the userspace. Let us say if there is no debugger and >>there is only userspace probes userspace pobes does the same single >>stepping but in the kernel space. Another important distinction to >>remember here is each of them have their own handlers to run when the >>break point is hit. Based on the above background there is a global >>registry of the breakpoints in the kernel that is only used to notify >>who all would like to handle this breakpoint but it is up to each of >>the owners to run their own handlers and as well as handle single >>stepping. >> >> > >Sure, I see no issue when only a debugger or a userspace probe >instruments a given address - they just single-step it themselves. The >issue I was referring to is when a debugger and a userspace probe try to >instrument the same address. In that scenario it's not clear who should >single-step the original instruction and how. > > I didn't do a good job of explaining above, if there is a breakpoint installed at a given address by both probes and debuggers, we only need to single step the original instruction ones and that single stepping needs to be done by the debugger as it gets to handle the breakpoint the last. I am assuming for this discussion we only support one debugger debugging the program which is the current Linux model anyway. In other words the sequence when we have a uprobe and debugger breakpoint at the same address is break point is hit Uprobes pre-handler is executed (no single stepping) pass the control to the debugger debugger runs the pre handler (which is gives the control to the user) single steps the original instruction >It's made worse if probes & debuggers are registered & removed in an >interleaved fashion - e.g. probe A registers; debugger B registers; A is >removed; what happens to B? > > > Answer to your above scenario is global registry will still have breakpoint at the address with single handler for the debugger after uprobe is removed. I think order in which who placed the breakpoint doesn't matter as long as the global registry has the knowledge if there are multiple handlers or single handler for the given breakpoint. If there are multiple handlers protocol is always userspace probes first and then debugger. In our current implementation user space probes do allow pre-handler and post-handler to my knowledge only pre-handler is the common usage. If we have to support post-handler as well then it become bit tricky but can be handled. We need to then register a notifier to get the control back to user space probes code so that we can run post-handler. We may be able to do some optimization and avoid this over head if there are no mulitple handlers at a given address. >>I think it would have been o.k to disallow but due to common uses like >>strace people might object to userspace probes feature. >> >> > >You only have to disallow probes & debugger breakpoints coexisting at >the same address - if they're at different addresses then there's no >problem. If that's still too restrictive, then we need to resolve the >issues above. > > Like i said due to common usage of strace kind of utilities we can not really do that. If we allow break points if there is no collision but not allow when there is no collision, strace could potentially give you wrong results as some of its breakpoints are disallowed or it may fail to run. There is one aspect of the problem that Jim reminded me that i didn't address which is copy on write (COW). Usually when multiple people wants to run the same program text segments are shared, but if one of the person wants to debug the program, OS makes a copy on write for the debugger and removes the write protection on that copy. This poses some challenge with our current implementation of userspace probes which are for the program not process hence we don't do the copy on write unlike a debugger. As you can imagine if we have put one user space probes in a process, multiple copies of which are being run. Then comes along a user who wants to debug the program and put one break point due to COW that copy might end up with two break points now, one from userspace probes and one from the debugger that is bad. We may have to rethink our our program based approach vs process based approach for user space probes. If we go the route of program based approach we could do a COW of our own and debugger could come along later and do a cow of its own which makes this problem go away completely as each of them have their own copies. I have not looked at the kernel code to see if you already have a text page with write permission will it make another write copy version or not. I need to think a bit more about this issue. >Josh > > >