From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 864 invoked by alias); 23 Nov 2009 23:19:47 -0000 Received: (qmail 851 invoked by uid 22791); 23 Nov 2009 23:19:45 -0000 X-SWARE-Spam-Status: No, hits=-2.5 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 23 Nov 2009 23:19:39 +0000 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id nANNJc8n024543 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 23 Nov 2009 18:19:38 -0500 Received: from dhcp-100-2-132.bos.redhat.com (dhcp-100-2-132.bos.redhat.com [10.16.2.132]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id nANNJafN024496; Mon, 23 Nov 2009 18:19:36 -0500 From: Masami Hiramatsu Subject: [PATCH -tip v5 10/10] [RFC] kprobes/x86: Use text_poke_fixup() for jump optimization To: Frederic Weisbecker , Ingo Molnar , Ananth N Mavinakayanahalli , lkml Cc: systemtap, DLE, Masami Hiramatsu , Ananth N Mavinakayanahalli , Ingo Molnar , Jim Keniston , Srikar Dronamraju , Christoph Hellwig , Steven Rostedt , Frederic Weisbecker , "H. Peter Anvin" , Anders Kaseorg , Tim Abbott , Andi Kleen , Jason Baron , Mathieu Desnoyers Date: Mon, 23 Nov 2009 23:19:00 -0000 Message-ID: <20091123232234.22071.68883.stgit@dhcp-100-2-132.bos.redhat.com> In-Reply-To: <20091123232115.22071.71558.stgit@dhcp-100-2-132.bos.redhat.com> References: <20091123232115.22071.71558.stgit@dhcp-100-2-132.bos.redhat.com> User-Agent: StGIT/0.14.3 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2009-q4/txt/msg00649.txt.bz2 Use text_poke_fixup() for jump optimization instead of text_poke() with stop_machine(). Signed-off-by: Masami Hiramatsu Cc: Ananth N Mavinakayanahalli Cc: Ingo Molnar Cc: Jim Keniston Cc: Srikar Dronamraju Cc: Christoph Hellwig Cc: Steven Rostedt Cc: Frederic Weisbecker Cc: H. Peter Anvin Cc: Anders Kaseorg Cc: Tim Abbott Cc: Andi Kleen Cc: Jason Baron Cc: Mathieu Desnoyers --- arch/x86/kernel/kprobes.c | 54 +++------------------------------------------ 1 files changed, 3 insertions(+), 51 deletions(-) diff --git a/arch/x86/kernel/kprobes.c b/arch/x86/kernel/kprobes.c index 6d81c11..3c5e30f 100644 --- a/arch/x86/kernel/kprobes.c +++ b/arch/x86/kernel/kprobes.c @@ -49,7 +49,6 @@ #include #include #include -#include #include #include @@ -1355,54 +1354,6 @@ int __kprobes arch_prepare_optimized_kprobe(struct optimized_kprobe *op) return 0; } -/* - * Cross-modifying kernel text with stop_machine(). - * This code originally comes from immediate value. - * This does _not_ protect against NMI and MCE. However, - * since kprobes can't probe NMI/MCE handler, it is OK for kprobes. - */ -static atomic_t stop_machine_first; -static int wrote_text; - -struct text_poke_param { - void *addr; - const void *opcode; - size_t len; -}; - -static int __kprobes stop_machine_multibyte_poke(void *data) -{ - struct text_poke_param *tpp = data; - - if (atomic_dec_and_test(&stop_machine_first)) { - text_poke(tpp->addr, tpp->opcode, tpp->len); - smp_wmb(); /* Make sure other cpus see that this has run */ - wrote_text = 1; - } else { - while (!wrote_text) - smp_rmb(); - sync_core(); - } - - flush_icache_range((unsigned long)tpp->addr, - (unsigned long)tpp->addr + tpp->len); - return 0; -} - -static void *__kprobes __multibyte_poke(void *addr, const void *opcode, - size_t len) -{ - struct text_poke_param tpp; - - tpp.addr = addr; - tpp.opcode = opcode; - tpp.len = len; - atomic_set(&stop_machine_first, 1); - wrote_text = 0; - stop_machine(stop_machine_multibyte_poke, (void *)&tpp, NULL); - return addr; -} - /* Replace a breakpoint (int3) with a relative jump. */ int __kprobes arch_optimize_kprobe(struct optimized_kprobe *op) { @@ -1418,7 +1369,8 @@ int __kprobes arch_optimize_kprobe(struct optimized_kprobe *op) jmp_code[0] = RELATIVEJUMP_OPCODE; *(s32 *)(&jmp_code[1]) = rel; - __multibyte_poke(op->kp.addr, jmp_code, RELATIVEJUMP_SIZE); + text_poke_fixup(op->kp.addr, jmp_code, RELATIVEJUMP_SIZE, + op->optinsn.insn); return 0; } @@ -1430,7 +1382,7 @@ void __kprobes arch_unoptimize_kprobe(struct optimized_kprobe *op) /* Set int3 to first byte for kprobes */ buf[0] = BREAKPOINT_INSTRUCTION; memcpy(buf + 1, op->optinsn.copied_insn, RELATIVE_ADDR_SIZE); - __multibyte_poke(op->kp.addr, buf, RELATIVEJUMP_SIZE); + text_poke_fixup(op->kp.addr, buf, RELATIVEJUMP_SIZE, op->optinsn.insn); } static int __kprobes setup_detour_execution(struct kprobe *p, -- Masami Hiramatsu Software Engineer Hitachi Computer Products (America), Inc. Software Solutions Division e-mail: mhiramat@redhat.com