From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 12679 invoked by alias); 28 Jan 2009 16:59:02 -0000 Received: (qmail 12665 invoked by uid 22791); 28 Jan 2009 16:59:00 -0000 X-SWARE-Spam-Status: No, hits=-0.1 required=5.0 tests=AWL,BAYES_50,J_CHICKENPOX_45,KAM_MX,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: sourceware.org Received: from mx1.redhat.com (HELO mx1.redhat.com) (66.187.233.31) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 28 Jan 2009 16:58:56 +0000 Received: from int-mx1.corp.redhat.com (int-mx1.corp.redhat.com [172.16.52.254]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id n0SGwsOZ018159 for ; Wed, 28 Jan 2009 11:58:54 -0500 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx1.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n0SGwtUo004435 for ; Wed, 28 Jan 2009 11:58:55 -0500 Received: from [10.16.2.60] (dhcp-100-2-60.bos.redhat.com [10.16.2.60]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n0SGwqcx023937; Wed, 28 Jan 2009 11:58:54 -0500 Message-ID: <49808EEF.1020700@redhat.com> Date: Wed, 28 Jan 2009 17:58:00 -0000 From: Masami Hiramatsu User-Agent: Thunderbird 2.0.0.19 (X11/20090105) MIME-Version: 1.0 To: Mathieu Desnoyers CC: Nick Piggin , LKML , Ananth N Mavinakayanahalli , Jim Keniston , systemtap-ml , "Frank Ch. Eigler" Subject: Re: [BUG][kprobes][vunmap?]: kprobes may cause memory corruption References: <497FC3B1.7050805@redhat.com> <497FE895.1080708@redhat.com> <20090128154824.GA6025@Krystal> In-Reply-To: <20090128154824.GA6025@Krystal> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2009-q1/txt/msg00282.txt.bz2 Mathieu Desnoyers wrote: > * Masami Hiramatsu (mhiramat@redhat.com) wrote: >> Masami Hiramatsu wrote: > Hi Masami, > > This would not surprise me if it came from bug in the new vmap() > implementation done in this commit : > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=db64fe02258f1507e13fe5212a989922323685ce > > Especially because going from vmap -> vm_map_ram makes this behavior > disappear. > > Looking at the commit, I notice that it delays vunmap so it's done in > batch to minimize locking effect. I think it would be good to create a > test case to try to isolate this, without any kprobes/text_poke > involved, which does something like this : > > load module (this is also doing vmalloc, so it might be part of the > problem) > for i (i=0; i < 400; i++) { > vmap() > vfree() ^^^^^ vunmap? > } > unload module > > Another interesting test would be : > > for i (i=0; i < 400; i++) { > vmalloc() > vfree() > } Hi Mathieu, Thank you for test ideas. I made both of above two tests and run it. Both test modules do NOT cause memory corruption... > All this called in a loop. This would help isolating the "vmap" part of > the issue. If this test is not enough, then we should maybe try > something like this in a kernel module (which does what text_poke does > with vmalloc, more or less) in a loop : > > char somedata[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); > char copydata[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); Should both of them have PAGE_SIZE*2? > > void test_vmap(void) > } > struct page *pages[2]; > char *vaddr; > int i; > > for (i = 0; i < 2 * PAGE_SIZE; i++) > copydata[i] = somedata[i]; > page[0] = virt_to_page(&somedata); > BUG_ON(!page[0]); > page[1] = virt_to_page(&somedata + PAGE_SIZE); > BUG_ON(!page[1]); > vaddr = vmap(pages, 2, VM_MAP, PAGE_KERNEL); > BUG_ON(!vaddr); > > for (i = 0; i < 2 * PAGE_SIZE; i++) > vaddr[i] = copydata[i] + 1; > > vunmap(vaddr); > > for (i = 0; i < 2 * PAGE_SIZE; i++) > BUG_ON(somedata[i] != copydata[i] + 1); > } Hmm, when I ran above code, it hit the last BUG_ON(). I checked that somedata[i] didn't updated. > Given you don't seem to have hit the > for (i = 0; i < len; i++) > BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]); > test at the end of text_poke, However, when I ran kprobe-based test, it doesn't hit the BUG_ON() in text_poke(). > I suspect the write through the vmapped > area is correctly done, but that the problem may lay in the mm layer. > Maybe it's running out of pre-allocated vmap areas or something like > this ? I haven't seen vmalloc failure message on 2.6.29-rc2. Thank you again, -- Masami Hiramatsu Software Engineer Hitachi Computer Products (America) Inc. Software Solutions Division e-mail: mhiramat@redhat.com