From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 4494 invoked by alias); 11 Dec 2015 17:02:26 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 1856 invoked by uid 89); 11 Dec 2015 17:02:24 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.0 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Fri, 11 Dec 2015 17:02:22 +0000 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id A1A7DA37C3; Fri, 11 Dec 2015 17:02:21 +0000 (UTC) Received: from [10.13.129.159] (dhcp129-159.rdu.redhat.com [10.13.129.159]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tBBH2L9X020538; Fri, 11 Dec 2015 12:02:21 -0500 Subject: Re: Recent aarch64 kprobes and uprobes patch systemtap testing To: David Long , systemtap@sourceware.org References: <5669DF98.3030601@redhat.com> <5669EABE.7040507@linaro.org> Cc: Pratyush Anand From: William Cohen Message-ID: <566B019D.1000309@redhat.com> Date: Fri, 11 Dec 2015 17:02:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <5669EABE.7040507@linaro.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-IsSubscribed: yes X-SW-Source: 2015-q4/txt/msg00268.txt.bz2 On 12/10/2015 04:12 PM, David Long wrote: > On 12/10/2015 03:24 PM, William Cohen wrote: >> Hi All, >> >> Dave Long and Pratyush Anand have been working on kprobe and uprobe patches for aarch64. I have built a local version the uprobe/upstream_arm64_devel branch of https://github.com/pratyushanand/linux which includes those patches in a linux-4.4.0-rc3 kernel. >> >> The tests seemed to run fairly well and the results have been uploaded to dejazilla: >> >> https://web.elastic.org/~dejazilla/viewsummary.php?summary=%3D%27%3C56698DCC.3090207%40redhat.com%3E%27 >> >> === systemtap Summary === >> >> # of expected passes 6096 >> # of unexpected failures 111 >> # of unexpected successes 2 >> # of expected failures 333 >> # of unknown successes 2 >> # of known failures 89 >> # of untested testcases 97 >> # of unsupported tests 27 >> runtest completed at Thu Dec 10 05:54:32 2015 >> >> >> There are still some areas needing for for aarch64 such as stack backtrace support. >> >> The following failure looks suspect because the child process died: >> >> >> spawn stap -g ./systemtap.examples/process/threadstacks.stp -Gsize=65536 -c /root/systemtap_write/systemtap/testsuite/pthread_stacks.x 1024 0 -d /root/systemtap_write/systemtap/testsuite/pthread_stacks.x >> >> pthread_stacks.x: ./systemtap.base/pthread_stacks.c:67: main: Assertion `rc == 0' failed. >> >> WARNING: Child process exited with signal 6 (Aborted) >> >> pthread_stacks.[3567] overwrote __default_stacksize@0x3ffb3be4338 (8388608->65536) >> >> WARNING: /root/systemtap_write/install/bin/staprun exited with status: 1 >> >> Pass 5: run failed. [man error::pass5] >> >> FAIL: pthread_stacks -Gsize (0 0) >> >> The fslatency-nd and fsslower-nd tests need further investigation: >> >> PASS: ./systemtap.examples/lwtools/fslatency-nd build >> meta taglines 'test_installcheck: stap fslatency-nd.stp 1 1' tag 'test_installcheck' value 'stap fslatency-nd.stp 1 1' >> attempting command stap fslatency-nd.stp 1 1 >> OUT ERROR: read fault [man error::fault] at 0x0000000000000034 (addr) near operator '@cast' at fslatency-nd.stp:66:15 >> Tracing FS sync reads and writes... Output every 1 secs. >> WARNING: Number of errors: 1, skipped probes: 1 >> WARNING: /root/systemtap_write/install/bin/staprun exited with status: 1 >> Pass 5: run failed. [man error::pass5] >> child process exited abnormally >> RC 1 >> FAIL: ./systemtap.examples/lwtools/fslatency-nd run >> >> PASS: ./systemtap.examples/lwtools/fsslower-nd build >> meta taglines 'test_installcheck: stap fsslower-nd.stp -c "sleep 1"' tag 'test_installcheck' value 'stap fsslower-nd.stp -c "sleep 1"' >> attempting command stap fsslower-nd.stp -c "sleep 1" >> OUT ERROR: read fault [man error::fault] at 0x0000000000000034 (addr) near operator '@cast' at fsslower-nd.stp:68:15 >> Tracing FS sync reads and writes slower than 10 ms... Hit Ctrl-C to end. >> TIME PID COMM FUNC SIZE LAT(ms) >> WARNING: Number of errors: 1, skipped probes: 1 >> WARNING: /root/systemtap_write/install/bin/staprun exited with status: 1 >> Pass 5: run failed. [man error::pass5] >> child process exited abnormally >> RC 1 >> FAIL: ./systemtap.examples/lwtools/fsslower-nd run >> >> >> Also a number of network tests failed like the following >> >> >> TEST PWD=/root/systemtap_write/systemtap/testsuite/systemtap.examples/network >> meta taglines 'test_check: stap -g -p4 netfilter_drop.stp TCP 1' tag 'test_check' value 'stap -g -p4 netfilter_drop.stp TCP 1' >> attempting command stap -g -p4 netfilter_drop.stp TCP 1 >> OUT /tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.c:2731:1: error: initialization from incompatible pointer type [-Werror] >> .hook = enter_netfilter_probe_0, >> ^ >> /tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.c:2731:1: error: (near initialization for 'netfilter_opts_0.hook') [-Werror] >> /tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.c:2732:1: error: unknown field 'owner' specified in initializer >> .owner = THIS_MODULE, >> ^ >> /tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.c:2732:1: error: initialization from incompatible pointer type [-Werror] >> /tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.c:2732:1: error: (near initialization for 'netfilter_opts_0.dev') [-Werror] >> cc1: all warnings being treated as errors >> make[4]: *** [/tmp/stapbIEqFl/stap_0c0db8521e3ea3b2d26dcde208a77baf_21082_src.o] Error 1 >> make[3]: *** [_module_/tmp/stapbIEqFl] Error 2 >> WARNING: kbuild exited with status: 2 >> Pass 4: compilation failed. [man error::pass4] >> child process exited abnormally >> RC 1 >> FAIL: ./systemtap.examples/network/netfilter_drop build >> >> -Will > > > Cool. Wish I could make sense of systemtap error messages. Hi Dave, This was a data dump, so I haven't made sense of some of it either. :) The ".hook=..." and ".owner=..." are problems in the systemtap code generation for newer kernel and don't concern the aarch64 kprobes/uprobes work. The read faults for fslatency-nd.stp and fsslower-nd,stp need to be check more carefully. but they are likely issues with systemtap (they do work on linux-4.2.0 on x86_64). The "FAIL: pthread_stacks -Gsize (0 0)" looks like it could be an issue with uprobes affecting the running of the program. Pratyush are you able to run this systemtap test locally? > > At Will Deacon's suggested I tested probing the instruction in __copy_to_user that can cause a captured kernel exception when an application passes in a bad buffer address. Unfortunately the result was a hang. So copy_to/from user is going to have to be blacklisted for now, unless there turns out to be a simple fix. I'm worried there might be other places in the kernel where an otherwise probeable instruction might be expected to generate an exception. > > -dl So the problem is the issue of nested exceptions in the single step debug exception? Are there other place in the kernel where similar exceptions could occur, such as memory management code probing an address to verify it is valid? -Will