Re: exercising current aarch64 kprobe support with systemtap

[William Cohen <wcohen@redhat.com>]

On 06/22/2016 11:18 PM, David Long wrote:
> On 06/22/2016 04:24 PM, William Cohen wrote:
>> Hi all,
>>
>> When running the current systemtap checked out from the git repository
>> and a locally built kernel with the kprobes64-v13 patches (the
>> test_upstream_arm64_devel branch of
>> https://github.com/pratyushanand/linux) on Fedora 23 machine one of
>> the kprobes_onthefly.exp tests is causing the machine to get in a
>> state that requires rebooting to fix.  This can be triggered by running a
>> portion of the systemtap tests with:
>>
>>   make installcheck RUNTESTFLAGS="--debug systemtap.onthefly/kprobes_onthefly.exp"
>>
>> When it gets to the kprobes_onthefly - otf_stress_max_iter_5000 test the
>> console starts spewing the following and needs to be rebooted:
>>
>> [23394.036860] Unexpected kernel single-step exception at EL1
>> [23394.042434] Unexpected kernel single-step exception at EL1
>> [23394.048008] Unexpected kernel single-step exception at EL1
>> [23394.053541] Unexpected kernel single-step exception at EL1
>> [23394.059053] Unexpected kernel single-step exception at EL1
>> [23394.064545] Unexpected kernel single-step exception at EL1
>>
>> Sorry I don't have the start of the failure it scrolled off the screen very quickly.
>>
>> -Will
>>
>>
> 
> I'll take a look and see what I can figure out.
> 
> In the meantime I did just push a v14 branch.  I'm doubtful that it will address the above problem even though it contains a few bug fixes.
> 
> -dl
> 

Hi Dave,

I have pulled the v14 branch and have it built, and gave it a try.


When running the tests on various kernels I typically see message about the hrtimer interrupt like the following during the run:

...
Running /root/systemtap_write/systemtap/testsuite/systemtap.onthefly/kprobes_onthefly.exp ...
[  197.217699] stap_ab30d3b6ca1d8e512f40980479fd57e_18018: module verification failed: signature and/or required key missing - tainting kernel
[  534.166223] Scheduler tracepoints stat_sleep, stat_iowait, stat_blocked and stat_runtime require the kernel parameter schedstats=enabled or kernel.sched_schedstats=1
[  534.205668] hrtimer: interrupt took 241848 ns
...


Hmmm, it successfully completed the run with the v14 branch four times.

# make installcheck RUNTESTFLAGS="--debug systemtap.onthefly/kprobes_onthefly.exp"
...
tail systemtap.sum
PASS: kprobes_onthefly - otf_stress_1ms_iter_50 (survived)
PASS: kprobes_onthefly - otf_stress_500us_iter_50 (survived)
PASS: kprobes_onthefly - otf_stress_100us_iter_50 (survived)
PASS: kprobes_onthefly - otf_stress_prof_iter_2000 (survived)
PASS: kprobes_onthefly - otf_stress_hard_iter_2000 (survived)
PASS: kprobes_onthefly - otf_stress_max_iter_5000 (survived)

		=== systemtap Summary ===

# of expected passes		22



What were the bug fixes and changes in v14?  Alternatively could there be something else in the kernel I was using previously (e.g. the uprobes patches).

-Will