Re: Segmenation fault for systemtap tests using the arm uprobes support

Re: Segmenation fault for systemtap tests using the arm uprobes support

[William Cohen]

On 10/29/2013 04:50 PM, David Long wrote:
> OK, I've seen your error once now.  I guess I just need to run the test a number of times looking for it.  I am investigating.
> 
> -dl
> 

Hi David,

I finally got the uprobe-v2 branch of https://git.linaro.org/gitweb?p=people/davelong/linux.git;a=summary working on my samsung arm chromebook (needed to use a newer exynos5250-snow.dtb file). I reran the at_var test with the newer kernel and do not see the sigsegv.

I am going to rerun the test tests with this kernel and see if the problem went away with the newer version of the patches.

-Will

Re: Segmenation fault for systemtap tests using the arm uprobes support

[David Long]

On 11/05/13 15:36, William Cohen wrote:
> On 10/29/2013 04:50 PM, David Long wrote:
>> OK, I've seen your error once now.  I guess I just need to run the test a number of times looking for it.  I am investigating.
>>
>> -dl
>>
>
> Hi David,
>
> I finally got the uprobe-v2 branch of https://git.linaro.org/gitweb?p=people/davelong/linux.git;a=summary working on my samsung arm chromebook (needed to use a newer exynos5250-snow.dtb file). I reran the at_var test with the newer kernel and do not see the sigsegv.
>
> I am going to rerun the test tests with this kernel and see if the problem went away with the newer version of the patches.
>
> -Will
>


I'm still seeing the SEGV with my latest patches on V3.12 on a Panda.

-dl

Re: Segmenation fault for systemtap tests using the arm uprobes support

[William Cohen]

On 11/05/2013 07:02 PM, David Long wrote:
> On 11/05/13 15:36, William Cohen wrote:
>> On 10/29/2013 04:50 PM, David Long wrote:
>>> OK, I've seen your error once now.  I guess I just need to run the test a number of times looking for it.  I am investigating.
>>>
>>> -dl
>>>
>>
>> Hi David,
>>
>> I finally got the uprobe-v2 branch of https://git.linaro.org/gitweb?p=people/davelong/linux.git;a=summary working on my samsung arm chromebook (needed to use a newer exynos5250-snow.dtb file). I reran the at_var test with the newer kernel and do not see the sigsegv.
>>
>> I am going to rerun the test tests with this kernel and see if the problem went away with the newer version of the patches.
>>
>> -Will
>>
> 
> 
> I'm still seeing the SEGV with my latest patches on V3.12 on a Panda.
> 
> -dl
> 

Hi David,

With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:

FAIL: 32_BIT_UTRACE_SYSCALL_ARGS 

I have uploaded the test results of the systemtap run with the uprobes-v2 branch to the systemtap dejazilla system:

web.elastic.org/~dejazilla/viewsummary.php?summary=%3D%27%3C527A58AC.9000301%40redhat.com%3E%27

dejazilla allows comparison between different systemtap.sum files.  I looked through the list of recent arm7l runs:

https://web.elastic.org/~dejazilla/viewsummary.php?_offset=0&_limit=20&_sort=1A&summary=&age=&rg=&tool=&variant=%3D%27armv7l-unknown-linux-gnueabihf%27&versions=&pass=&fail=&kpass=&kfail=&xpass=&xfail=&untested=&unresolved=&unsupported=&warning=&error=

The following URL is a comparison between the older uprobes and uprobes-v2 runs:

https://web.elastic.org/~dejazilla/viewrgdiff.php?rg1=150513&rg2=822815&_sort=0A&_limit=1000

Some of the like minidebuginfo and process_by_cmd2.stp passed with the uprobes-v2 kernel.


When searching for ways to test uprobes support I cam across https://github.com/rabinv/uprobes-test . Are these tests useful or have they been orphaned?

-Will

Re: Segmenation fault for systemtap tests using the arm uprobes support

[David Long]

On 11/06/13 10:24, William Cohen wrote:
>
> Hi David,
>
> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>

Are you testing on the same hardware?  It's conceivable that could make 
a difference, especially if the number of CPU's is different.

>
> When searching for ways to test uprobes support I cam across https://github.com/rabinv/uprobes-test . Are these tests useful or have they been orphaned?
>

Yes, they are very useful.  I run them all the time.  Note there are 
some expected failures late in the run, also apparently these tests do 
not test uretprobes.

-dl

Re: Segmenation fault for systemtap tests using the arm uprobes support

[William Cohen]

On 11/06/2013 10:57 AM, David Long wrote:
> On 11/06/13 10:24, William Cohen wrote:
>>
>> Hi David,
>>
>> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>>
> 
> Are you testing on the same hardware?  It's conceivable that could make a difference, especially if the number of CPU's is different.
> 

Hi David,

I am using the same machine for all the kernels, a samsung ARM chromebook.  It is the fastest ARM-based machine I have access to.  It does have 2 processor.  I am compiling the kernel with SMP support.  One thing that is different betweenthe kernels is that the newer uprobes-v2 version was compiled without LPAE or KVM support.  Would those configure options make any difference in results?

>>
>> When searching for ways to test uprobes support I cam across https://github.com/rabinv/uprobes-test . Are these tests useful or have they been orphaned?
>>
> 
> Yes, they are very useful.  I run them all the time.  Note there are some expected failures late in the run, also apparently these tests do not test uretprobes.
> 
> -dl

Great to hear that the tests are being used to exercise the kprobes.  The tests haven't changed in 9 months, so I wasn't sure if they were being used.

-Will

Re: Segmenation fault for systemtap tests using the arm uprobes support

[David Long]

On 11/06/13 11:33, William Cohen wrote:
> On 11/06/2013 10:57 AM, David Long wrote:
>> On 11/06/13 10:24, William Cohen wrote:
>>>
>>> Hi David,
>>>
>>> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>>>
>>
>> Are you testing on the same hardware?  It's conceivable that could make a difference, especially if the number of CPU's is different.
>>
>

I wouldn't expect those options to make a difference for uprobes.  They 
might for kprobes.

I just finally found the bug in ARM uretprobe's.  It's a simple fix.

-dl

Re: Segmenation fault for systemtap tests using the arm uprobes support

[William Cohen]

On 11/08/2013 01:12 PM, David Long wrote:
> On 11/06/13 11:33, William Cohen wrote:
>> On 11/06/2013 10:57 AM, David Long wrote:
>>> On 11/06/13 10:24, William Cohen wrote:
>>>>
>>>> Hi David,
>>>>
>>>> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>>>>
>>>
>>> Are you testing on the same hardware?  It's conceivable that could make a difference, especially if the number of CPU's is different.
>>>
>>
> 
> I wouldn't expect those options to make a difference for uprobes.  They might for kprobes.
> 
> I just finally found the bug in ARM uretprobe's.  It's a simple fix.
> 
> -dl
> 

Hi Dave,

Are there updated ARM uprobe-v3 patches somewhere?  I would be happy to try them. out.

-Will

Re: Segmenation fault for systemtap tests using the arm uprobes support

[David Long]

On 11/21/13 21:11, William Cohen wrote:
> On 11/08/2013 01:12 PM, David Long wrote:
>> On 11/06/13 11:33, William Cohen wrote:
>>> On 11/06/2013 10:57 AM, David Long wrote:
>>>> On 11/06/13 10:24, William Cohen wrote:
>>>>>
>>>>> Hi David,
>>>>>
>>>>> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>>>>>
>>>>
>>>> Are you testing on the same hardware?  It's conceivable that could make a difference, especially if the number of CPU's is different.
>>>>
>>>
>>
>> I wouldn't expect those options to make a difference for uprobes.  They might for kprobes.
>>
>> I just finally found the bug in ARM uretprobe's.  It's a simple fix.
>>
>> -dl
>>
>
> Hi Dave,
>
> Are there updated ARM uprobe-v3 patches somewhere?  I would be happy to try them. out.
>
> -Will
>


Almost.  Should be ready this weekend.

-dl

Re: Segmenation fault for systemtap tests using the arm uprobes support

[David Long]

On 11/21/13 21:11, William Cohen wrote:
> On 11/08/2013 01:12 PM, David Long wrote:
>> On 11/06/13 11:33, William Cohen wrote:
>>> On 11/06/2013 10:57 AM, David Long wrote:
>>>> On 11/06/13 10:24, William Cohen wrote:
>>>>>
>>>>> Hi David,
>>>>>
>>>>> With the uprobe-v2 version I only saw one test that appeared to fail because of a sigsegv:
>>>>>
>>>>
>>>> Are you testing on the same hardware?  It's conceivable that could make a difference, especially if the number of CPU's is different.
>>>>
>>>
>>
>> I wouldn't expect those options to make a difference for uprobes.  They might for kprobes.
>>
>> I just finally found the bug in ARM uretprobe's.  It's a simple fix.
>>
>> -dl
>>
>
> Hi Dave,
>
> Are there updated ARM uprobe-v3 patches somewhere?  I would be happy to try them. out.
>
> -Will
>


Hi,

I just now pushed V3 of the uprobes changes to my repo on the linaro 
website:

git://git.linaro.org/people/davelong/linux.git

It's the uprobes-v3 branch.  It's now based on V3.13-RC1.  You need at 
least that recent a kernel as Oleg has pushed some uprobes changes of 
his and mine that these patches depend on.

Thanks,
-dl

Re: Segmenation fault for systemtap tests using the arm uprobes support

[William Cohen]

On 11/26/2013 12:25 AM, David Long wrote:
 
> Hi,
> 
> I just now pushed V3 of the uprobes changes to my repo on the linaro website:
> 
> git://git.linaro.org/people/davelong/linux.git
> 
> It's the uprobes-v3 branch.  It's now based on V3.13-RC1.  You need at least that recent a kernel as Oleg has pushed some uprobes changes of his and mine that these patches depend on.
> 
> Thanks,
> -dl
> 

Hi Dave,

Thanks for the revised version of the patches.

I built a version of uprobes-v3 kernel on the Samsung ARM cromebook.  It booted successfully and is running the systemtap "make installcheck".

-Will

Results for arm uprobes-v3 posted

[William Cohen]

On 11/26/2013 04:06 PM, William Cohen wrote:
> On 11/26/2013 12:25 AM, David Long wrote:
>  
>> Hi,
>>
>> I just now pushed V3 of the uprobes changes to my repo on the linaro website:
>>
>> git://git.linaro.org/people/davelong/linux.git
>>
>> It's the uprobes-v3 branch.  It's now based on V3.13-RC1.  You need at least that recent a kernel as Oleg has pushed some uprobes changes of his and mine that these patches depend on.
>>
>> Thanks,
>> -dl
>>
> 
> Hi Dave,
> 
> Thanks for the revised version of the patches.
> 
> I built a version of uprobes-v3 kernel on the Samsung ARM cromebook.  It booted successfully and is running the systemtap "make installcheck".
> 
> -Will
> 

Hi Dave and Naresh,

I was able to run systemtap tests last night.  These results are posted in dejazilla:

https://web.elastic.org/~dejazilla/viewsummary.php?summary=%3D%27%3C5296108F.2020903%40redhat.com%3E%27

The at_var_cu.exp test looks like it might need some closer examination it got a Segmentation fault. The process_by_cmd.stp, process_by_cmd2.stp,32_BIT_UTRACE_SYSCALL_ARGS startup, and global_var-m32 tests also seemed to fail in similar way.  Below is the output from the systemtap.log for at_var_cu.exp test:

Running ./systemtap.base/at_var.exp ...
Executing on host: gcc ./systemtap.base/at_var.c  -O2 -g  -lm   -o at_var    (timeout = 300)
spawn -ignore SIGHUP gcc ./systemtap.base/at_var.c -O2 -g -lm -o at_var

executing: stap ./systemtap.base/at_var.stp -c ./at_var
FAIL: at_var
line 11: expected "user_int(&$foo->bar): 40"
Got "WARNING: Child process exited with signal 11 (Segmentation fault)"
    "WARNING: /run/media/wcohen/wasteland/wcohen/systemtap_write/install/bin/staprun exited with status: 1"
    "Pass 5: run failed.  [man error::pass5]"
testcase ./systemtap.base/at_var.exp completed in 15 seconds
Running ./systemtap.base/at_var_cu.exp ...
Executing on host: gcc ./systemtap.base/at_var_cu_1.c ./systemtap.base/at_var_cu_2.c ./systemtap.base/at_var_cu_3.c  -O2 -g  -lm   -o /run/media/wcohen/wasteland/wcohen/systemtap_write/systemtap/testsuite/at_var_cu    (timeout = 300)
spawn -ignore SIGHUP gcc ./systemtap.base/at_var_cu_1.c ./systemtap.base/at_var_cu_2.c ./systemtap.base/at_var_cu_3.c -O2 -g -lm -o /run/media/wcohen/wasteland/wcohen/systemtap_write/systemtap/testsuite/at_var_cu

executing: stap ./systemtap.base/at_var_cu.stp -c ./at_var_cu /run/media/wcohen/wasteland/wcohen/systemtap_write/systemtap/testsuite/at_var_cu
FAIL: at_var_cu
line 13: expected "bah: @var("counter", @1): 8"
Got "bah: @var("counter", @1): 3"
    "bah: @var("counter@at_var_cu_2.c", @1): 3"
    "bah: @var("counter@at_var_cu_3.c", @1): 0"
    "bah: @var("counter@at_var_cu*.c", @1): 3"
    "bah': @var("counter@at_var_cu*.c"): 3"
    "bah': @var("main_global"): 5"
testcase ./systemtap.base/at_var_cu.exp completed in 12 seconds


-Will

Re: Results for arm uprobes-v3 posted

[David Long]

On 11/27/13 11:23, William Cohen wrote:
> On 11/26/2013 04:06 PM, William Cohen wrote:
>> On 11/26/2013 12:25 AM, David Long wrote:
>>
>>> Hi,
>>>
>>> I just now pushed V3 of the uprobes changes to my repo on the linaro website:
>>>
>>> git://git.linaro.org/people/davelong/linux.git
>>>
>>> It's the uprobes-v3 branch.  It's now based on V3.13-RC1.  You need at least that recent a kernel as Oleg has pushed some uprobes changes of his and mine that these patches depend on.
>>>
>>> Thanks,
>>> -dl
>>>
>>
>> Hi Dave,
>>
>> Thanks for the revised version of the patches.
>>
>> I built a version of uprobes-v3 kernel on the Samsung ARM cromebook.  It booted successfully and is running the systemtap "make installcheck".
>>
>> -Will

Are these more failures than were found with the v2 patches?

-dl

Re: Results for arm uprobes-v3 posted

[William Cohen]

On 11/27/2013 11:26 AM, David Long wrote:
> On 11/27/13 11:23, William Cohen wrote:
>> On 11/26/2013 04:06 PM, William Cohen wrote:
>>> On 11/26/2013 12:25 AM, David Long wrote:
>>>
>>>> Hi,
>>>>
>>>> I just now pushed V3 of the uprobes changes to my repo on the linaro website:
>>>>
>>>> git://git.linaro.org/people/davelong/linux.git
>>>>
>>>> It's the uprobes-v3 branch.  It's now based on V3.13-RC1.  You need at least that recent a kernel as Oleg has pushed some uprobes changes of his and mine that these patches depend on.
>>>>
>>>> Thanks,
>>>> -dl
>>>>
>>>
>>> Hi Dave,
>>>
>>> Thanks for the revised version of the patches.
>>>
>>> I built a version of uprobes-v3 kernel on the Samsung ARM cromebook.  It booted successfully and is running the systemtap "make installcheck".
>>>
>>> -Will
> 
> Are these more failures than were found with the v2 patches?
> 
> -dl
> 
> 

Hi Dave,

Here is the comparison between an earlier systemtap "make installcheck" with the older set of patches: https://web.elastic.org/~dejazilla/viewrgdiff.php?rg1=903451&rg2=893427&_sort=0A&_limit=1000

The at_var_cu.exp, global var-m32, global var-m32-O process_by_cmd2.stp appear to be regressions.

-Will