Systmetap Testsuite results and areas to fix

Systmetap Testsuite results and areas to fix

[William Cohen]

Overall on x86_64 RHEL8/RHEL9/F38/rawhide the systemtap testsuite
complete.  There are still on the order of 1500 failures on the x86_64
machine, which is more than desired.

I went looking for failures/issues in the testsuite that cause multiple
failures in the the test results.  For example previously fixed
runtime to avoid conflicts with added glibc functions to allow dyninst
code to build without error on Fedora rawhide and turn off gcc warning
about string ops for the syscall tests to allow the test to be built.

- A number of the syscall test fail because the syscall string
  argument is not properly fetched and printed.  This appears to be a
  large portion of the 700 syscall.exp failures.


- About 200 of the "listing_mode" tests fail.  They all reported:

  received: "child process exited abnormally"

  Going to investigate this more closely.
  

- About 150 of the tracepoints_list tests fail.  Most are reporting
  "not found by systemtap" but a few ('dlm', sof', and 'sof_int') fail
  compilation because headers are not found.


- Approximately 130 failures in the tapset_functions.exp tests.
  This looks like an issue with the test itself rather than
  systemtap.  A lot of messages of the form:
  
  FAIL: tapset_functions_stress (list_values(unknown))
  FAIL: tapset_functions_stress (default_value(unknown))

  Think this might have been by conscious language initiative changes
  (d1804e051dd) around november 2020.  Going to check to see if
  systemtap-4.4 built before these patches has the
  systemtap.stress/tapset_functions.exp test work.

-Will

Re: Systmetap Testsuite results and areas to fix

[Martin Cermak]

On  Thu  2023-11-02  13:48 , William Cohen via Systemtap wrote:
> Overall on x86_64 RHEL8/RHEL9/F38/rawhide the systemtap testsuite
> complete.  There are still on the order of 1500 failures on the x86_64
> machine, which is more than desired.
> 
> I went looking for failures/issues in the testsuite that cause multiple
> failures in the the test results.  For example previously fixed
> runtime to avoid conflicts with added glibc functions to allow dyninst
> code to build without error on Fedora rawhide and turn off gcc warning
> about string ops for the syscall tests to allow the test to be built.
> 
> - A number of the syscall test fail because the syscall string
>   argument is not properly fetched and printed.  This appears to be a
>   large portion of the 700 syscall.exp failures.
> 
> 
> - About 200 of the "listing_mode" tests fail.  They all reported:
> 
>   received: "child process exited abnormally"
> 
>   Going to investigate this more closely.
>   
> 
> - About 150 of the tracepoints_list tests fail.  Most are reporting
>   "not found by systemtap" but a few ('dlm', sof', and 'sof_int') fail
>   compilation because headers are not found.
> 
> 
> - Approximately 130 failures in the tapset_functions.exp tests.
>   This looks like an issue with the test itself rather than
>   systemtap.  A lot of messages of the form:
>   
>   FAIL: tapset_functions_stress (list_values(unknown))
>   FAIL: tapset_functions_stress (default_value(unknown))

I've looked into this one.  I've found some issues there, but
none of them appeared to be a systemtap bug.  After checking with
Frank on the channel, however, I've dropped this testcase entirely.

m.

> 
>   Think this might have been by conscious language initiative changes
>   (d1804e051dd) around november 2020.  Going to check to see if
>   systemtap-4.4 built before these patches has the
>   systemtap.stress/tapset_functions.exp test work.
> 
> -Will
> 

Re: Systmetap Testsuite results and areas to fix

[William Cohen]

On 11/3/23 10:58, Martin Cermak wrote:
> On  Thu  2023-11-02  13:48 , William Cohen via Systemtap wrote:
>> Overall on x86_64 RHEL8/RHEL9/F38/rawhide the systemtap testsuite
>> complete.  There are still on the order of 1500 failures on the x86_64
>> machine, which is more than desired.
>>
>> I went looking for failures/issues in the testsuite that cause multiple
>> failures in the the test results.  For example previously fixed
>> runtime to avoid conflicts with added glibc functions to allow dyninst
>> code to build without error on Fedora rawhide and turn off gcc warning
>> about string ops for the syscall tests to allow the test to be built.
>>
>> - A number of the syscall test fail because the syscall string
>>   argument is not properly fetched and printed.  This appears to be a
>>   large portion of the 700 syscall.exp failures.
>>
>>
>> - About 200 of the "listing_mode" tests fail.  They all reported:
>>
>>   received: "child process exited abnormally"
>>
>>   Going to investigate this more closely.

I tracked this down to DWARF5 call site information being different than DWARF4 (pr31028).  A fix has been pushed upstream for this and should eliminate these (and some other failures).

>>   
>>
>> - About 150 of the tracepoints_list tests fail.  Most are reporting
>>   "not found by systemtap" but a few ('dlm', sof', and 'sof_int') fail
>>   compilation because headers are not found.
>>
>>
>> - Approximately 130 failures in the tapset_functions.exp tests.
>>   This looks like an issue with the test itself rather than
>>   systemtap.  A lot of messages of the form:
>>   
>>   FAIL: tapset_functions_stress (list_values(unknown))
>>   FAIL: tapset_functions_stress (default_value(unknown))
> 
> I've looked into this one.  I've found some issues there, but
> none of them appeared to be a systemtap bug.  After checking with
> Frank on the channel, however, I've dropped this testcase entirely.
> 
> m.

The elimination of tapset_functions.exp test and addition of the DWARF5 callee will reduce the number of FAILs significantly.


-Will>>
>>   Think this might have been by conscious language initiative changes
>>   (d1804e051dd) around november 2020.  Going to check to see if
>>   systemtap-4.4 built before these patches has the
>>   systemtap.stress/tapset_functions.exp test work.
>>
>> -Will
>>
> 

Re: Systmetap Testsuite results and areas to fix

[William Cohen]

On 11/3/23 10:58, Martin Cermak wrote:
> On  Thu  2023-11-02  13:48 , William Cohen via Systemtap wrote:
>> Overall on x86_64 RHEL8/RHEL9/F38/rawhide the systemtap testsuite
>> complete.  There are still on the order of 1500 failures on the x86_64
>> machine, which is more than desired.
>>
>> I went looking for failures/issues in the testsuite that cause multiple
>> failures in the the test results.  For example previously fixed
>> runtime to avoid conflicts with added glibc functions to allow dyninst
>> code to build without error on Fedora rawhide and turn off gcc warning
>> about string ops for the syscall tests to allow the test to be built.
>>
>> - A number of the syscall test fail because the syscall string
>>   argument is not properly fetched and printed.  This appears to be a
>>   large portion of the 700 syscall.exp failures.
>>
>>
>> - About 200 of the "listing_mode" tests fail.  They all reported:
>>
>>   received: "child process exited abnormally"
>>
>>   Going to investigate this more closely.

I tracked this down to DWARF5 call site information being different than DWARF4 (pr31028).  A fix has been pushed upstream for this and should eliminate these (and some other failures).

>>   
>>
>> - About 150 of the tracepoints_list tests fail.  Most are reporting
>>   "not found by systemtap" but a few ('dlm', sof', and 'sof_int') fail
>>   compilation because headers are not found.
>>
>>
>> - Approximately 130 failures in the tapset_functions.exp tests.
>>   This looks like an issue with the test itself rather than
>>   systemtap.  A lot of messages of the form:
>>   
>>   FAIL: tapset_functions_stress (list_values(unknown))
>>   FAIL: tapset_functions_stress (default_value(unknown))
> 
> I've looked into this one.  I've found some issues there, but
> none of them appeared to be a systemtap bug.  After checking with
> Frank on the channel, however, I've dropped this testcase entirely.
> 
> m.

The elimination of tapset_functions.exp test and addition of the DWARF5 callee will reduce the number of FAILs significantly.


-Will>>
>>   Think this might have been by conscious language initiative changes
>>   (d1804e051dd) around november 2020.  Going to check to see if
>>   systemtap-4.4 built before these patches has the
>>   systemtap.stress/tapset_functions.exp test work.
>>
>> -Will
>>
>