Improving the "make installcheck" results for the syscalls test

Improving the "make installcheck" results for the syscalls test

[William Cohen]

Hi,

Been going through the syscall test result to see what can be done to
improve the syscall test results.  There were 359 failures on the
x86_64 RHEL9 guest VM I tried things out on.  There seems to be four
different ways that the syscall tests are failing:

1) Pointers rather than actual strings printed
2) Incomplete output from the instrumenttion scripts
3) Some fields in struct timespec not being printed properly
4) Matching issue (for example 4th arg printed as hex, but matching expects decimal for preadv


For the pointers in place of strings

The reason for systemtap printing the pointer rather than the actual
string is that memory page is not available when the systemtap probe
fires.  This typically happens for the syscall tests that have strings
stored as read only constants.  When the initial syscall with a
pointer to the read only constant is made that particular area of
memory is not yet mapped in on entry to the syscall.  However, the
kernel does map in the page with the string, but that occurs after the
systemtap probe handler for syscall entry runs.

A work around for this behavior is to have some code do a sprintf of
the string constant in the dummy buffer before the syscall is made.
This causes the string constants to be mapped in before the syscalls
are made.  I have implements this on about 20 of syscall test driver
programs.  There looks to be about 90 that have strings in them.


Incomplete output

When reviewing the systemtap.log output to detemine why tests failed I
found a number of tests where output was truncated or parts of the
output were dropped.  The places where things were dropped varied between runs.
Would see things like:


spawn -ignore SIGHUP gcc /home/wcohen/systemtap_write/systemtap/testsuite/systemtap.syscall/domainname.c -m64 -lrt -Wno-stringop-overflow -lm -o /home/wcohen/systemtap_write/systemtap/testsuite/staptestv2on6W/domainname

domainname FAILED. output of "staprun -R /home/wcohen/systemtap_write/systemtap/testsuite/syscall_longlonglong.ko -c /home/wcohen/systemtap_write/systemtap/testsuite/staptestv2on6W/domainname" was:
------------------------------------------
stapio: rt_sigaction (SIGUSR1, {SIG_DFL}, 0x0, 8) = WARNING: There were 1 transport failures. Try stap -s to increase the buffer size from 16.
------------------------------------------
RESULTS: ('*' = MATCHED EXPECTED)
--------- EXPECTED and NOT MATCHED ----------
^domainname: setdomainname \(0x[f]+, [\-0-9]+\) = -[\-0-9]+
^domainname: setdomainname \("([a-zA-Z0-9\.-]+|\(none\))", -1\) = -[\-0-9]+
FAIL: 64-bit domainname syscall

A number of the tests seem to stop having output before the driver
program is run.  Also have cases like the following where the return
value and carrige return aren't printed:


msg_queue: msgget (-1, IPC_CREAT|IPC_EXCL|0600) = msg_queue: msgsnd (-1, 0x404060, 136, 0x0) = -22 (EINVAL)

Which tests this incomplete output occurs on appears to vary between runs.


Printing of struct timestruct field apppear to be wrong for some syscalls

One example of the questionable output is for the futex syscall test.
The diagnostic output shows matching the futex syscalls until it gets
to calls that have fields with floating point values.  The output has
values of 0.000000000, but expecting 0.000000789

*futex: futex (0x7f9e546dd000, FUTEX_WAKE, 1) = 1
futex: wait4 (23149, 0x7ffe4ecdd328, 0x0, 0x0) = 23149
futex: munmap (0x7f9e546dd000, 4) = 0
*futex: futex (0xffffffffffffffff, FUTEX_WAKE, 1) = -22 (EINVAL)
*futex: futex (0x0, 0xffffffff, 1) = -38 (ENOSYS)
*futex: futex (0x0, FUTEX_CMP_REQUEUE, -1, 0, 0x0, 0) = -22 (EINVAL)
*futex: futex (0x0, FUTEX_CMP_REQUEUE, 1, 18446744073709551615, 0x0, 0) = -22 (EINVAL)
*futex: futex (0x0, FUTEX_CMP_REQUEUE, 1, 0, 0xffffffffffffffff, 0) = -14 (EFAULT)
*futex: futex (0x0, FUTEX_CMP_REQUEUE, 1, 0, 0x0, -1) = -14 (EFAULT)
*futex: futex (0x0, FUTEX_WAKE_BITSET, 1, 0xffffffffffffffff) = -14 (EFAULT)
futex: futex (0x0, FUTEX_WAIT, 1, [0.000000000]) = -14 (EFAULT)
futex: futex (0x0, FUTEX_WAIT_BITSET, 1, [0.000000000], 0xbeef) = -14 (EFAULT)
futex: futex (0x0, FUTEX_REQUEUE, 1, 2, 0x0) = -14 (EFAULT)
futex: futex (0x0, FUTEX_CMP_REQUEUE, 1, 2, 0x3, -1) = -14 (EFAULT)
futex: futex (0x0, FUTEX_WAKE_OP, 1, 2, 0x3, {FUTEX_OP_SET, 2, FUTEX_OP_CMP_EQ, 1}) = -14 (EFAULT)
futex: futex (0x0, FUTEX_WAIT_REQUEUE_PI, 1, [0.000000000], 0x0) = -22 (EINVAL)
futex: exit_group (0) = 
--------- EXPECTED and NOT MATCHED ----------
^futex: futex \(0x0, FUTEX_WAIT, 1, \[0.000000789\]\) = [\-0-9]+
^futex: futex \(0x0, FUTEX_WAIT_BITSET, 1, \[0.000000789\], 0xbeef\) = [\-0-9]+
^futex: futex \(0x0, FUTEX_REQUEUE, 1, 2, 0x0\) = [\-0-9]+
^futex: futex \(0x0, FUTEX_CMP_REQUEUE, 1, 2, 0x3, -1\) = [\-0-9]+
^futex: futex \(0x0, FUTEX_WAKE_OP, 1, 2, 0x3, {FUTEX_OP_SET, 2, FUTEX_OP_CMP_EQ, 1}\) = [\-0-9]+
^futex: futex \(0x0, FUTEX_WAIT_REQUEUE_PI, 1, \[0.000000789\], 0x0\) = [\-0-9]+
FAIL: 64-bit futex syscall



Matching issues

For a few of the syscall tests there appears to be missmatches between
what is outputed and what is expected.  For example in preadv the
preadv2 syscall prints out the 4th argument in hexadecimal but the
matching expects a decimal quantity.


-Will