* Labrat reports for 2.6.27-RC snapshot tests
@ 2008-10-24 7:06 Wenji Huang
2008-10-24 11:35 ` Mark Wielaard
0 siblings, 1 reply; 4+ messages in thread
From: Wenji Huang @ 2008-10-24 7:06 UTC (permalink / raw)
To: SystemTAP; +Cc: ZANNONI,ELENA, VAN HEES,KRIS
Hi,
The following is the test report generated by Labrat(Kris Van Hee). It
can track the changing of some tests result done on 2.6.27-rc.
N/A means no previous result or timeout and may be out of consideration.
We mainly focus on PASS->FAIL.
Brief Analysis:
* process-all-probes.stp fails because users process probing is
included but without utrace support in kernel.
* Some syscall tests failed lack of 32 library on 64 hosts
or the wrapper of syscall will mess the output.
* Some of them are already fixed, such as nodwfXX.stp
* Some need specific CONFIG/code like uprobes.exp, bz6850.exp.
In the future, we will do the same thing on 2.6.28-RC and find the
possible regression.
Regards,
Wenji
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc4-snapshot)
Previous build: 2008/08/14 07:18:32 - 2008/08/14 07:18:32
Current build: 2008/08/21 18:53:30 - 2008/08/21 18:53:30
Host: rctest-32
Platform: Linux 2.6.27-rc4 i386 (EL4U6) (before: Linux 2.6.27-rc3
i386 (EL4U6))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-32&pkg=systemtap_snapshot&date=20080821-185330
First failure: test
Test result: 789: 715 +, 35 -, 39 S, 0 E (before: 788: 716 +, 33 -,
39 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
PASS -> FAIL test -
systemtap.pass1-4/buildok.exp(buildok/process-all-probes.stp)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit uid16)
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit trunc)
FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit signal)
FAIL -> N/A test - systemtap.base/procfs.exp(PROCFS shutdown (eof))
N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300))
N/A -> PASS test - systemtap.base/procfs.exp(PROCFS shutdown and
output)
N/A -> PASS test - systemtap.samples/syscalls2.exp(syscalls-run (102))
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit uid)
N/A -> FAIL test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow (2))
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit rt_signal)
PASS -> FAIL test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph
build)
PASS -> N/A test - systemtap.samples/syscalls2.exp(syscalls-run (101))
PASS -> FAIL test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph run)
N/A -> PASS test - systemtap.pass1-4/semok.exp(semok/thirtytwo.stp)
FAIL -> PASS test - systemtap.base/vars.exp(vars parms/locals)
PASS -> N/A test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit chmod)
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit readwrite)
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc4-snapshot)
Previous build: 2008/08/13 20:47:40 - 2008/08/13 20:47:40
Current build: 2008/08/21 11:55:56 - 2008/08/21 11:55:56
Host: rctest-64
Platform: Linux 2.6.27-rc4 x86_64 (EL5.2) (before: Linux
2.6.27-rc3 x86_64 (EL5.2))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-64&pkg=systemtap_snapshot&date=20080821-115556
First failure: test
Test result: 824: 753 +, 30 -, 41 S, 0 E (before: 820: 753 +, 26 -,
41 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
N/A -> FAIL test -
systemtap.stress/current.exp(systemtap.stress/current.stp shutdown (eof))
PASS -> FAIL test -
systemtap.pass1-4/buildok.exp(buildok/process-all-probes.stp)
N/A -> PASS test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup)
N/A -> FAIL test - systemtap.syscall/syscall.exp(64-bit mount)
FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
PASS -> N/A test - systemtap.syscall/syscall.exp(mount)
PASS -> N/A test -
systemtap.base/timers.exp(systemtap.base/timers.stp shutdown and output)
N/A -> PASS test -
systemtap.stress/current.exp(systemtap.stress/current.stp load generation)
PASS -> FAIL test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph
build)
N/A -> FAIL test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph run)
N/A -> PASS test - systemtap.pass1-4/semok.exp(semok/thirtytwo.stp)
FAIL -> PASS test - systemtap.base/vars.exp(vars parms/locals)
N/A -> FAIL test - systemtap.samples/gtod.exp(gtod (1))
N/A -> FAIL test -
systemtap.base/timers.exp(systemtap.base/timers.stp shutdown (eof))
FAIL -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup (timeout))
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc6-snapshot)
Previous build: 2008/08/21 11:55:56 - 2008/08/21 11:55:56
Current build: 2008/09/10 08:04:46 - 2008/09/10 08:04:46
Host: rctest-64
Platform: Linux 2.6.27-rc6 x86_64 (EL5.2) (before: Linux
2.6.27-rc4 x86_64 (EL5.2))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-64&pkg=systemtap_snapshot&date=20080910-080446
First failure: test
Test result: 842: 760 +, 36 -, 46 S, 0 E (before: 824: 753 +, 30 -,
41 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
FAIL -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp shutdown (eof))
PASS -> N/A test - systemtap.base/bench.exp(bench (15))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf01.stp)
PASS -> N/A test - systemtap.base/timeofday.exp(timeofday test
load generation)
N/A -> PASS test - systemtap.base/cmd_parse.exp(cmd_parse13)
FAIL -> PASS test -
systemtap.pass1-4/buildok.exp(buildok/process-all-probes.stp)
N/A -> PASS test - systemtap.base/bench.exp(bench (18))
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace21.stp)
FAIL -> N/A test - systemtap.syscall/syscall.exp(64-bit mount)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace15.stp)
PASS -> N/A test - systemtap.base/timeofday.exp(timeofday test
startup)
N/A -> PASS test - systemtap.context/context.exp(backtrace of
yyy_func4)
N/A -> FAIL test - systemtap.samples/gtod.exp(gtod (0))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf03.stp)
N/A -> PASS test - systemtap.base/global_end.exp(global_end (11))
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace20.stp)
N/A -> PASS test - systemtap.syscall/syscall.exp(mount)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace18.stp)
N/A -> PASS test -
systemtap.base/timers.exp(systemtap.base/timers.stp shutdown and output)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf06.stp)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf02.stp)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace17.stp)
N/A -> PASS test -
systemtap.stress/current.exp(systemtap.stress/current.stp shutdown and
output)
PASS -> N/A test - systemtap.base/timeofday.exp(timeofday test
shutdown and output)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf05.stp)
N/A -> SKIP test - systemtap.base/utrace_p5.exp(UTRACE_P5_07 : no
kernel utrace support found)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace22.stp)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace16.stp)
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER18 : no
kernel markers present)
FAIL -> N/A test - systemtap.context/context.exp(backtrace of
yyy_func4 (1))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf04.stp)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace19.stp)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf07.stp)
N/A -> FAIL test - systemtap.base/uprobes.exp(uprobes -p4)
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER20 : no
kernel markers present)
N/A -> PASS test - systemtap.base/uprobes.exp(uprobes prep)
N/A -> FAIL test - systemtap.base/uprobes.exp(uprobes -p5 (0))
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER19 : no
kernel markers present)
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit sync)
N/A -> PASS test - systemtap.context/context.exp(print_stack of
yyy_func4)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf09.stp)
N/A -> PASS test - systemtap.base/list.exp(cmd_parse13)
FAIL -> N/A test - systemtap.context/context.exp(print_stack of
yyy_func4 (1))
FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (1))
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER21 : no
kernel markers present)
N/A -> PASS test - systemtap.base/uprobes.exp(uprobes compile)
N/A -> FAIL test -
systemtap.pass1-4/buildok.exp(buildok/per-process-syscall.stp)
FAIL -> N/A test -
systemtap.base/timers.exp(systemtap.base/timers.stp shutdown (eof))
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc7-snapshot)
Previous build: 2008/08/21 18:53:30 - 2008/08/21 18:53:30
Current build: 2008/09/24 13:30:03 - 2008/09/24 13:30:03
Host: rctest-32
Platform: Linux 2.6.27-rc7 i386 (EL4U6) (before: Linux 2.6.27-rc4
i386 (EL4U6))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-32&pkg=systemtap_snapshot&date=20080924-133003
First failure: test
Test result: 815: 737 +, 33 -, 45 S, 0 E (before: 789: 715 +, 35 -,
39 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
FAIL -> N/A test - systemtap.base/declaration.exp(empty-struct
resolve-pass was cached)
FAIL -> PASS test - systemtap.pass1-4/semok.exp(semok/nodwf03.stp)
PASS -> N/A test - systemtap.base/bench.exp(bench (15))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf01.stp)
N/A -> PASS test - systemtap.base/cmd_parse.exp(cmd_parse13)
FAIL -> PASS test -
systemtap.pass1-4/buildok.exp(buildok/process-all-probes.stp)
N/A -> PASS test - systemtap.context/context.exp(long function
arguments -- numeric --kelf --ignore-dwarf)
N/A -> PASS test - systemtap.base/bench.exp(bench (18))
N/A -> PASS test - systemtap.context/context.exp(unsigned function
arguments -- numeric --kelf --ignore-dwarf)
FAIL -> PASS test - systemtap.pass1-4/semok.exp(semok/nodwf02.stp)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit trunc)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace21.stp)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace15.stp)
N/A -> PASS test - systemtap.context/context.exp(backtrace of
yyy_func4)
FAIL -> N/A test - systemtap.context/context.exp(function
arguments -- numeric --kelf --ignore-dwarf: unexpected timeout)
FAIL -> PASS test - systemtap.pass1-4/semok.exp(semok/nodwf05.stp)
N/A -> FAIL test - systemtap.samples/gtod.exp(gtod (0))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf03.stp)
N/A -> PASS test - systemtap.base/global_end.exp(global_end (11))
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace20.stp)
N/A -> FAIL test - systemtap.base/procfs.exp(PROCFS shutdown (eof))
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace18.stp)
N/A -> FAIL test -
systemtap.pass1-4/buildok.exp(buildok/sched_test.stp (PRMS: 1155))
PASS -> N/A test - systemtap.samples/gtod.exp(gtod (300))
PASS -> N/A test - systemtap.base/procfs.exp(PROCFS shutdown and
output)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf06.stp)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf02.stp)
PASS -> FAIL test - systemtap.samples/queue_demo.exp(queue_demo)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace17.stp)
PASS -> N/A test - systemtap.samples/syscalls2.exp(syscalls-run (102))
FAIL -> PASS test - systemtap.pass1-4/semok.exp(semok/nodwf07.stp)
PASS -> N/A test -
systemtap.pass1-4/buildok.exp(buildok/sched_test.stp (PRMS 1155))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf05.stp)
N/A -> SKIP test - systemtap.base/utrace_p5.exp(UTRACE_P5_07 : no
kernel utrace support found)
N/A -> PASS test - systemtap.base/probe_list.exp(probe list is
correct)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace22.stp)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit swap)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace16.stp)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit uid)
FAIL -> N/A test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow (2))
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER18 : no
kernel markers present)
FAIL -> PASS test - systemtap.pass1-4/semok.exp(semok/nodwf06.stp)
FAIL -> N/A test - systemtap.context/context.exp(backtrace of
yyy_func4 (1))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf04.stp)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit forkwait)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit rt_signal)
N/A -> PASS test - systemtap.pass1-4/semko.exp(semko/utrace19.stp)
N/A -> SKIP test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow requires smp)
N/A -> FAIL test - systemtap.base/uprobes.exp(uprobes -p4)
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER20 : no
kernel markers present)
N/A -> PASS test - systemtap.samples/syscalls2.exp(syscalls-run (101))
N/A -> PASS test - systemtap.base/uprobes.exp(uprobes prep)
N/A -> FAIL test - systemtap.base/uprobes.exp(uprobes -p5 (0))
N/A -> PASS test - systemtap.base/declaration.exp(empty-struct
resolve-pass wasn't cached)
N/A -> PASS test - systemtap.context/context.exp(int64 function
arguments -- numeric --kelf --ignore-dwarf)
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER19 : no
kernel markers present)
N/A -> PASS test - systemtap.context/context.exp(char function
arguments -- numeric --kelf --ignore-dwarf)
PASS -> FAIL test - systemtap.syscall/syscall.exp(32-bit chmod)
N/A -> PASS test - systemtap.context/context.exp(print_stack of
yyy_func4)
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/nodwf09.stp)
N/A -> PASS test - systemtap.context/context.exp(string function
arguments -- numeric --kelf --ignore-dwarf)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit readwrite)
FAIL -> N/A test - systemtap.context/context.exp(print_stack of
yyy_func4 (1))
N/A -> SKIP test - systemtap.base/marker.exp(K_MARKER21 : no
kernel markers present)
N/A -> PASS test - systemtap.base/uprobes.exp(uprobes compile)
N/A -> FAIL test -
systemtap.pass1-4/buildok.exp(buildok/per-process-syscall.stp)
N/A -> PASS test - systemtap.context/context.exp(integer function
arguments -- numeric --kelf --ignore-dwarf)
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc7-snapshot)
Previous build: 2008/09/10 08:04:46 - 2008/09/10 08:04:46
Current build: 2008/09/24 16:48:53 - 2008/09/24 16:48:53
Host: rctest-64
Platform: Linux 2.6.27-rc7 x86_64 (EL5.2) (before: Linux
2.6.27-rc6 x86_64 (EL5.2))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-64&pkg=systemtap_snapshot&date=20080924-164853
First failure: test
Test result: 845: 764 +, 34 -, 47 S, 0 E (before: 842: 760 +, 36 -,
46 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
N/A -> FAIL test - systemtap.base/declaration.exp(empty-struct
resolve-pass was cached)
N/A -> PASS test - systemtap.base/timeofday.exp(timeofday test
load generation)
N/A -> PASS test - systemtap.base/timeofday.exp(timeofday test
startup)
N/A -> PASS test - systemtap.syscall/syscall.exp(forkwait)
PASS -> FAIL test - systemtap.samples/queue_demo.exp(queue_demo)
FAIL -> N/A test - systemtap.syscall/syscall.exp(64-bit forkwait)
N/A -> PASS test - systemtap.base/timeofday.exp(timeofday test
shutdown and output)
N/A -> PASS test - systemtap.base/probe_list.exp(probe list is
correct)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit forkwait)
FAIL -> PASS test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph
build)
N/A -> SKIP test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow requires smp)
PASS -> N/A test - systemtap.samples/syscalls1.exp(syscalls-count
(333))
PASS -> FAIL test - systemtap.pass1-4/semko.exp(semko/fortyfive.stp)
FAIL -> PASS test -
systemtap.examples/check.exp(systemtap.examples/general/para-callgraph run)
N/A -> PASS test - systemtap.samples/syscalls1.exp(syscalls-count
(328))
PASS -> N/A test - systemtap.base/declaration.exp(empty-struct
resolve-pass wasn't cached)
PASS -> N/A test -
systemtap.maps/pmap_agg_overflow.exp(pmap_agg_overflow)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit sync)
PASS -> N/A test -
systemtap.examples/check.exp(systemtap.examples/profiling/functioncallcount
run)
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc9-snapshot)
Previous build: 2008/09/24 16:48:53 - 2008/09/24 16:48:53
Current build: 2008/10/07 12:47:17 - 2008/10/07 12:47:17
Host: rctest-64
Platform: Linux 2.6.27-rc9 x86_64 (EL5.2) (before: Linux
2.6.27-rc7 x86_64 (EL5.2))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-64&pkg=systemtap_snapshot&date=20081007-124717
First failure: test
Test result: 847: 791 +, 29 -, 27 S, 0 E (before: 845: 764 +, 34 -,
47 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
N/A -> PASS test - systemtap.syscall/syscall.exp(chmod)
FAIL -> N/A test - systemtap.base/declaration.exp(empty-struct
resolve-pass was cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER16 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER02 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER16 wasn't cached)
N/A -> PASS test - systemtap.samples/syscalls1.exp(syscalls-count
(327))
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup)
FAIL -> N/A test - systemtap.syscall/syscall.exp(64-bit link)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER11 : no
kernel markers present)
N/A -> PASS test - systemtap.syscall/syscall.exp(access)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER04 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER08 : no
kernel markers present)
N/A -> FAIL test - systemtap.base/bz6850.exp(bz6850 -p5)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER18 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER15 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER06 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER01 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER10 : no
kernel markers present)
FAIL -> N/A test - systemtap.base/procfs.exp(PROCFS shutdown (eof))
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER17 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER21 wasn't cached)
N/A -> PASS test - systemtap.base/procfs.exp(PROCFS shutdown and
output)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER05 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER08 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER12 : no
kernel markers present)
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp shutdown and
output)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER09 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER05 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER04 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER11 wasn't cached)
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp load generation)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER20 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER18 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER07 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER06 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER13 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER13 : no
kernel markers present)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit access)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER20 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER09 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER01 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER15 wasn't cached)
N/A -> PASS test - systemtap.base/bz6850.exp(bz6850 compile)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER10 wasn't cached)
FAIL -> N/A test - systemtap.syscall/syscall.exp(64-bit access)
PASS -> N/A test - systemtap.samples/syscalls1.exp(syscalls-count
(328))
N/A -> PASS test - systemtap.pass1-4/semok.exp(semok/thirtythree.stp)
N/A -> PASS test - systemtap.base/declaration.exp(empty-struct
resolve-pass wasn't cached)
N/A -> PASS test - systemtap.syscall/syscall.exp(link)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER19 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER17 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER12 wasn't cached)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit chmod)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit link)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER07 : no
kernel markers present)
N/A -> FAIL test - systemtap.base/bz6850.exp(bz6850 -p4)
FAIL -> N/A test - systemtap.syscall/syscall.exp(64-bit chmod)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER14 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER19 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER21 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER14 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER02 : no
kernel markers present)
N/A -> FAIL test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup (timeout))
------------------------------------------------------------------------------
Package: systemtap_snapshot (0.7.1/0.131-2.6.27_rc9-snapshot)
Previous build: 2008/09/24 13:30:03 - 2008/09/24 13:30:03
Current build: 2008/10/07 08:38:20 - 2008/10/07 08:38:20
Host: rctest-32
Platform: Linux 2.6.27-rc9 i386 (EL4U6) (before: Linux 2.6.27-rc7
i386 (EL4U6))
URL:
http://build.alchar.org/cgi/showBuild?host=rctest-32&pkg=systemtap_snapshot&date=20081007-083820
First failure: test
Test result: 817: 759 +, 33 -, 25 S, 0 E (before: 815: 737 +, 33 -,
45 S, 0 E)
Old -> New Test
----- ----- ----------------------------------
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER16 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER02 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER16 wasn't cached)
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup)
N/A -> PASS test - systemtap.samples/syscalls1.exp(syscalls-count
(338))
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER11 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER04 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER08 : no
kernel markers present)
N/A -> FAIL test - systemtap.base/bz6850.exp(bz6850 -p5)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER18 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER15 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER06 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER01 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER10 : no
kernel markers present)
FAIL -> N/A test - systemtap.base/procfs.exp(PROCFS shutdown (eof))
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER17 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER21 wasn't cached)
N/A -> PASS test - systemtap.base/procfs.exp(PROCFS shutdown and
output)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER05 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER08 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER12 : no
kernel markers present)
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp shutdown and
output)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER09 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER05 wasn't cached)
PASS -> N/A test - systemtap.base/alternatives.exp(LOCAL1)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER04 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER11 wasn't cached)
PASS -> N/A test -
systemtap.stress/current.exp(systemtap.stress/current.stp load generation)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER20 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER18 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER07 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER06 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER13 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER13 : no
kernel markers present)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit access)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER20 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER09 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER01 : no
kernel markers present)
N/A -> FAIL test - systemtap.base/alternatives.exp(LOCAL1 (0))
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER15 wasn't cached)
N/A -> PASS test - systemtap.base/bz6850.exp(bz6850 compile)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER10 wasn't cached)
N/A -> PASS test - systemtap.pass1-4/semok.exp(semok/thirtythree.stp)
PASS -> N/A test - systemtap.samples/syscalls1.exp(syscalls-count
(339))
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER19 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER17 wasn't cached)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER12 wasn't cached)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit chmod)
FAIL -> PASS test - systemtap.syscall/syscall.exp(32-bit link)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER07 : no
kernel markers present)
N/A -> FAIL test - systemtap.base/bz6850.exp(bz6850 -p4)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER14 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER19 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER21 : no
kernel markers present)
N/A -> PASS test - systemtap.base/marker.exp(K_MARKER14 wasn't cached)
SKIP -> N/A test - systemtap.base/marker.exp(K_MARKER02 : no
kernel markers present)
N/A -> FAIL test -
systemtap.stress/current.exp(systemtap.stress/current.stp startup (timeout))
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Labrat reports for 2.6.27-RC snapshot tests
2008-10-24 7:06 Labrat reports for 2.6.27-RC snapshot tests Wenji Huang
@ 2008-10-24 11:35 ` Mark Wielaard
2008-10-27 17:02 ` Kris Van Hees
0 siblings, 1 reply; 4+ messages in thread
From: Mark Wielaard @ 2008-10-24 11:35 UTC (permalink / raw)
To: wenji.huang; +Cc: SystemTAP, ZANNONI,ELENA, VAN HEES,KRIS
Hi Wenji,
On Fri, 2008-10-24 at 15:00 +0800, Wenji Huang wrote:
> In the future, we will do the same thing on 2.6.28-RC and find the
> possible regression.
This is useful. Thanks. If you could post such an overview whenever a
new rc comes out for 2.6.28 that would be really helpful.
There is some noise in the output because the script seems to not
understand some of the error messages when things fail. e.g.
FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
[...]
N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300))
It would be useful to keep either the kernel version or the systemtap
snapshot version constant between runs. It looks like currently both are
updated at the same time. Although it isn't completely clear, the
version is always reported as 0.7.1, it would be good to mention the
last git commit (as in stap -V).
Still, having this automated is much better than how I currently do it
by hand and then just grep through the log files to see what happened.
Do you have an overview "base run" against 2.6.27 final?
Thanks,
Mark
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Labrat reports for 2.6.27-RC snapshot tests
2008-10-24 11:35 ` Mark Wielaard
@ 2008-10-27 17:02 ` Kris Van Hees
2008-10-28 12:14 ` Mark Wielaard
0 siblings, 1 reply; 4+ messages in thread
From: Kris Van Hees @ 2008-10-27 17:02 UTC (permalink / raw)
To: Mark Wielaard; +Cc: wenji.huang, SystemTAP, ZANNONI,ELENA, VAN HEES,KRIS
On Fri, Oct 24, 2008 at 01:34:26PM +0200, Mark Wielaard wrote:
<< snip>>
> There is some noise in the output because the script seems to not
> understand some of the error messages when things fail. e.g.
>
> FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
> [...]
> N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300))
>
> It would be useful to keep either the kernel version or the systemtap
> snapshot version constant between runs. It looks like currently both are
> updated at the same time. Although it isn't completely clear, the
> version is always reported as 0.7.1, it would be good to mention the
> last git commit (as in stap -V).
This is actually not something that can easily be solved without hardcoding
test-specific logic to strip out dynamic content in these messages. Since
labrat is not a project-specific reporting system, implementing such hardcoded
rules would be less desirable. Also note that wiring this logic in a way
that is not test specific is somewhat dangerous because you may end stripping
out important information from the messages unless people are extremely
careful in how they write the pass/fail messages.
If there is a very strict convention (one that gets enforced) on how dynamic
data is included in these messages, I can definitely look at implementing the
logic to strip it out based on the established conventions.
Kris
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Labrat reports for 2.6.27-RC snapshot tests
2008-10-27 17:02 ` Kris Van Hees
@ 2008-10-28 12:14 ` Mark Wielaard
0 siblings, 0 replies; 4+ messages in thread
From: Mark Wielaard @ 2008-10-28 12:14 UTC (permalink / raw)
To: Kris Van Hees; +Cc: wenji.huang, SystemTAP, ZANNONI,ELENA
Hi Kris,
On Mon, 2008-10-27 at 13:01 -0400, Kris Van Hees wrote:
> On Fri, Oct 24, 2008 at 01:34:26PM +0200, Mark Wielaard wrote:
> << snip>>
> > There is some noise in the output because the script seems to not
> > understand some of the error messages when things fail. e.g.
> >
> > FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0))
> > [...]
> > N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300))
> >
> > It would be useful to keep either the kernel version or the systemtap
> > snapshot version constant between runs. It looks like currently both are
> > updated at the same time. Although it isn't completely clear, the
> > version is always reported as 0.7.1, it would be good to mention the
> > last git commit (as in stap -V).
>
> This is actually not something that can easily be solved without hardcoding
> test-specific logic to strip out dynamic content in these messages. Since
> labrat is not a project-specific reporting system, implementing such hardcoded
> rules would be less desirable. Also note that wiring this logic in a way
> that is not test specific is somewhat dangerous because you may end stripping
> out important information from the messages unless people are extremely
> careful in how they write the pass/fail messages.
>
> If there is a very strict convention (one that gets enforced) on how dynamic
> data is included in these messages, I can definitely look at implementing the
> logic to strip it out based on the established conventions.
"extra" output is always between brackets. You can filter on that.
You can get the git commit number with:
$ git rev-parse HEAD | sed -n 's/^\(.\{8\}\).*/\1/p'
You can get the git branch with:
$ git symbolic-ref HEAD | sed -n 's|^refs/heads/||p'
See also the git git_version.sh. That should all be pretty generic.
Cheers,
Mark
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2008-10-28 12:14 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-10-24 7:06 Labrat reports for 2.6.27-RC snapshot tests Wenji Huang
2008-10-24 11:35 ` Mark Wielaard
2008-10-27 17:02 ` Kris Van Hees
2008-10-28 12:14 ` Mark Wielaard
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).