nightly checkout and testing script

nightly checkout and testing script

[William Cohen]

[-- Attachment #1: Type: text/plain, Size: 651 bytes --]

To make sure systemtap tests are run on a regular basis I have set up 
cron jobs on my local machines (FC4/Rawhide and a couple RHEL4-U3 beta 
machines) to check out the current systemtap sources and tests, 
configure them, build them, and run the tests nightly at 4:30am Eastern 
Time. The attached simple-minded script is what is run. The script 
expects that there is a copy of the elfutils sources around in the 
appropriate place. The script sets up a clean directory to do this work 
in, systemtap_${date_time}

The script doesn't send report back the results yet. That is the next step.

Any comments on the script would be appreciated.

-Will

[-- Attachment #2: stap_testing --]
[-- Type: text/plain, Size: 1553 bytes --]

#! /bin/bash
#
# A simple script to automate checkup and running of the tests
#
#
# Will Cohen
# 20060116

#variable definitions

export DATE_TIME=`date +%Y%m%d%H%M`
export SYSTEMTAP=~/systemtap_${DATE_TIME}
export ELFUTILS=~/rh-rpms/BUILD/elfutils-0.118
export STAP_OBJ=${SYSTEMTAP}/obj
export STAP_SRC=${SYSTEMTAP}/src
export LD_LIBRARY_PATH=${STAP_OBJ}/lib-elfutils:${STAP_OBJ}/lib-elfutils/systemtap:$LD_LIBRARY_PATH
export PATH=${STAP_OBJ}:$PATH
export SYSTEMTAP_TAPSET=${STAP_SRC}/tapset
export SYSTEMTAP_RUNTIME=${STAP_SRC}/runtime
export STAP_TESTS=${SYSTEMTAP}/tests
export CVSROOT=:pserver:anoncvs@sources.redhat.com:/cvs/systemtap

check_error()
{
  if test $1 != 0; then
    echo $2
    exit $1
  fi
}


#check out src and test directory
mkdir -p ${STAP_OBJ}
cd ${SYSTEMTAP}
#echo "anonymous\n" > cvs login
cvs co src
check_error $? "problem checking out src"
cvs co tests
check_error $? "problem checking out tests"

#configure and build
cd ${STAP_OBJ}
${STAP_SRC}/configure --with-elfutils=${ELFUTILS}
check_error $? "problem configure"
make gcov >& ${DATE_TIME}.problems
check_error $? "problem building stap"

# do the runtime tests with test coverage
cd ${STAP_TESTS}/testsuite
mkdir -p ./coverage
#lcov --directory $STAP_OBJ --zerocounters
runtest --tool=systemtap
#lcov --directory $STAP_OBJ --capture --output-file ./coverage/stap.info
#genhtml -o ./coverage ./coverage/stap.info

# FIXME Extract the bits of information and report the results
# Pull out the FAILS and summary from the testsuite
# list architecture
# send the email

Re: nightly checkout and testing script

[William Cohen]

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

I have made a number of revisions to the testing script. It nows sends a 
summary of the test results. I just have it sending me the email, but it 
would be trivial to have the results be mailed to the systemtap mailing 
list. Do people want me to set this to automatically send the test 
results to the systemtap mailing list? Could we set this up on ppc64 and 
ia64 machines to keep watch of how well the translator is working for them.

I have a crontab job like the following set up to run the tests nightly 
in the early morning:

$ crontab -l
30 04 * * * /home/wcohen/bin/stap_testing > /tmp/stap_testing.log

-Will

[-- Attachment #2: stap_testing --]
[-- Type: text/plain, Size: 2350 bytes --]

#! /bin/bash
#
# A simple script to automate checkup and running of the tests
#
#
# Will Cohen
# 20060116

#variable definitions

export DATE_TIME=`date -u +%Y%m%d%H%M`
export USER=`whoami`
export KERNEL=`uname -s -r -v -m -p -i -o`
export TEST_NAME=stap_testing_${DATE_TIME}
export SYSTEMTAP=~/${TEST_NAME}
export ELFUTILS=~/rh-rpms/BUILD/elfutils-0.119
export STAP_OBJ=${SYSTEMTAP}/obj
export STAP_SRC=${SYSTEMTAP}/src
export LD_LIBRARY_PATH=${STAP_OBJ}/lib-elfutils:${STAP_OBJ}/lib-elfutils/systemtap:$LD_LIBRARY_PATH
export PATH=${STAP_OBJ}:$PATH
export SYSTEMTAP_TAPSET=${STAP_SRC}/tapset
export SYSTEMTAP_RUNTIME=${STAP_SRC}/runtime
export STAP_TESTS=${SYSTEMTAP}/tests
export CVSROOT=:pserver:anoncvs@sources.redhat.com:/cvs/systemtap
export MAIL_TO_ADDR=wcohen@redhat.com
export BUILD_RESULTS=${STAP_OBJ}/${DATE_TIME}.problems
export TEST_RESULTS=/tmp/${TEST_NAME}.results

check_error()
{
  if test $1 != 0; then
    echo $2
    exit $1
  fi
}


#check out src and test directory
mkdir -p ${STAP_OBJ}
cd ${SYSTEMTAP}
echo "anonymous\n" > cvs login
cvs co src
check_error $? "problem checking out src"
cvs co tests
check_error $? "problem checking out tests"

#configure and build
cd ${STAP_OBJ}
${STAP_SRC}/configure --with-elfutils=${ELFUTILS}
check_error $? "problem configure"
make gcov >& ${BUILD_RESULTS}
check_error $? "problem building stap"

# do the runtime tests with test coverage
cd ${STAP_TESTS}/testsuite
mkdir -p ./coverage
#lcov --directory $STAP_OBJ --zerocounters
runtest --tool=systemtap
#lcov --directory $STAP_OBJ --capture --output-file ./coverage/stap.info
#genhtml -o ./coverage ./coverage/stap.info

# Extract the bits of information and report the results
# list architecture
# Pull out the FAILS for translator and summary from the testsuite

echo "Date: ${DATE_TIME}" > ${TEST_RESULTS}
echo "User: ${USER}" >> ${TEST_RESULTS}
echo "Kernel: ${KERNEL}" >> ${TEST_RESULTS}
echo -e "\nStap translator summary of failed tests" >> ${TEST_RESULTS}
cd ${STAP_OBJ}; find -name "*.err" -exec basename {} .err \;  >> ${TEST_RESULTS}
echo -e "\nTestsuite summary of failed tests" >> ${TEST_RESULTS}
cd ${STAP_TESTS}/testsuite
grep "^FAIL:" systemtap.sum >> ${TEST_RESULTS} 
awk '/=== systemtap Summary ===/,EOF' systemtap.log >> ${TEST_RESULTS}

# send the email
cat ${TEST_RESULTS} | mail -s "${TEST_RESULTS}" $MAIL_TO_ADDR