From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 26715 invoked by alias); 14 Apr 2010 16:18:44 -0000 Received: (qmail 26677 invoked by uid 22791); 14 Apr 2010 16:18:34 -0000 X-SWARE-Spam-Status: No, hits=1.1 required=5.0 tests=BAYES_50,TW_CZ,TW_DP,TW_XJ,TW_XZ,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from e23smtp01.au.ibm.com (HELO e23smtp01.au.ibm.com) (202.81.31.143) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 14 Apr 2010 16:18:29 +0000 Received: from d23relay04.au.ibm.com (d23relay04.au.ibm.com [202.81.31.246]) by e23smtp01.au.ibm.com (8.14.3/8.13.1) with ESMTP id o3EGGAQt007769 for ; Thu, 15 Apr 2010 02:16:10 +1000 Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay04.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o3EGBvci1679502 for ; Thu, 15 Apr 2010 02:11:57 +1000 Received: from d23av02.au.ibm.com (loopback [127.0.0.1]) by d23av02.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o3EGIOLC009404 for ; Thu, 15 Apr 2010 02:18:25 +1000 Received: from apj.in.ibm.com ([9.77.202.215]) by d23av02.au.ibm.com (8.14.3/8.13.1/NCO v10.0 AVin) with ESMTP id o3EGIMZp009394; Thu, 15 Apr 2010 02:18:23 +1000 Message-ID: <4BC5EACD.9070306@linux.vnet.ibm.com> Date: Wed, 14 Apr 2010 22:32:00 -0000 From: Anithra P Janakiraman User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4pre) Gecko/20090922 Fedora/3.0-3.9.b4.fc12 Thunderbird/3.0b4 MIME-Version: 1.0 To: "Frank Ch. Eigler" CC: "systemtap@sourceware.org" Subject: Re: [RFC] Framework for easy distribution of SystemTap scripts (V4) References: <4B1FE14D.5070307@linux.vnet.ibm.com> <4B685611.5090402@linux.vnet.ibm.com> <4BA7D773.5050106@linux.vnet.ibm.com> <4BBC4EA1.9040608@linux.vnet.ibm.com> <4BBF536D.60908@linux.vnet.ibm.com> <20100409163949.GE28054@redhat.com> <4BC4A701.30801@linux.vnet.ibm.com> <20100413205600.GC2897@redhat.com> In-Reply-To: <20100413205600.GC2897@redhat.com> Content-Type: multipart/mixed; boundary="------------010100080600020700080805" X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2010-q2/txt/msg00122.txt.bz2 This is a multi-part message in MIME format. --------------010100080600020700080805 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-length: 3146 On 04/14/2010 02:26 AM, Frank Ch. Eigler wrote: > Hi - > > >>> Why *not* install it? How are people supposed to get a hold of your >>> script if it doesn't get installed? >> >> This script will be used by a very select set of users - say admins who >> wish to debug a customer machine, don't think it would be right to >> install it as part of the systemtap/systemtap-runtime rpms as it is not >> going to be used by the 'regular' stap user. It can be obtained from the >> source. > > Well, if you are targeting this to such a specific subset of users, > then perhaps there is little need to include the script within the > systemtap source tree. It could be as easily downloaded from the wiki > for example, no? That should be ok too.. > > >>> (There still appears some ps -ef | grepping in the new version; probably >>> all of that is unnecessary, if you save child pids properly.) >> >> With the -start option, staprun executes in the background while the >> script exits. To stop users need to run the package with -stop, it would >> not be possible to store the child pids in this scenario. > > (If you used the normal flight-recorder mode startup/shutdown options, > the pids wouldn't be needed, since the module name is used as the > key.) > We are assuming that SystemTap might not be installed on the machine where the package will be executed. We package the bare minimal (staprun/stapio) needed to run the module. The initscripts might not be available. > >> [...] >>> To have a separate configuration file for this only, you'd need to >>> argue why some options have to be treated differently from others. >> >> The problem here is that we are not aware of the options or the no of >> options before hand. This would vary from script to script. > > I guess the help text is not specific as to what these option strings > actually do. They appear to be made available to the hand-coded > template file as shell variables, which happens to use $time. How > else are these options conceivably used? Not staprun module-options; > not apparently post-processing-script environment variables. The config file and parameters should have been passed to the post-processing script. I had completely overlooked that. Thanks for pointing it out. Attached the modified template. The time options is used by the wrapper/template script and the rest are passed to the post-processing script. The help text would be specific to what the options strings do. For eg: if the config file had a default: ipaddress=127.0.0.1 The help text would contain: ipaddress=[ip] ip is a standard IPv4 address. The output will be filtered for this address The resulting package would have the following --help Options: options explained Parameters: time=[x] x is in minutes. Runs the script for x minutes. valid for --run(-r) o --start(-s) or --all(-a) options only ipaddress=[ip] ip is a standard IPv4 address. The output will be filtered for this address If an ipaddress is specified at the command line when executing the module it would be passed to the post-processing script. Thanks, Anithra --------------010100080600020700080805 Content-Type: text/plain; name="distributionframework_v4.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="distributionframework_v4.patch" Content-length: 12292 diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg new file mode 100644 index 0000000..443f841 --- /dev/null +++ b/scripts/build_pkg/README.stap-buildpkg @@ -0,0 +1,70 @@ +SystemTap distribution framework + +This is free software. +See the COPYING file for redistribution/modification terms. + +Introduction: + +The SystemTap distribution framework aims to: + +1. packages a set of systemtap scripts in a format that circumvents as much as +possible, external dependencies like kernel-debuginfo. +2. generate a package that will self-extract, run the script, and post +process the output with minimal intervention from users. + +Usage: + +stap-buildpkg scriptfile [options] [stap-options] +Options: + -n pkg_name : specify a package name + -v pkg_version : specify a package version + -p script : post processing script + -c pkg_config : provide config parameters through a file + -h help_config : help text for config parameters + All additonal options will be passed to the stap command + +pkg_name and pkg_version will be used to set the name and version of the +resulting executable. For the -p script option, script is a file name, whose +contents will be copied into the new package. It will be executed post +completion of the staprun command. The post processing script can be used to +process the output any which way the user choses. This could include filtering +the output,offering suggestions based on the output data, or even forwarding the +output to a support team. +An optional config file that contains post_processing script specific config +options and corresponding help entries can also be specified using the -c and +-h options respectively.For eg: If a script expects the pid and ip address as +options the pkg_config file could contain the following: +pid=5632 +ip=127.0.0.1 +Note: These options are the 'default' options and can be overridden through +the command line when the resulting package is executed. These options will be +passed to post_processing script to filter the output and not to the +stap/staprun command. The help_config contains help entries for any +additional parameters. Eg: +ipaddress=[ip] ip is a standard IPv4 address. +pid=[pid] pid of the process to be monitored. +The help text would be appended to the usage/help instructions of the +resulting package. +The resulting package is a self-extracting binary. When invoked it would untar +itself, execute the script(by invoking staprun) and post process. + +The usage for the the resulting package: + + [options] [parameters] + +Options: + * --run -r Runs the scripts + for x minutes where x can be passed as a parameter + in the form time=[x]. The default value is 10 minutes. + Post processing is performed after the script completes. + * --start -s Invokes the script as a background process. + * --stop -x Stops the script and performs post processing. + * --all Same as --run + * --help Displays this usage text. + +Parameters: + + * time=[x] x is in minutes. Runs the script for x minutes. valid + for --run(-r) o --start(-s) or --all(-a) options only + Any other parameter specified in the help_config. + diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg new file mode 100755 index 0000000..61ba597 --- /dev/null +++ b/scripts/build_pkg/stap-buildpkg @@ -0,0 +1,192 @@ +##!/bin/bash +# +# Script to auto-generate executable packages from stap scripts +# + +prog=stap-buildpkg + +echo_usage () { + echo $"Usage: $prog scriptfile [options] [stap-options]" + echo $"Options:" + echo $" -n pkg_name : specify a package name" + echo $" -v pkg_version : specify a package version" + echo $" -c pkg_config : provide config parameters through a file" + echo $" -p post_process script : post processing script" + echo $" -h help_config : help text for config parameters" + echo $" All additonal options will be passed to the stap command" +} + +parse_args() { +while [ -n "$1" ]; do + case $1 in + -n) PKG_NAME=$2 + shift 2 + echo $PKG_NAME + ;; + -v) PKG_VERSION=$2 + shift 2 + echo $PKG_VERSION + ;; + -c) PKG_INPUT_CONFIG=$2 + echo $PKG_INPUT_CONFIG + shift 2 + ;; + -p) PKG_POST_SCRIPT=$2 + echo $PKG_POST_SCRIPT + shift 2 + ;; + -h) PKG_HELP_CONFIG=$2 + echo $PKG_HELP_CONFIG + shift 2 + ;; + *) STAP_OPTIONS=$@ + echo $STAP_OPTIONS + shift $# +esac +done +} + +validate_args() { + if test -z $PKG_NAME; then + PKG_NAME=`basename $script .stp` + fi + if test -z $PKG_VERSION; then + PKG_VERSION=1.0 + fi + if test -z $PKG_RELEASE; then + PKG_RELEASE="0" + fi + PKG_ARCH=`uname -i` +} + +module_gen() { +stap -p4 -m $PKG_NAME.ko $script $STAP_OPTIONS +if [ -f $PKG_NAME.ko ] +then + echo " Module $PKG_NAME.ko generated successfully "; +else + echo " Module $PKG_NAME.ko generation failed. Exiting now... " + rm -rf $TEMPDIR + exit; +fi +} + +create_wrappers() { +echo "Creating install wrapper from template.."; +echo "s/TEMPLATE_PKG_NAME/$PKG_NAME/g" > $TEMPDIR/sd-installer-patterns +echo "s/TEMPLATE_PKG_VERSION/$PKG_VERSION/g" >> $TEMPDIR/sd-installer-patterns +echo "s/TEMPLATE_PKG_RELEASE/$PKG_RELEASE/" >> $TEMPDIR/sd-installer-patterns +echo "s/TEMPLATE_ARCH/$PKG_ARCH/g" >> $TEMPDIR/sd-installer-patterns + +sed -f $TEMPDIR/sd-installer-patterns template.buildpkg > $TEMPDIR/wrapper +csplit -s $TEMPDIR/wrapper '/extractor template/' +mv xx00 ${PKG_NAME}_install +mv xx01 ${PKG_NAME}_binextractor + +chmod a+x ${PKG_NAME}_install +rm $TEMPDIR/sd-installer-patterns +rm $TEMPDIR/wrapper +} + +copy_files_to_temp() { + +runtime=systemtap-runtime +if test -z $(rpm -qa $runtime); then + runtime=systemtap +fi + +echo "using systemtap runtime executables from $runtime rpm" + +cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/. +cp `rpm -ql $runtime | grep stapio` $TEMPDIR/. + +echo "Copying Systemtap module.." +mv $PKG_NAME.ko $TEMPDIR/ +echo "Copying config file.." +if [ -f $PKG_INPUT_CONFIG ] && [ "$PKG_INPUT_CONFIG" != "" ] +then + cp $PKG_INPUT_CONFIG $TEMPDIR/$PKG_NAME.config +else + echo " " > $TEMPDIR/$PKG_NAME.config +fi +# Copying help file, or adding an empty file if none is provided. +if [ -f $PKG_HELP_CONFIG ] && [ "$PKG_HELP_CONFIG" != "" ] +then + cp $PKG_HELP_CONFIG $TEMPDIR/$PKG_NAME.help +else + echo " " > $TEMPDIR/$PKG_NAME.help +fi +echo "Copying the post processing script" +# copying the post processing script, or adding an empty file if none is provided. +if [ -f $PKG_POST_SCRIPT ] && [ "$PKG_POST_SCRIPT" != "" ] +then + cp $PKG_POST_SCRIPT $TEMPDIR/$PKG_NAME.post +else + echo " " > $TEMPDIR/$PKG_NAME.post +fi +chmod +x $TEMPDIR/$PKG_NAME.post +mv ${PKG_NAME}_install $TEMPDIR/ +} + +#------------------------------------------------------------------ +# Mainline script +#------------------------------------------------------------------ +script=$1 +echo $script +shift 1 + +if test -z $script; then + echo "Please provide a script name" + echo_usage + exit +fi + +if [ ! -f "$script" ]; then + echo "Script file does not exist" + echo "Please provide a valid script name" + echo_usage + exit +fi + + +#OPTS=`getopt -s bash -u -o 'n:v:R:c:p:h:' -- $@` + +#parse arguments +parse_args $@ +validate_args + +# generate a tmp directory to store all components:remember to clean it up ! +TEMPDIR=/tmp/$PKG_NAME-$PKG_VERSION; +mkdir $TEMPDIR + +# Generate Stap module +echo "Generating Stap module.."; +module_gen + +#generate installer(wrapper) scripts +create_wrappers + +#copy systemtap-runtime files +copy_files_to_temp + +MY_PWD=`pwd` +cd $TEMPDIR ; tar -cjf $PKG_NAME-$PKG_VERSION.tar.bz2 * +cd $MY_PWD; + +echo "Building the self-extracting binary.." +mkdir $TEMPDIR/$PKG_NAME +mv $TEMPDIR/${PKG_NAME}_install $TEMPDIR/$PKG_NAME/ +tar -xjf $TEMPDIR/$PKG_NAME-$PKG_VERSION.tar.bz2 -C $TEMPDIR/$PKG_NAME/ +mv ${PKG_NAME}_binextractor $TEMPDIR/${PKG_NAME}_binextractor 2>/dev/null + +MY_PWD=`pwd` +cd $TEMPDIR; +tar -czf - $PKG_NAME/ >> ${PKG_NAME}_binextractor +cd $MY_PWD; +mv $TEMPDIR/${PKG_NAME}_binextractor ./$PKG_NAME-$PKG_VERSION +echo "The installer is at `pwd`/$PKG_NAME-$PKG_VERSION" +chmod +x $PKG_NAME-$PKG_VERSION + +#Cleanup the mess in temp dir +rm -rf $TEMPDIR +exit diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg new file mode 100644 index 0000000..0bb790e --- /dev/null +++ b/scripts/build_pkg/template.buildpkg @@ -0,0 +1,117 @@ +#!/bin/bash +RUN_DIR=`pwd` +time=2m + +function run() +{ +echo "Gathering data... Please wait" +# Log file appears as systemtap_tcpipstat.out +TS=`date "+%Y%m%d-%H:%M:%S"` +export SYSTEMTAP_STAPIO=$RUN_DIR/stapio +$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 & +stappid=$! +sleep $time +echo "Completed" +mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out +echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory" +kill -SIGINT $stappid +echo "Postprocessing..." +$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out $RUN_DIR/TEMPLATE_PKG_NAME.config $postoptions +rm $RUN_DIR/staprun +rm $RUN_DIR/stapio +} + +function start() +{ + # check to see if tapset is installed +echo "Gathering data..." +export SYSTEMTAP_STAPIO=$RUN_DIR/stapio +$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out 2>&1 & +echo "Run TEMPLATE_PKG_NAME --stop to stop the script and process output" +} + +function stop() +{ +TS=`date "+%Y%m%d-%H:%M:%S"` +echo "Stopping systemtap script" +ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT +echo "Done" +mv $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out +echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory" +echo "Postprocessing..." +$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out $RUN_DIR/TEMPLATE_PKG_NAME.config $postoptions +rm $RUN_DIR/staprun +rm $RUN_DIR/stapio +} + +function helptext() +{ + echo " " + echo "USAGE: TEMPLATE_PKG_NAME [options] [parameters]" + echo " " + echo "Options: " + echo " --run -r Runs the scripts for x minutes where x can be passed as a parameter using time=[x]. " + echo " The default value is 10 minutes. Post processing is performed after the script completes." + echo " --start -s Runs the script as a background process" + echo " --stop -x Stops the script and performs post processing" + echo " --all Runs the script and processes the output" + echo " --help Displays this usage text" + echo " " + echo "Parameters: " + echo " " + echo " time=[x] x is in minutes. Runs the script for x minutes. valid for --run(-r)" + echo " --start(-s) or --all(-a) options only" + cat $RUN_DIR/TEMPLATE_PKG_NAME.help + echo "" +} + + +option=$1 +shift + + + postoptions=$@ + + for i in $@ + do + tmpa=${1%%=*} + if [ "$tmpa" == "time" ] + then + tmpb=${1#$tmpa=} + export $tmpa=$tmpb + break + fi + shift + done + +case $option in + -r | --run ) + run + exit + ;; + -s | --start ) + start + exit + ;; + -h | --help ) helptext + exit + ;; + -x | --stop ) stop + exit + ;; + -a | --all ) run + exit + ;; + * ) helptext + exit 1 +esac + +## extractor template +#!/bin/sh -e +sed -e '1,/^exit$/d' "$0" | tar -xzf - +cd TEMPLATE_PKG_NAME + ./TEMPLATE_PKG_NAME_install $@ +cd .. +rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME.* +rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME_install +exit --------------010100080600020700080805--