[RFC] Framework for easy distribution of SystemTap scripts.

[RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 3054 bytes --]

Hi all,

We've been looking at a simplified distribution framework for SystemTap
scripts that :
1. packages a set of systemtap scripts and dependent
tapsets in a form that circumvents , as much as possible the need for
external dependencies(like kernel-debuginfo)
2. installs, runs the script, post processes the output and uninstalls
out of the box

We feel this would be especially useful in scenarios where support
admins wish to run a particular set of script(s) on a machine for debug
or monitoring purposes.  I'm attaching an rpm-based framework that does
the following:
1. Compiles a set of scripts into a set of kernel modules
on another machine of identical architecture that has SystemTap
installed and running
2. Creates an rpm on the fly that would consist of
i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
package in the machine that has SystemTap installed, iii) post
processing script to process the output(optional)
3. Bundles the above
rpm with an install script that has options to i) install the rpm ii)
run / stop iii) uninstall iv) all of the above.
4 . The rpm and the
install script are packaged into a self-extracting binary that would
extract itself and execute 1 of the four steps above.

The framework would mainly consist of
1. rpm-generating script
(rpm-generator.sh) that does all of the above
2. spec file template that
will be modified by the rpm-generator on the fly and used to build the
rpm
3. install script template that will be modified by the
rpm-generator and bundled with the rpm
4. Configuration file - that
would specify the location of the scripts/tapsets

The rpm is packaged as a self extracting binary for ease of use. When
the binary is executed it extracts the rpm package and based on the
parameters provided either installs/runs/uninstalls the rpm. Help
detailing the available options are also provided

USAGE: <package_name> [options] [parameters]

Options:
    * --install -i Install the tapset rpm.
    * --run -r Runs the scripts
for n minutes where n can be passed as a parameter.  o The default value
is 10 minutes. Post processing is performed after the script completes.
    * --start -s Invokes the script as a background process.
    * --stop -x Stops the script and performs post processing.
    * --uninstall -u Stops the script if running and uninstalls the rpm.
    * --all Installs the rpm, runs the scrip, processes the output and
            uninstalls the rpm.
    * --help Displays this usage text.

Parameters:

    * time=[x] x is in minutes. Runs the script for x minutes. valid for
              --run(-r) o --start(-s) or --all(-a) options only



One disadvantage of the above framework is that the end binary that is
created is tied to a particular kernel version/architecture. However the
users now need not install systemtap, debuginfo and need not even be
familiar with SystemTap. This should help users and admins who wish to
run a set of scripts to solve a specific problem(maybe on a customer's
machine).

Please comment,

Regards,
Anithra.





[-- Attachment #2: framework.tgz --]
[-- Type: application/x-compressed-tar, Size: 4261 bytes --]

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 4051 bytes --]

Hi,

I've attached a modified framework. The names of the template/config 
files have been changed(no change in functionality). I've attempted to 
keeps the names compatible with other scripts in the SystemTap package. 
We would like to keep the framework in a seperate directory (similar to 
the initscript dir) for the sake of simplicity. (called 
distribution-framework or distribution-fw?)

The directory would contain the following
template.specfile
template.install
template.binextractor
config.rpmoptions
systemtap-rpm-generator   (This is the executable script)
README


Let me know if any changes are needed (names of files/directory or 
contents). If there is any naming convention that i should adhere to 
please let me know. If there are no concerns, or suggestions i would 
like to go ahead and commit the framework.

Thanks.

Regards,
Anithra



On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
> Hi all,
>
> We've been looking at a simplified distribution framework for SystemTap
> scripts that :
> 1. packages a set of systemtap scripts and dependent
> tapsets in a form that circumvents , as much as possible the need for
> external dependencies(like kernel-debuginfo)
> 2. installs, runs the script, post processes the output and uninstalls
> out of the box
>
> We feel this would be especially useful in scenarios where support
> admins wish to run a particular set of script(s) on a machine for debug
> or monitoring purposes. I'm attaching an rpm-based framework that does
> the following:
> 1. Compiles a set of scripts into a set of kernel modules
> on another machine of identical architecture that has SystemTap
> installed and running
> 2. Creates an rpm on the fly that would consist of
> i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
> package in the machine that has SystemTap installed, iii) post
> processing script to process the output(optional)
> 3. Bundles the above
> rpm with an install script that has options to i) install the rpm ii)
> run / stop iii) uninstall iv) all of the above.
> 4 . The rpm and the
> install script are packaged into a self-extracting binary that would
> extract itself and execute 1 of the four steps above.
>
> The framework would mainly consist of
> 1. rpm-generating script
> (rpm-generator.sh) that does all of the above
> 2. spec file template that
> will be modified by the rpm-generator on the fly and used to build the
> rpm
> 3. install script template that will be modified by the
> rpm-generator and bundled with the rpm
> 4. Configuration file - that
> would specify the location of the scripts/tapsets
>
> The rpm is packaged as a self extracting binary for ease of use. When
> the binary is executed it extracts the rpm package and based on the
> parameters provided either installs/runs/uninstalls the rpm. Help
> detailing the available options are also provided
>
> USAGE: <package_name> [options] [parameters]
>
> Options:
> * --install -i Install the tapset rpm.
> * --run -r Runs the scripts
> for n minutes where n can be passed as a parameter. o The default value
> is 10 minutes. Post processing is performed after the script completes.
> * --start -s Invokes the script as a background process.
> * --stop -x Stops the script and performs post processing.
> * --uninstall -u Stops the script if running and uninstalls the rpm.
> * --all Installs the rpm, runs the scrip, processes the output and
> uninstalls the rpm.
> * --help Displays this usage text.
>
> Parameters:
>
> * time=[x] x is in minutes. Runs the script for x minutes. valid for
> --run(-r) o --start(-s) or --all(-a) options only
>
>
>
> One disadvantage of the above framework is that the end binary that is
> created is tied to a particular kernel version/architecture. However the
> users now need not install systemtap, debuginfo and need not even be
> familiar with SystemTap. This should help users and admins who wish to
> run a set of scripts to solve a specific problem(maybe on a customer's
> machine).
>
> Please comment,
>
> Regards,
> Anithra.
>
>
>
>


[-- Attachment #2: distribution-fw.tgz --]
[-- Type: application/x-compressed-tar, Size: 4970 bytes --]

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Frank Ch. Eigler]

anithra wrote:

> [...] We would like to keep the framework in a seperate directory
> (similar to the initscript dir) for the sake of simplicity. (called
> distribution-framework or distribution-fw?)

You need to think about fit-and-finish issues such as man pages & test
cases, the installation siting of these files (changes to "make
install"), and packaging issues (changes to systemtap.spec).  Source
level location is a lesser matter.

> The directory would contain the following
> template.specfile
> template.install
> template.binextractor
> config.rpmoptions
> systemtap-rpm-generator   (This is the executable script)
> README

(<bikeshed>consider renaming systemtap-rpm-generator to stap-buildrpm</>)

> Let me know if any changes are needed (names of files/directory or
> contents). If there is any naming convention that i should adhere to
> please let me know.  [...]

Have you reviewed fedora packaging guidelines?  These scripts themselves
should be installed in a generally compatible manner.  Plus the *products*
of these scripts should also be generally compatible.


> [...]
>> Options:
>> * --install -i Install the tapset rpm.
>> * --run -r Runs the scripts
>> for n minutes where n can be passed as a parameter. o The default value
>> is 10 minutes. Post processing is performed after the script completes.
>> * --start -s Invokes the script as a background process.
>> * --stop -x Stops the script and performs post processing.
>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>> * --all Installs the rpm, runs the scrip, processes the output and
>> uninstalls the rpm.
>> * --help Displays this usage text.

It seems odd to have the file created by the framework-script to both
manage installation as well as execution of the compiled systemtap
materials.  Why install if it can be run directly?  If installed, how
else to execute the systemtap module except via this file?

Perhaps the stap-buildrpm widget should create a plain rpm that is
self-explanatory to install, within which a /usr/bin/FOO shell script
manage the execution of the script.

It may be nice if stap-buildrpm interfaced with stap-initscripts, so
that rpms produced this way were easily activatable from that scheme.


- FChE

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Prerna Saxena]

On 02/03/2010 01:57 AM, Frank Ch. Eigler wrote:
>
> anithra wrote:
>
>>> Options:
>>> * --install -i Install the tapset rpm.
>>> * --run -r Runs the scripts
>>> for n minutes where n can be passed as a parameter. o The default value
>>> is 10 minutes. Post processing is performed after the script completes.
>>> * --start -s Invokes the script as a background process.
>>> * --stop -x Stops the script and performs post processing.
>>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>>> * --all Installs the rpm, runs the scrip, processes the output and
>>> uninstalls the rpm.
>>> * --help Displays this usage text.
>
> It seems odd to have the file created by the framework-script to both
> manage installation as well as execution of the compiled systemtap
> materials.  Why install if it can be run directly?  If installed, how
> else to execute the systemtap module except via this file?
>
> Perhaps the stap-buildrpm widget should create a plain rpm that is
> self-explanatory to install, within which a /usr/bin/FOO shell script
> manage the execution of the script.
>
> It may be nice if stap-buildrpm interfaced with stap-initscripts, so
> that rpms produced this way were easily activatable from that scheme.
>
>
> - FChE

A little background on this framework might be of help in understanding 
the design goals for it. The idea was to build small diagnostic 
utilities based on systemtap that could be deployed on external systems 
(which may or may not have systemtap and dependencies like debuginfo 
installed). These could be used to glean specific information about the 
system.

Systemtap-generated kernel modules, along with systemtap runtime looked 
like a good mechanism to distribute such utilities.

We decided to package the precompiled kernel modules along with 
systemtap runtime binaries in an rpm. Also, we thought that the rpm 
installation ; systemtap module insertion, and data filtering and 
logging ought to be transparent to the support admin who is running this 
-- so we packaged the rpm alongwith a wrapper shell script that 
automates all these steps for the user. This was bundled in a 
self-extracting archive for simplicity of use.

This framework can be employed to build such diagnostic utilities on the 
fly.

The 'install' option in the shell script merely installs the rpm, ie, it 
copies the systemtap kernel module and runtime in a specified location. 
The 'uninstall' option removes systemtap kernel module and other files 
installed by the rpm. The 'run' option launches staprun to insert the 
systemtap generated kernel module, employs an optional post-processing 
script to filter the data logged, stores logs in a pre-specifed log 
file, and causes the systemtap kernel module to be clean-removed. The 
'all' option does all these steps in order.

Interfacing with initscripts is a good add-on, but the primary purpose 
was to deploy such diagnostics on installations that lack systemtap and 
dependencies -- it was meant to be self-sufficient.

The framework can be enhanced if it is found useful to begin with :-)

Regards,
-- 
Prerna Saxena
  as possible
Linux Technology Centre,
IBM Systems and Technology Lab,
Bangalore, India

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 4750 bytes --]


Hi,

I've attached a modified framework. The systemtap_pkg_generator now 
generates a package that includes staprun,stapio, an install script and 
the generated kernel module. The files are not packaged into an rpm, and 
no installation happens. The resulting 'package' is a self-extracting 
binary file that extracts all files to the current dir, invokes the 
install script which in turn invokes staprun from the current dir. This 
binary file can only be executed by root users.

Please comment.

Regards,
Anithra.


On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
> Hi,
>
> I've attached a modified framework. The names of the template/config
> files have been changed(no change in functionality). I've attempted to
> keeps the names compatible with other scripts in the SystemTap package.
> We would like to keep the framework in a seperate directory (similar to
> the initscript dir) for the sake of simplicity. (called
> distribution-framework or distribution-fw?)
>
> The directory would contain the following
> template.specfile
> template.install
> template.binextractor
> config.rpmoptions
> systemtap-rpm-generator (This is the executable script)
> README
>
>
> Let me know if any changes are needed (names of files/directory or
> contents). If there is any naming convention that i should adhere to
> please let me know. If there are no concerns, or suggestions i would
> like to go ahead and commit the framework.
>
> Thanks.
>
> Regards,
> Anithra
>
>
>
> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>> Hi all,
>>
>> We've been looking at a simplified distribution framework for SystemTap
>> scripts that :
>> 1. packages a set of systemtap scripts and dependent
>> tapsets in a form that circumvents , as much as possible the need for
>> external dependencies(like kernel-debuginfo)
>> 2. installs, runs the script, post processes the output and uninstalls
>> out of the box
>>
>> We feel this would be especially useful in scenarios where support
>> admins wish to run a particular set of script(s) on a machine for debug
>> or monitoring purposes. I'm attaching an rpm-based framework that does
>> the following:
>> 1. Compiles a set of scripts into a set of kernel modules
>> on another machine of identical architecture that has SystemTap
>> installed and running
>> 2. Creates an rpm on the fly that would consist of
>> i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
>> package in the machine that has SystemTap installed, iii) post
>> processing script to process the output(optional)
>> 3. Bundles the above
>> rpm with an install script that has options to i) install the rpm ii)
>> run / stop iii) uninstall iv) all of the above.
>> 4 . The rpm and the
>> install script are packaged into a self-extracting binary that would
>> extract itself and execute 1 of the four steps above.
>>
>> The framework would mainly consist of
>> 1. rpm-generating script
>> (rpm-generator.sh) that does all of the above
>> 2. spec file template that
>> will be modified by the rpm-generator on the fly and used to build the
>> rpm
>> 3. install script template that will be modified by the
>> rpm-generator and bundled with the rpm
>> 4. Configuration file - that
>> would specify the location of the scripts/tapsets
>>
>> The rpm is packaged as a self extracting binary for ease of use. When
>> the binary is executed it extracts the rpm package and based on the
>> parameters provided either installs/runs/uninstalls the rpm. Help
>> detailing the available options are also provided
>>
>> USAGE: <package_name> [options] [parameters]
>>
>> Options:
>> * --install -i Install the tapset rpm.
>> * --run -r Runs the scripts
>> for n minutes where n can be passed as a parameter. o The default value
>> is 10 minutes. Post processing is performed after the script completes.
>> * --start -s Invokes the script as a background process.
>> * --stop -x Stops the script and performs post processing.
>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>> * --all Installs the rpm, runs the scrip, processes the output and
>> uninstalls the rpm.
>> * --help Displays this usage text.
>>
>> Parameters:
>>
>> * time=[x] x is in minutes. Runs the script for x minutes. valid for
>> --run(-r) o --start(-s) or --all(-a) options only
>>
>>
>>
>> One disadvantage of the above framework is that the end binary that is
>> created is tied to a particular kernel version/architecture. However the
>> users now need not install systemtap, debuginfo and need not even be
>> familiar with SystemTap. This should help users and admins who wish to
>> run a set of scripts to solve a specific problem(maybe on a customer's
>> machine).
>>
>> Please comment,
>>
>> Regards,
>> Anithra.
>>
>>
>>
>>
>


[-- Attachment #2: distribution-fw.tgz --]
[-- Type: application/x-compressed-tar, Size: 3095 bytes --]

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 5192 bytes --]

Hi,

Please do let me know if there are any concerns/suggestions with the 
distribution framework.
We are thinking a man page would not be necessary as usage is trivial. 
Let me know if it is needed ((README has been included).

Regards,
Anithra.




On 02/17/2010 11:39 PM, Anithra P Janakiraman wrote:
>
> Hi,
>
> I've attached a modified framework. The systemtap_pkg_generator now
> generates a package that includes staprun,stapio, an install script and
> the generated kernel module. The files are not packaged into an rpm, and
> no installation happens. The resulting 'package' is a self-extracting
> binary file that extracts all files to the current dir, invokes the
> install script which in turn invokes staprun from the current dir. This
> binary file can only be executed by root users.
>
> Please comment.
>
> Regards,
> Anithra.
>
>
> On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
>> Hi,
>>
>> I've attached a modified framework. The names of the template/config
>> files have been changed(no change in functionality). I've attempted to
>> keeps the names compatible with other scripts in the SystemTap package.
>> We would like to keep the framework in a seperate directory (similar to
>> the initscript dir) for the sake of simplicity. (called
>> distribution-framework or distribution-fw?)
>>
>> The directory would contain the following
>> template.specfile
>> template.install
>> template.binextractor
>> config.rpmoptions
>> systemtap-rpm-generator (This is the executable script)
>> README
>>
>>
>> Let me know if any changes are needed (names of files/directory or
>> contents). If there is any naming convention that i should adhere to
>> please let me know. If there are no concerns, or suggestions i would
>> like to go ahead and commit the framework.
>>
>> Thanks.
>>
>> Regards,
>> Anithra
>>
>>
>>
>> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>>> Hi all,
>>>
>>> We've been looking at a simplified distribution framework for SystemTap
>>> scripts that :
>>> 1. packages a set of systemtap scripts and dependent
>>> tapsets in a form that circumvents , as much as possible the need for
>>> external dependencies(like kernel-debuginfo)
>>> 2. installs, runs the script, post processes the output and uninstalls
>>> out of the box
>>>
>>> We feel this would be especially useful in scenarios where support
>>> admins wish to run a particular set of script(s) on a machine for debug
>>> or monitoring purposes. I'm attaching an rpm-based framework that does
>>> the following:
>>> 1. Compiles a set of scripts into a set of kernel modules
>>> on another machine of identical architecture that has SystemTap
>>> installed and running
>>> 2. Creates an rpm on the fly that would consist of
>>> i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
>>> package in the machine that has SystemTap installed, iii) post
>>> processing script to process the output(optional)
>>> 3. Bundles the above
>>> rpm with an install script that has options to i) install the rpm ii)
>>> run / stop iii) uninstall iv) all of the above.
>>> 4 . The rpm and the
>>> install script are packaged into a self-extracting binary that would
>>> extract itself and execute 1 of the four steps above.
>>>
>>> The framework would mainly consist of
>>> 1. rpm-generating script
>>> (rpm-generator.sh) that does all of the above
>>> 2. spec file template that
>>> will be modified by the rpm-generator on the fly and used to build the
>>> rpm
>>> 3. install script template that will be modified by the
>>> rpm-generator and bundled with the rpm
>>> 4. Configuration file - that
>>> would specify the location of the scripts/tapsets
>>>
>>> The rpm is packaged as a self extracting binary for ease of use. When
>>> the binary is executed it extracts the rpm package and based on the
>>> parameters provided either installs/runs/uninstalls the rpm. Help
>>> detailing the available options are also provided
>>>
>>> USAGE: <package_name> [options] [parameters]
>>>
>>> Options:
>>> * --install -i Install the tapset rpm.
>>> * --run -r Runs the scripts
>>> for n minutes where n can be passed as a parameter. o The default value
>>> is 10 minutes. Post processing is performed after the script completes.
>>> * --start -s Invokes the script as a background process.
>>> * --stop -x Stops the script and performs post processing.
>>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>>> * --all Installs the rpm, runs the scrip, processes the output and
>>> uninstalls the rpm.
>>> * --help Displays this usage text.
>>>
>>> Parameters:
>>>
>>> * time=[x] x is in minutes. Runs the script for x minutes. valid for
>>> --run(-r) o --start(-s) or --all(-a) options only
>>>
>>>
>>>
>>> One disadvantage of the above framework is that the end binary that is
>>> created is tied to a particular kernel version/architecture. However the
>>> users now need not install systemtap, debuginfo and need not even be
>>> familiar with SystemTap. This should help users and admins who wish to
>>> run a set of scripts to solve a specific problem(maybe on a customer's
>>> machine).
>>>
>>> Please comment,
>>>
>>> Regards,
>>> Anithra.
>>>
>>>
>>>
>>>
>>
>


[-- Attachment #2: distribution-fw.tgz --]
[-- Type: application/x-compressed-tar, Size: 7665 bytes --]

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Ahmed Taha]

Hi,

Please help me understand the jump that had just happened. What you  
are saying is that to troubleshoot an abnormality or a kernel hanging  
or a softlock due to a random process or a stateless nfsv4 hanging- 
kernel, I do NOT have to install a release-specific systemtap rpm with  
its debug info kernel, but all what I do now is to generate the  
systemtap rpm customized to a certain config parameters Provided I  
have already initial snapshots of systemtap runs on another machine ?

Thanks, --Ahmed



On Feb 28, 2010, at 3:58, Anithra P Janakiraman <anithra@linux.vnet.ibm.com 
 > wrote:

> Hi,
>
> Please do let me know if there are any concerns/suggestions with the  
> distribution framework.
> We are thinking a man page would not be necessary as usage is  
> trivial. Let me know if it is needed ((README has been included).
>
> Regards,
> Anithra.
>
>
>
>
> On 02/17/2010 11:39 PM, Anithra P Janakiraman wrote:
>>
>> Hi,
>>
>> I've attached a modified framework. The systemtap_pkg_generator now
>> generates a package that includes staprun,stapio, an install script  
>> and
>> the generated kernel module. The files are not packaged into an  
>> rpm, and
>> no installation happens. The resulting 'package' is a self-extracting
>> binary file that extracts all files to the current dir, invokes the
>> install script which in turn invokes staprun from the current dir.  
>> This
>> binary file can only be executed by root users.
>>
>> Please comment.
>>
>> Regards,
>> Anithra.
>>
>>
>> On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
>>> Hi,
>>>
>>> I've attached a modified framework. The names of the template/config
>>> files have been changed(no change in functionality). I've  
>>> attempted to
>>> keeps the names compatible with other scripts in the SystemTap  
>>> package.
>>> We would like to keep the framework in a seperate directory  
>>> (similar to
>>> the initscript dir) for the sake of simplicity. (called
>>> distribution-framework or distribution-fw?)
>>>
>>> The directory would contain the following
>>> template.specfile
>>> template.install
>>> template.binextractor
>>> config.rpmoptions
>>> systemtap-rpm-generator (This is the executable script)
>>> README
>>>
>>>
>>> Let me know if any changes are needed (names of files/directory or
>>> contents). If there is any naming convention that i should adhere to
>>> please let me know. If there are no concerns, or suggestions i would
>>> like to go ahead and commit the framework.
>>>
>>> Thanks.
>>>
>>> Regards,
>>> Anithra
>>>
>>>
>>>
>>> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>>>> Hi all,
>>>>
>>>> We've been looking at a simplified distribution framework for  
>>>> SystemTap
>>>> scripts that :
>>>> 1. packages a set of systemtap scripts and dependent
>>>> tapsets in a form that circumvents , as much as possible the need  
>>>> for
>>>> external dependencies(like kernel-debuginfo)
>>>> 2. installs, runs the script, post processes the output and  
>>>> uninstalls
>>>> out of the box
>>>>
>>>> We feel this would be especially useful in scenarios where support
>>>> admins wish to run a particular set of script(s) on a machine for  
>>>> debug
>>>> or monitoring purposes. I'm attaching an rpm-based framework that  
>>>> does
>>>> the following:
>>>> 1. Compiles a set of scripts into a set of kernel modules
>>>> on another machine of identical architecture that has SystemTap
>>>> installed and running
>>>> 2. Creates an rpm on the fly that would consist of
>>>> i) kernel modules ii) staprun & stapio taken from the systemtap- 
>>>> runtime
>>>> package in the machine that has SystemTap installed, iii) post
>>>> processing script to process the output(optional)
>>>> 3. Bundles the above
>>>> rpm with an install script that has options to i) install the rpm  
>>>> ii)
>>>> run / stop iii) uninstall iv) all of the above.
>>>> 4 . The rpm and the
>>>> install script are packaged into a self-extracting binary that  
>>>> would
>>>> extract itself and execute 1 of the four steps above.
>>>>
>>>> The framework would mainly consist of
>>>> 1. rpm-generating script
>>>> (rpm-generator.sh) that does all of the above
>>>> 2. spec file template that
>>>> will be modified by the rpm-generator on the fly and used to  
>>>> build the
>>>> rpm
>>>> 3. install script template that will be modified by the
>>>> rpm-generator and bundled with the rpm
>>>> 4. Configuration file - that
>>>> would specify the location of the scripts/tapsets
>>>>
>>>> The rpm is packaged as a self extracting binary for ease of use.  
>>>> When
>>>> the binary is executed it extracts the rpm package and based on the
>>>> parameters provided either installs/runs/uninstalls the rpm. Help
>>>> detailing the available options are also provided
>>>>
>>>> USAGE: <package_name> [options] [parameters]
>>>>
>>>> Options:
>>>> * --install -i Install the tapset rpm.
>>>> * --run -r Runs the scripts
>>>> for n minutes where n can be passed as a parameter. o The default  
>>>> value
>>>> is 10 minutes. Post processing is performed after the script  
>>>> completes.
>>>> * --start -s Invokes the script as a background process.
>>>> * --stop -x Stops the script and performs post processing.
>>>> * --uninstall -u Stops the script if running and uninstalls the  
>>>> rpm.
>>>> * --all Installs the rpm, runs the scrip, processes the output and
>>>> uninstalls the rpm.
>>>> * --help Displays this usage text.
>>>>
>>>> Parameters:
>>>>
>>>> * time=[x] x is in minutes. Runs the script for x minutes. valid  
>>>> for
>>>> --run(-r) o --start(-s) or --all(-a) options only
>>>>
>>>>
>>>>
>>>> One disadvantage of the above framework is that the end binary  
>>>> that is
>>>> created is tied to a particular kernel version/architecture.  
>>>> However the
>>>> users now need not install systemtap, debuginfo and need not even  
>>>> be
>>>> familiar with SystemTap. This should help users and admins who  
>>>> wish to
>>>> run a set of scripts to solve a specific problem(maybe on a  
>>>> customer's
>>>> machine).
>>>>
>>>> Please comment,
>>>>
>>>> Regards,
>>>> Anithra.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>
> <distribution-fw.tgz>

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

On 03/01/2010 01:56 AM, Ahmed Taha wrote:
> Hi,
>
> Please help me understand the jump that had just happened. What you are
> saying is that to troubleshoot an abnormality or a kernel hanging or a
> softlock due to a random process or a stateless nfsv4 hanging-kernel, I
> do NOT have to install a release-specific systemtap rpm with its debug
> info kernel, but all what I do now is to generate the systemtap rpm
> customized to a certain config parameters Provided I have already
> initial snapshots of systemtap runs on another machine ?
>

There is no change in the existing SystemTap packaging and/or 
distribution. This is *ONLY* for distribution of specific SystemTap 
scripts.

This will be useful if you have SystemTap installed on another machine 
which has the same configuration(arch,kernel,..) as the machine you wish 
to troubleshoot. You can generate a custom package for the script that 
you wish to run using the stap-buildpkg on the machine that has 
SystemTap installed. The resulting 'custom package' is an executable 
that can be distributed to other machines that have an identical 
configuration(that may or may not have SystemTap installed). When the 
'custom package' is invoked  it would execute a pre-compiled version of 
the script that you had provided.


Regards,
Anithra

>
>
>
> On Feb 28, 2010, at 3:58, Anithra P Janakiraman
> <anithra@linux.vnet.ibm.com> wrote:
>
>> Hi,
>>
>> Please do let me know if there are any concerns/suggestions with the
>> distribution framework.
>> We are thinking a man page would not be necessary as usage is trivial.
>> Let me know if it is needed ((README has been included).
>>
>> Regards,
>> Anithra.
>>
>>
>>
>>
>> On 02/17/2010 11:39 PM, Anithra P Janakiraman wrote:
>>>
>>> Hi,
>>>
>>> I've attached a modified framework. The systemtap_pkg_generator now
>>> generates a package that includes staprun,stapio, an install script and
>>> the generated kernel module. The files are not packaged into an rpm, and
>>> no installation happens. The resulting 'package' is a self-extracting
>>> binary file that extracts all files to the current dir, invokes the
>>> install script which in turn invokes staprun from the current dir. This
>>> binary file can only be executed by root users.
>>>
>>> Please comment.
>>>
>>> Regards,
>>> Anithra.
>>>
>>>
>>> On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
>>>> Hi,
>>>>
>>>> I've attached a modified framework. The names of the template/config
>>>> files have been changed(no change in functionality). I've attempted to
>>>> keeps the names compatible with other scripts in the SystemTap package.
>>>> We would like to keep the framework in a seperate directory (similar to
>>>> the initscript dir) for the sake of simplicity. (called
>>>> distribution-framework or distribution-fw?)
>>>>
>>>> The directory would contain the following
>>>> template.specfile
>>>> template.install
>>>> template.binextractor
>>>> config.rpmoptions
>>>> systemtap-rpm-generator (This is the executable script)
>>>> README
>>>>
>>>>
>>>> Let me know if any changes are needed (names of files/directory or
>>>> contents). If there is any naming convention that i should adhere to
>>>> please let me know. If there are no concerns, or suggestions i would
>>>> like to go ahead and commit the framework.
>>>>
>>>> Thanks.
>>>>
>>>> Regards,
>>>> Anithra
>>>>
>>>>
>>>>
>>>> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>>>>> Hi all,
>>>>>
>>>>> We've been looking at a simplified distribution framework for
>>>>> SystemTap
>>>>> scripts that :
>>>>> 1. packages a set of systemtap scripts and dependent
>>>>> tapsets in a form that circumvents , as much as possible the need for
>>>>> external dependencies(like kernel-debuginfo)
>>>>> 2. installs, runs the script, post processes the output and uninstalls
>>>>> out of the box
>>>>>
>>>>> We feel this would be especially useful in scenarios where support
>>>>> admins wish to run a particular set of script(s) on a machine for
>>>>> debug
>>>>> or monitoring purposes. I'm attaching an rpm-based framework that does
>>>>> the following:
>>>>> 1. Compiles a set of scripts into a set of kernel modules
>>>>> on another machine of identical architecture that has SystemTap
>>>>> installed and running
>>>>> 2. Creates an rpm on the fly that would consist of
>>>>> i) kernel modules ii) staprun & stapio taken from the
>>>>> systemtap-runtime
>>>>> package in the machine that has SystemTap installed, iii) post
>>>>> processing script to process the output(optional)
>>>>> 3. Bundles the above
>>>>> rpm with an install script that has options to i) install the rpm ii)
>>>>> run / stop iii) uninstall iv) all of the above.
>>>>> 4 . The rpm and the
>>>>> install script are packaged into a self-extracting binary that would
>>>>> extract itself and execute 1 of the four steps above.
>>>>>
>>>>> The framework would mainly consist of
>>>>> 1. rpm-generating script
>>>>> (rpm-generator.sh) that does all of the above
>>>>> 2. spec file template that
>>>>> will be modified by the rpm-generator on the fly and used to build the
>>>>> rpm
>>>>> 3. install script template that will be modified by the
>>>>> rpm-generator and bundled with the rpm
>>>>> 4. Configuration file - that
>>>>> would specify the location of the scripts/tapsets
>>>>>
>>>>> The rpm is packaged as a self extracting binary for ease of use. When
>>>>> the binary is executed it extracts the rpm package and based on the
>>>>> parameters provided either installs/runs/uninstalls the rpm. Help
>>>>> detailing the available options are also provided
>>>>>
>>>>> USAGE: <package_name> [options] [parameters]
>>>>>
>>>>> Options:
>>>>> * --install -i Install the tapset rpm.
>>>>> * --run -r Runs the scripts
>>>>> for n minutes where n can be passed as a parameter. o The default
>>>>> value
>>>>> is 10 minutes. Post processing is performed after the script
>>>>> completes.
>>>>> * --start -s Invokes the script as a background process.
>>>>> * --stop -x Stops the script and performs post processing.
>>>>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>>>>> * --all Installs the rpm, runs the scrip, processes the output and
>>>>> uninstalls the rpm.
>>>>> * --help Displays this usage text.
>>>>>
>>>>> Parameters:
>>>>>
>>>>> * time=[x] x is in minutes. Runs the script for x minutes. valid for
>>>>> --run(-r) o --start(-s) or --all(-a) options only
>>>>>
>>>>>
>>>>>
>>>>> One disadvantage of the above framework is that the end binary that is
>>>>> created is tied to a particular kernel version/architecture.
>>>>> However the
>>>>> users now need not install systemtap, debuginfo and need not even be
>>>>> familiar with SystemTap. This should help users and admins who wish to
>>>>> run a set of scripts to solve a specific problem(maybe on a customer's
>>>>> machine).
>>>>>
>>>>> Please comment,
>>>>>
>>>>> Regards,
>>>>> Anithra.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>> <distribution-fw.tgz>

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 4859 bytes --]

Hi,

I've attached a modified framework that contains the following changes 
(as per comments received on IRC):
1. Config options need not be specified in a separate config file. All 
options are passed through the command line.
2. The stap-buildpkg script does not parse the kernel release/additional 
tapset directories options. Instead all 'additional' options are 
directly passed to the stap command.
3. The package now contains only 1 template file and README apart from 
the stap-buildpkg script itself.

The framework was tested on RHEL5.4(x86,x86_64,ppc64), 
SLES10(ppc64),F12(x86).

Please comment.

Regards,
Anithra.


On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
> Hi,
>
> I've attached a modified framework. The names of the template/config
> files have been changed(no change in functionality). I've attempted to
> keeps the names compatible with other scripts in the SystemTap package.
> We would like to keep the framework in a seperate directory (similar to
> the initscript dir) for the sake of simplicity. (called
> distribution-framework or distribution-fw?)
>
> The directory would contain the following
> template.specfile
> template.install
> template.binextractor
> config.rpmoptions
> systemtap-rpm-generator (This is the executable script)
> README
>
>
> Let me know if any changes are needed (names of files/directory or
> contents). If there is any naming convention that i should adhere to
> please let me know. If there are no concerns, or suggestions i would
> like to go ahead and commit the framework.
>
> Thanks.
>
> Regards,
> Anithra
>
>
>
> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>> Hi all,
>>
>> We've been looking at a simplified distribution framework for SystemTap
>> scripts that :
>> 1. packages a set of systemtap scripts and dependent
>> tapsets in a form that circumvents , as much as possible the need for
>> external dependencies(like kernel-debuginfo)
>> 2. installs, runs the script, post processes the output and uninstalls
>> out of the box
>>
>> We feel this would be especially useful in scenarios where support
>> admins wish to run a particular set of script(s) on a machine for debug
>> or monitoring purposes. I'm attaching an rpm-based framework that does
>> the following:
>> 1. Compiles a set of scripts into a set of kernel modules
>> on another machine of identical architecture that has SystemTap
>> installed and running
>> 2. Creates an rpm on the fly that would consist of
>> i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
>> package in the machine that has SystemTap installed, iii) post
>> processing script to process the output(optional)
>> 3. Bundles the above
>> rpm with an install script that has options to i) install the rpm ii)
>> run / stop iii) uninstall iv) all of the above.
>> 4 . The rpm and the
>> install script are packaged into a self-extracting binary that would
>> extract itself and execute 1 of the four steps above.
>>
>> The framework would mainly consist of
>> 1. rpm-generating script
>> (rpm-generator.sh) that does all of the above
>> 2. spec file template that
>> will be modified by the rpm-generator on the fly and used to build the
>> rpm
>> 3. install script template that will be modified by the
>> rpm-generator and bundled with the rpm
>> 4. Configuration file - that
>> would specify the location of the scripts/tapsets
>>
>> The rpm is packaged as a self extracting binary for ease of use. When
>> the binary is executed it extracts the rpm package and based on the
>> parameters provided either installs/runs/uninstalls the rpm. Help
>> detailing the available options are also provided
>>
>> USAGE: <package_name> [options] [parameters]
>>
>> Options:
>> * --install -i Install the tapset rpm.
>> * --run -r Runs the scripts
>> for n minutes where n can be passed as a parameter. o The default value
>> is 10 minutes. Post processing is performed after the script completes.
>> * --start -s Invokes the script as a background process.
>> * --stop -x Stops the script and performs post processing.
>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>> * --all Installs the rpm, runs the scrip, processes the output and
>> uninstalls the rpm.
>> * --help Displays this usage text.
>>
>> Parameters:
>>
>> * time=[x] x is in minutes. Runs the script for x minutes. valid for
>> --run(-r) o --start(-s) or --all(-a) options only
>>
>>
>>
>> One disadvantage of the above framework is that the end binary that is
>> created is tied to a particular kernel version/architecture. However the
>> users now need not install systemtap, debuginfo and need not even be
>> familiar with SystemTap. This should help users and admins who wish to
>> run a set of scripts to solve a specific problem(maybe on a customer's
>> machine).
>>
>> Please comment,
>>
>> Regards,
>> Anithra.
>>
>>
>>
>>
>


[-- Attachment #2: framework.tgz --]
[-- Type: application/x-compressed-tar, Size: 3643 bytes --]

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 5292 bytes --]

Hi ,

I'm attaching the framework as a patch as per Frank's suggestion in the 
call. Hope this makes it easier to review. If there are no 
comments/objections  i would like to commit this early next week.

Thanks,
Regards,
Anithra



On 03/23/2010 02:17 AM, Anithra P Janakiraman wrote:
> Hi,
>
> I've attached a modified framework that contains the following changes
> (as per comments received on IRC):
> 1. Config options need not be specified in a separate config file. All
> options are passed through the command line.
> 2. The stap-buildpkg script does not parse the kernel release/additional
> tapset directories options. Instead all 'additional' options are
> directly passed to the stap command.
> 3. The package now contains only 1 template file and README apart from
> the stap-buildpkg script itself.
>
> The framework was tested on RHEL5.4(x86,x86_64,ppc64),
> SLES10(ppc64),F12(x86).
>
> Please comment.
>
> Regards,
> Anithra.
>
>
> On 02/02/2010 10:12 PM, Anithra P Janakiraman wrote:
>> Hi,
>>
>> I've attached a modified framework. The names of the template/config
>> files have been changed(no change in functionality). I've attempted to
>> keeps the names compatible with other scripts in the SystemTap package.
>> We would like to keep the framework in a seperate directory (similar to
>> the initscript dir) for the sake of simplicity. (called
>> distribution-framework or distribution-fw?)
>>
>> The directory would contain the following
>> template.specfile
>> template.install
>> template.binextractor
>> config.rpmoptions
>> systemtap-rpm-generator (This is the executable script)
>> README
>>
>>
>> Let me know if any changes are needed (names of files/directory or
>> contents). If there is any naming convention that i should adhere to
>> please let me know. If there are no concerns, or suggestions i would
>> like to go ahead and commit the framework.
>>
>> Thanks.
>>
>> Regards,
>> Anithra
>>
>>
>>
>> On 12/09/2009 11:11 PM, Anithra P Janakiraman wrote:
>>> Hi all,
>>>
>>> We've been looking at a simplified distribution framework for SystemTap
>>> scripts that :
>>> 1. packages a set of systemtap scripts and dependent
>>> tapsets in a form that circumvents , as much as possible the need for
>>> external dependencies(like kernel-debuginfo)
>>> 2. installs, runs the script, post processes the output and uninstalls
>>> out of the box
>>>
>>> We feel this would be especially useful in scenarios where support
>>> admins wish to run a particular set of script(s) on a machine for debug
>>> or monitoring purposes. I'm attaching an rpm-based framework that does
>>> the following:
>>> 1. Compiles a set of scripts into a set of kernel modules
>>> on another machine of identical architecture that has SystemTap
>>> installed and running
>>> 2. Creates an rpm on the fly that would consist of
>>> i) kernel modules ii) staprun & stapio taken from the systemtap-runtime
>>> package in the machine that has SystemTap installed, iii) post
>>> processing script to process the output(optional)
>>> 3. Bundles the above
>>> rpm with an install script that has options to i) install the rpm ii)
>>> run / stop iii) uninstall iv) all of the above.
>>> 4 . The rpm and the
>>> install script are packaged into a self-extracting binary that would
>>> extract itself and execute 1 of the four steps above.
>>>
>>> The framework would mainly consist of
>>> 1. rpm-generating script
>>> (rpm-generator.sh) that does all of the above
>>> 2. spec file template that
>>> will be modified by the rpm-generator on the fly and used to build the
>>> rpm
>>> 3. install script template that will be modified by the
>>> rpm-generator and bundled with the rpm
>>> 4. Configuration file - that
>>> would specify the location of the scripts/tapsets
>>>
>>> The rpm is packaged as a self extracting binary for ease of use. When
>>> the binary is executed it extracts the rpm package and based on the
>>> parameters provided either installs/runs/uninstalls the rpm. Help
>>> detailing the available options are also provided
>>>
>>> USAGE: <package_name> [options] [parameters]
>>>
>>> Options:
>>> * --install -i Install the tapset rpm.
>>> * --run -r Runs the scripts
>>> for n minutes where n can be passed as a parameter. o The default value
>>> is 10 minutes. Post processing is performed after the script completes.
>>> * --start -s Invokes the script as a background process.
>>> * --stop -x Stops the script and performs post processing.
>>> * --uninstall -u Stops the script if running and uninstalls the rpm.
>>> * --all Installs the rpm, runs the scrip, processes the output and
>>> uninstalls the rpm.
>>> * --help Displays this usage text.
>>>
>>> Parameters:
>>>
>>> * time=[x] x is in minutes. Runs the script for x minutes. valid for
>>> --run(-r) o --start(-s) or --all(-a) options only
>>>
>>>
>>>
>>> One disadvantage of the above framework is that the end binary that is
>>> created is tied to a particular kernel version/architecture. However the
>>> users now need not install systemtap, debuginfo and need not even be
>>> familiar with SystemTap. This should help users and admins who wish to
>>> run a set of scripts to solve a specific problem(maybe on a customer's
>>> machine).
>>>
>>> Please comment,
>>>
>>> Regards,
>>> Anithra.
>>>
>>>
>>>
>>>
>>
>


[-- Attachment #2: distributionframework.patch --]
[-- Type: text/plain, Size: 12043 bytes --]

diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
new file mode 100644
index 0000000..d87a3de
--- /dev/null
+++ b/scripts/build_pkg/README.stap-buildpkg
@@ -0,0 +1,45 @@
+systemtap distribution framework
+
+This is free software.
+See the COPYING file for redistribution/modification terms.
+
+Introduction:
+
+The SystemTap distribution framework aims to:
+
+1. packages a set of systemtap scripts in a format that circumvents as much 
+as possible, external dependencies like kernel-debuginfo.  
+2. generate a package that will self-extract,  run the script, and post 
+process the output with minimal intervention from  users.
+
+Usage:  
+
+stap-buildpkg scriptfile [options] [stap-options]
+Options:
+       -n pkg_name   		: specify a package name
+       -v pkg_version		: specify a package version
+       -c pkg_config  		: provide config parameters through a file
+       -p post_process script   : post processing script
+       -h help_config		: help text for config parameters
+       All additonal options will be passed to the stap command
+
+pkg_name and pkg_version will be used to set the name and version of the resulting executable. An optional config file that contains script specific config options and corresponding help entries can also be specified using the -c and -h options respectively. A post processing script can be provided using the -p option.The resulting package is a self-extracting binary. When invoked it would untar itself, execute the script(by invoking staprun) and post process.
+The usage for the the resulting package:
+
+<package_name> [options] [parameters]
+
+Options:
+   * --run -r   Runs the scripts
+                for n minutes where n can be passed as a parameter. The default
+                value is 10 minutes. Post processing is performed after the 
+                script completes.
+   * --start -s Invokes the script as a background process.
+   * --stop -x  Stops the script and performs post processing.
+   * --all      Same as --run
+   * --help     Displays this usage text.
+
+Parameters:
+
+   * time=[x]   x is in minutes. Runs the script for x minutes. valid for
+                --run(-r) o --start(-s) or --all(-a) options only
+
diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
new file mode 100755
index 0000000..dd04d47
--- /dev/null
+++ b/scripts/build_pkg/stap-buildpkg
@@ -0,0 +1,200 @@
+##!/bin/bash
+# 
+#  Script to auto-generate rpms from stap scripts
+#
+
+
+#Checking if script is run by root
+if [ $EUID -ne 0 ]
+then
+	echo "ERROR :You need to be root to run this !"
+	exit
+fi
+
+prog=stap-buildpkg
+
+echo_usage () {
+  echo $"Usage: $prog scriptfile [options] [stap-options]"
+  echo $"Options:"
+  echo $"       -n pkg_name   		: specify a package name"
+  echo $"       -v pkg_version		: specify a package version"
+  echo $"       -c pkg_config  		: provide config parameters through a file"
+  echo $"       -p post_process script   : post processing script"
+  echo $"       -h help_config		: help text for config parameters"
+  echo $"       All additonal options will be passed to the stap command"
+}
+
+parse_args() {
+while [ -n "$1" ]; do
+     case "$1" in
+        -n)                    PKG_NAME="$2"
+				shift 2
+				echo $PKG_NAME
+                                ;;
+        -v)			PKG_VERSION=$2
+                                shift 2
+				echo $PKG_VERSION
+                                ;;
+        -c) 		        PKG_INPUT_CONFIG=$2
+				echo $PKG_INPUT_CONFIG
+				shift 2
+                                ;;
+        -p)			PKG_POST_SCRIPT=$2
+				echo $PKG_POST_SCRIPT
+				shift 2
+                                ;;
+        -H) 		        PKG_HELP_CONFIG=$2
+				echo $PKG_HELP_CONFIG
+                                shift 2
+                                ;;
+        *)                     STAP_OPTIONS=$@ 
+				echo $STAP_OPTIONS
+                                shift $#
+esac
+done
+}
+
+validate_args() {
+ if test -z $PKG_NAME; then
+   PKG_NAME=`basename $script .stp`
+ fi
+ if test -z $PKG_VERSION; then
+   PKG_VERSION=1.0
+ fi
+ if test -z $PKG_RELEASE; then
+   PKG_RELEASE="0"
+ fi
+ PKG_ARCH=`uname -i`
+}
+
+module_gen() {
+stap -p4 -k -m $PKG_NAME.ko $script $STAP_OPTIONS
+if [ -f $PKG_NAME.ko ]
+then
+        echo " Module $PKG_NAME.ko generated successfully ";
+else
+        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
+        rm -rf $TEMPDIR
+        exit;
+fi
+}
+
+create_wrappers() {
+echo "Creating RPM wrapper from template..";
+echo "s/TEMPLATE_PKG_NAME/$PKG_NAME/g" > $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_VERSION/$PKG_VERSION/g" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_RELEASE/$PKG_RELEASE/" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_ARCH/$PKG_ARCH/g" >> $TEMPDIR/sd-installer-patterns
+
+sed -f $TEMPDIR/sd-installer-patterns template.buildpkg > $TEMPDIR/wrapper
+csplit -s $TEMPDIR/wrapper '/extractor template/'
+mv xx00 ${PKG_NAME}_install
+mv xx01 ${PKG_NAME}_binextractor
+
+chmod a+x ${PKG_NAME}_install
+rm $TEMPDIR/sd-installer-patterns
+rm $TEMPDIR/wrapper
+}
+
+copy_files_to_temp() {
+
+runtime=systemtap-runtime
+if test -z $(rpm -qa $runtime); then
+ runtime=systemtap 
+fi
+
+echo "using systemtap runtime executables from $runtime rpm"
+
+cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
+cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.
+
+echo "Copying Systemtap module.."
+mv $PKG_NAME.ko $TEMPDIR/ 
+echo "Copying config file.."
+echo "time=2m" > $TEMPDIR/$PKG_NAME.config
+if [ -f $PKG_INPUT_CONFIG ] && [ "$PKG_INPUT_CONFIG" != "" ]
+then 
+   cat $PKG_INPUT_CONFIG >> $TEMPDIR/$PKG_NAME.config
+fi
+# Copying help file, or adding an empty file if none is provided.
+if [ -f $PKG_HELP_CONFIG ] && [ "$PKG_HELP_CONFIG" != "" ]
+then
+        cp $PKG_HELP_CONFIG $TEMPDIR/$PKG_NAME.help
+else
+        echo " " > $TEMPDIR/$PKG_NAME.help
+fi
+echo "Copying the post processing script"
+# copying the post processing script, or adding an empty file if none is provided.
+if [ -f $PKG_POST_SCRIPT ] && [ "$PKG_POST_SCRIPT" != "" ]
+then
+    cp $PKG_POST_SCRIPT  $TEMPDIR/$PKG_NAME.post
+else
+   echo " " > $TEMPDIR/$PKG_NAME.post
+fi
+chmod +x $TEMPDIR/$PKG_NAME.post
+mv ${PKG_NAME}_install $TEMPDIR/
+}
+
+#------------------------------------------------------------------
+# Mainline script
+#------------------------------------------------------------------
+script=$1
+echo $script
+shift 1
+
+if test -z $script; then
+ echo "Please provide a script name"
+ echo_usage
+ exit
+fi
+
+if [ ! -f "$script" ]; then
+ echo "Script file does not exist"
+ echo "Please provide a valid script name"
+ echo_usage
+ exit
+fi
+
+ 
+#OPTS=`getopt -s bash -u -o 'n:v:R:c:p:h:' -- $@`
+
+#parse arguments
+parse_args $@
+validate_args
+
+# generate a tmp directory to store all components:remember to clean it up !
+TEMPDIR=/tmp/$PKG_NAME-$PKG_VERSION;
+mkdir  $TEMPDIR
+
+# Generate Stap module
+echo "Generating Stap module..";
+module_gen
+
+#generate installer(wrapper) scripts
+create_wrappers
+
+#copy systemtap-runtime files
+copy_files_to_temp
+
+MY_PWD=`pwd`
+cd $TEMPDIR ; tar -cjf $PKG_NAME-$PKG_VERSION.tar.bz2 *
+cd $MY_PWD;
+
+echo "Building the self-extracting binary.."
+mkdir $TEMPDIR/$PKG_NAME
+mv $TEMPDIR/${PKG_NAME}_install $TEMPDIR/$PKG_NAME/
+tar -xjf $TEMPDIR/$PKG_NAME-$PKG_VERSION.tar.bz2 -C $TEMPDIR/$PKG_NAME/
+mv  ${PKG_NAME}_binextractor $TEMPDIR/${PKG_NAME}_binextractor 2>/dev/null
+#mv /tmp/$PKG_NAME-$PKG_VERSION.tar.bz2 $TEMPDIR/
+
+MY_PWD=`pwd`
+cd $TEMPDIR; 
+tar -czf - $PKG_NAME/ >> ${PKG_NAME}_binextractor
+cd $MY_PWD;
+mv $TEMPDIR/${PKG_NAME}_binextractor ./$PKG_NAME-$PKG_VERSION
+echo "The installer is at `pwd`/$PKG_NAME-$PKG_VERSION"
+chmod +x $PKG_NAME-$PKG_VERSION
+
+#Cleanup the mess in temp dir
+#rm -rf $TEMPDIR
+exit
diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
new file mode 100644
index 0000000..8dadb5f
--- /dev/null
+++ b/scripts/build_pkg/template.buildpkg
@@ -0,0 +1,153 @@
+#!/bin/bash
+SCRIPT_LOC=/usr/tapsetrpms/TEMPLATE_PKG_NAME-TEMPLATE_PKG_VERSION
+RUN_DIR=`pwd`
+
+function run()
+{
+echo "Gathering data... Please wait"
+# Log file appears as systemtap_tcpipstat.out
+TS=`date "+%Y%m%d-%H:%M:%S"`
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 &
+sleep $time
+echo "Completed"
+mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function start()
+{
+ # check to see if tapset is installed
+echo "Gathering data..."
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out 2>&1 &
+echo "Run TEMPLATE_PKG_NAME --stop to stop the script and process output"
+}
+
+function stop()
+{
+TS=`date "+%Y%m%d-%H:%M:%S"`
+echo "Stopping systemtap script"
+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
+echo "Done"
+mv $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function helptext()
+{
+ echo " "
+ echo "USAGE: TEMPLATE_PKG_NAME [options] [parameters]"
+ echo " "
+ echo "Options: "
+ echo " --run -r	Runs the scripts for n minutes where n can be passed as a parameter. "
+ echo "		The default value is 10 minutes. Post processing is performed after the script completes."
+ echo " --start -s	Runs the script as a background process"
+ echo " --stop -x	Stops the script and performs post processing"
+ echo " --all		Runs the script and processes the output"
+ echo " --help		Displays this usage text"
+ echo " "
+ echo "Parameters: "
+ echo " "
+ echo " time=[x]	x is in minutes. Runs the script for x minutes. valid for --run(-r)"
+ echo "		--start(-s) or --all(-a) options only"
+# echo " ipaddress=[ip]	ip is a standard IPv4 address. The output will be filtered for this address"
+ if [ $? -eq 0 ]
+ then
+ 	cat $RUN_DIR/TEMPLATE_PKG_NAME.help
+ fi
+ echo " "
+}
+
+function parseparameters()
+{
+for i in $@
+do
+        tmpa=${1%%=*}
+        tmpb=${1#$tmpa=}
+        export $tmpa=$tmpb
+        shift
+done
+}
+
+
+#checking if its run by root
+if [ $EUID -ne 0 ]
+then
+        echo "You need to be root to run this !"
+        exit;
+fi
+option=$1
+shift
+
+
+
+  FILE=$RUN_DIR/TEMPLATE_PKG_NAME.config
+   # make sure file exist and readable
+   if [ ! -f $FILE ]; then
+  	echo "$FILE : does not exist"
+   elif [ ! -r $FILE ]; then
+  	echo "$FILE: cannot read"
+   fi
+
+
+
+   exec 3<&0
+   exec 0<$FILE
+   while read line
+	do
+        tmpa=${line%%=*}
+        tmpb=${line#$tmpa=}
+        declare $tmpa=$tmpb
+	done
+   exec 0<&3
+
+   for i in $@
+	do
+	        tmpa=${1%%=*}
+        	tmpb=${1#$tmpa=}
+	        export $tmpa=$tmpb
+        	shift
+	done
+
+
+case $option in
+	-r | --run ) 	 			
+				run
+				exit
+				;;
+        -s | --start )         
+				start
+				exit
+                                ;;
+        -h | --help )           helptext
+                                exit
+                                ;;
+        -x | --stop )		stop
+				exit
+				;;
+	-a | --all )		run
+				exit
+				;;
+        * )                     helptext
+                                exit 1
+esac
+
+## extractor template
+#!/bin/sh -e
+sed -e '1,/^exit$/d' "$0" | tar -xzf - 
+cd TEMPLATE_PKG_NAME 
+ ./TEMPLATE_PKG_NAME_install $@
+cd ..
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME.*
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME_install
+exit

Re: [RFC] Framework for easy distribution of SystemTap scripts.

[Frank Ch. Eigler]

anithra wrote:

> diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
> new file mode 100644
> index 0000000..d87a3de
> --- /dev/null
> +++ b/scripts/build_pkg/README.stap-buildpkg

Could this be a normal man page (stap-buildpkg.1) ?

> +stap-buildpkg scriptfile [options] [stap-options]
> +Options:
> +       -n pkg_name   		: specify a package name
> +       -v pkg_version		: specify a package version
> +       -c pkg_config  		: provide config parameters through a file
> +       -p post_process script   : post processing script
> +       -h help_config		: help text for config parameters
> +       All additonal options will be passed to the stap command

> +pkg_name and pkg_version will be used to set the name and version
> +of the resulting executable.

Why does there need to be a separate 'version'?

> +An optional config file that contains script specific config
> +options and corresponding help entries can also be specified using
> +the -c and -h options respectively.

You need to elaborate what script-specific config options can be, what
they do, how they work.

> +A post processing script can be provided using the -p option.

You should specify what exactly this means.  A '/bin/sh -c' command
line to pipe stdout through?

> [...]
> +<package_name> [options] [parameters]
> +
> +Options:
> +   * --run -r   Runs the scripts
> +                for n minutes where n can be passed as a parameter. The default
> +                value is 10 minutes. Post processing is performed after the 
> +                script completes.
> +   * --start -s Invokes the script as a background process.
> +   * --stop -x  Stops the script and performs post processing.
> +   * --all      Same as --run
> +   * --help     Displays this usage text.
> +
> +Parameters:
> +
> +   * time=[x]   x is in minutes. Runs the script for x minutes. valid for
> +                --run(-r) o --start(-s) or --all(-a) options only

How does time=NNN relate to -r 'n'?

(We could add a stap and/or staprun option for a simple time-based
exit().)


> diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
> [...]
> +#Checking if script is run by root
> +if [ $EUID -ne 0 ]
> +then
> +	echo "ERROR :You need to be root to run this !"
> +	exit
> +fi

Why?


> +prog=stap-buildpkg
> +
> +echo_usage () {
> +  echo $"Usage: $prog scriptfile [options] [stap-options]"
> +  echo $"Options:"
> +  echo $"       -n pkg_name   		: specify a package name"
> +  echo $"       -v pkg_version		: specify a package version"
> +  echo $"       -c pkg_config  		: provide config parameters through a file"
> +  echo $"       -p post_process script   : post processing script"
> +  echo $"       -h help_config		: help text for config parameters"
> +  echo $"       All additonal options will be passed to the stap command"
> +}
> +
> +parse_args() {
> +while [ -n "$1" ]; do
> +     case "$1" in
> + [...]
> +        *)                     STAP_OPTIONS=$@ 
> +				echo $STAP_OPTIONS

You'll need to watch the quoting in these $foo variables.
They could contain spaces etc.


> +module_gen() {
> +stap -p4 -k -m $PKG_NAME.ko $script $STAP_OPTIONS
> +if [ -f $PKG_NAME.ko ]
> +then
> +        echo " Module $PKG_NAME.ko generated successfully ";
> +else
> +        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
> +        rm -rf $TEMPDIR
> +        exit;
> +fi
> +}

Why are you running stap with '-k'?  If you're only saving the .ko result,
then let stap give it to you in the present working directory.


> [...]
> +echo "using systemtap runtime executables from $runtime rpm"
> +cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
> +cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.

Have you observed systemtap being installed as a relocatable rpm somewhere
other than /usr/* ?  If not, you could just copy /usr/bin/staprun etc. by
normal path name.


> [...]
> +if [ ! -f "$script" ]; then
> + echo "Script file does not exist"
> + echo "Please provide a valid script name"
> + echo_usage
> + exit
> +fi

(Could a user not specify the script contents with   stap-foo  -e 'SCRIPT' ?)

> [...]
> +#Cleanup the mess in temp dir
> +#rm -rf $TEMPDIR

I assume not actually cleaning up the temporary directory is temporary.


> diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
> [...]

FWIW, you could include this template within the main buildpkg script, as in
 
echo > $PKG << 'END'
function run() 
....
END



> @@ -0,0 +1,153 @@
> +#!/bin/bash
> +SCRIPT_LOC=/usr/tapsetrpms/TEMPLATE_PKG_NAME-TEMPLATE_PKG_VERSION

What is SCRIPT_LOC for?

> [...]
> +$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 &
> +sleep $time
> +echo "Completed"
> +mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
> +echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"

(Is all this echoing necessary verbiage?)

> +ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT

You wouldn't need this is you stored $! after the "staprun ... &" a few lines ago.


> +function helptext()
> [...]
> +# echo " ipaddress=[ip]	ip is a standard IPv4 address. The output will be filtered for this address"

Really?  Perhaps the framework-maker should have a filter-command
option such as -P 'grep 127.0.0.1' that it could pass through.


> +  FILE=$RUN_DIR/TEMPLATE_PKG_NAME.config
> +   # make sure file exist and readable
> +   if [ ! -f $FILE ]; then
> +  	echo "$FILE : does not exist"
> +   elif [ ! -r $FILE ]; then
> +  	echo "$FILE: cannot read"
> +   fi

Why does this template need to exist?  The framework maker script
could transcribe all of the options right into the output script.


- FChE

Re: [RFC] Framework for easy distribution of SystemTap scripts (V2)

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 8350 bytes --]

On 04/08/2010 09:17 PM, Frank Ch. Eigler wrote:

Hi Frank,

Thanks for the review! I've attached a V2 of the patch.

>
> anithra wrote:
>
>> diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
>> new file mode 100644
>> index 0000000..d87a3de
>> --- /dev/null
>> +++ b/scripts/build_pkg/README.stap-buildpkg
>
> Could this be a normal man page (stap-buildpkg.1) ?

We have placed the script in the scripts directory, We dont want to 
install the script, so didnt see the use for a man page.

>
>> +stap-buildpkg scriptfile [options] [stap-options]
>> +Options:
>> +       -n pkg_name   		: specify a package name
>> +       -v pkg_version		: specify a package version
>> +       -c pkg_config  		: provide config parameters through a file
>> +       -p post_process script   : post processing script
>> +       -h help_config		: help text for config parameters
>> +       All additonal options will be passed to the stap command
>
>> +pkg_name and pkg_version will be used to set the name and version
>> +of the resulting executable.
>
> Why does there need to be a separate 'version'?
>

To be able to distinguish between packages built for different versions 
of the same script. We initially had a 'release' too :)

>> +An optional config file that contains script specific config
>> +options and corresponding help entries can also be specified using
>> +the -c and -h options respectively.
>
> You need to elaborate what script-specific config options can be, what
> they do, how they work.
>

Will do.  The optional config could be pid, ip in case of networking 
scripts... Have added an example. The parameters will be passed to the 
post-processing script to filter the output.

>> +A post processing script can be provided using the -p option.
>
> You should specify what exactly this means.  A '/bin/sh -c' command
> line to pipe stdout through?

The post processing script can do anything the administrator creating 
the package wishes to do, maybe even fwd the output to a support team, 
or provide suggestions based on the output. Have added an explanation.

>
>> [...]
>> +<package_name>  [options] [parameters]
>> +
>> +Options:
>> +   * --run -r   Runs the scripts
>> +                for n minutes where n can be passed as a parameter. The default
>> +                value is 10 minutes. Post processing is performed after the
>> +                script completes.
>> +   * --start -s Invokes the script as a background process.
>> +   * --stop -x  Stops the script and performs post processing.
>> +   * --all      Same as --run
>> +   * --help     Displays this usage text.
>> +
>> +Parameters:
>> +
>> +   * time=[x]   x is in minutes. Runs the script for x minutes. valid for
>> +                --run(-r) o --start(-s) or --all(-a) options only
>
> How does time=NNN relate to -r 'n'?

Text changed

>
> (We could add a stap and/or staprun option for a simple time-based
> exit().)
>
>
>> diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
>> [...]
>> +#Checking if script is run by root
>> +if [ $EUID -ne 0 ]
>> +then
>> +	echo "ERROR :You need to be root to run this !"
>> +	exit
>> +fi
>
> Why?

hmmm. this was there for the initial rpm version, we had to check for 
root because rpmbuild was being used in the script. Have removed it now.
>
>
>> +prog=stap-buildpkg
>> +
>> +echo_usage () {
>> +  echo $"Usage: $prog scriptfile [options] [stap-options]"
>> +  echo $"Options:"
>> +  echo $"       -n pkg_name   		: specify a package name"
>> +  echo $"       -v pkg_version		: specify a package version"
>> +  echo $"       -c pkg_config  		: provide config parameters through a file"
>> +  echo $"       -p post_process script   : post processing script"
>> +  echo $"       -h help_config		: help text for config parameters"
>> +  echo $"       All additonal options will be passed to the stap command"
>> +}
>> +
>> +parse_args() {
>> +while [ -n "$1" ]; do
>> +     case "$1" in
>> + [...]
>> +        *)                     STAP_OPTIONS=$@
>> +				echo $STAP_OPTIONS
>
> You'll need to watch the quoting in these $foo variables.
> They could contain spaces etc.

Didn't understand the comment. stap-buildpkg script.stp -n "package 
name" -v 1  would work.

>
>
>> +module_gen() {
>> +stap -p4 -k -m $PKG_NAME.ko $script $STAP_OPTIONS
>> +if [ -f $PKG_NAME.ko ]
>> +then
>> +        echo " Module $PKG_NAME.ko generated successfully ";
>> +else
>> +        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
>> +        rm -rf $TEMPDIR
>> +        exit;
>> +fi
>> +}
>
> Why are you running stap with '-k'?  If you're only saving the .ko result,
> then let stap give it to you in the present working directory.
>

changed

>
>> [...]
>> +echo "using systemtap runtime executables from $runtime rpm"
>> +cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
>> +cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.
>
> Have you observed systemtap being installed as a relocatable rpm somewhere
> other than /usr/* ?  If not, you could just copy /usr/bin/staprun etc. by
> normal path name.
>
>
stapio can be installed in different locations, which is the reason for 
the above check. We do the same for staprun for the same of uniformity

>> [...]
>> +if [ ! -f "$script" ]; then
>> + echo "Script file does not exist"
>> + echo "Please provide a valid script name"
>> + echo_usage
>> + exit
>> +fi
>
> (Could a user not specify the script contents with   stap-foo  -e 'SCRIPT' ?)
>

We are currently not providing this option.  Didnt think abt it 
actually. I'm assuming that if a package needs to be built for a script 
it would most likely not be a one-liner.

>> [...]
>> +#Cleanup the mess in temp dir
>> +#rm -rf $TEMPDIR
>
> I assume not actually cleaning up the temporary directory is temporary.

yes, uncommented.

>
>
>> diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
>> [...]
>
> FWIW, you could include this template within the main buildpkg script, as in
>
> echo>  $PKG<<  'END'
> function run()
> ....
> END
>

True. This is for simplicity of design. It is intuitively a template and 
a separate file, so we decided to keep it like that esp since we had no 
intentions of installing the build-pkg script.

>
>> @@ -0,0 +1,153 @@
>> +#!/bin/bash
>> +SCRIPT_LOC=/usr/tapsetrpms/TEMPLATE_PKG_NAME-TEMPLATE_PKG_VERSION
>
> What is SCRIPT_LOC for?

Redundant, this was another carry over from the prev version of the 
script,  removed.

>
>> [...]
>> +$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko>  $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1&
>> +sleep $time
>> +echo "Completed"
>> +mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
>> +echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
>
> (Is all this echoing necessary verbiage?)

I think so :). When we demonstrated this we were actually asked for more 
verbosity

>
>> +ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
>
> You wouldn't need this is you stored $! after the "staprun ...&" a few lines ago.

Yes. We had done it this way keeping the older versions of SystemTap in 
mind, the $! returns pid of 'staprun' in older versions. I've changed 
the template

>
>
>> +function helptext()
>> [...]
>> +# echo " ipaddress=[ip]	ip is a standard IPv4 address. The output will be filtered for this address"
>
> Really?  Perhaps the framework-maker should have a filter-command
> option such as -P 'grep 127.0.0.1' that it could pass through.

The above help option was commented. Deleted it now.

>
>> +  FILE=$RUN_DIR/TEMPLATE_PKG_NAME.config
>> +   # make sure file exist and readable
>> +   if [ ! -f $FILE ]; then
>> +  	echo "$FILE : does not exist"
>> +   elif [ ! -r $FILE ]; then
>> +  	echo "$FILE: cannot read"
>> +   fi
>
> Why does this template need to exist?  The framework maker script
> could transcribe all of the options right into the output script.

We want to provide a set of 'default' options. This framework could be 
used by administrators to create packages to diagnose problems on 
customer machines, in which case the admins might need to put in default 
values which they could use to filter the output. The config file will 
always exist because there would be atleast one default option (time).

Regards,
Anithra.



[-- Attachment #2: distributionframework_v2.patch --]
[-- Type: text/plain, Size: 12508 bytes --]

diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
new file mode 100644
index 0000000..7c358cd
--- /dev/null
+++ b/scripts/build_pkg/README.stap-buildpkg
@@ -0,0 +1,69 @@
+SystemTap distribution framework
+
+This is free software.
+See the COPYING file for redistribution/modification terms.
+
+Introduction:
+
+The SystemTap distribution framework aims to:
+
+1. packages a set of systemtap scripts in a format that circumvents as much as
+possible, external dependencies like kernel-debuginfo.  
+2.  generate a package that will self-extract,  run the script, and post
+process the output with minimal intervention from  users.
+
+Usage: 
+
+stap-buildpkg scriptfile [options] [stap-options]
+Options:
+       -n pkg_name              : specify a package name
+       -v pkg_version           : specify a package version
+       -p post_process script   : post processing script
+       -c pkg_config            : provide config parameters through a file
+       -h help_config           : help text for config parameters
+       All additonal options will be passed to the stap command
+
+pkg_name and pkg_version will be used to set the name and version of the
+resulting executable.A post processing script can be provided using the -p
+option. The post processing script can be used to process the output any which
+way the user choses. This could include filtering the output,offering
+suggestions based on the output data, or even forwarding the output to a
+support team.  
+An optional config file that contains post_processing script specific config
+options and corresponding help entries can also be specified using the -c and
+-h options respectively.For eg: If a script expects the pid and ip address as
+options the pkg_config file could contain the following: 
+pid=5632
+ip=127.0.0.1 
+Note: These options are the 'default' options and can be overridden through
+the command line when the resulting package is executed. These options will be
+passed to post_processing script to filter the output and not to the
+stap/staprun command.  The help_config contains help entries for any
+additional parameters. Eg: 
+ipaddress=[ip] ip is a standard IPv4 address.
+pid=[pid] pid of the process to be monitored.  
+The help text would be appended to the usage/help instructions of the
+resulting package.  
+The resulting package is a self-extracting binary. When invoked it would untar
+itself, execute the script(by invoking staprun) and post process.
+
+The usage for the the resulting package:
+
+<package_name> [options] [parameters]
+
+Options:
+   * --run -r   Runs the scripts
+                for x minutes where x can be passed as a parameter 	
+		in the form time=[x]. The default value is 10 minutes. 
+		Post processing is performed after the script completes.
+   * --start -s Invokes the script as a background process.
+   * --stop -x  Stops the script and performs post processing.
+   * --all      Same as --run
+   * --help     Displays this usage text.
+
+Parameters:
+
+   * time=[x]   x is in minutes. Runs the script for x minutes. valid
+                for --run(-r) o --start(-s) or --all(-a) options only
+   Any other parameter specified in the help_config.
+
diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
new file mode 100755
index 0000000..e70cfc8
--- /dev/null
+++ b/scripts/build_pkg/stap-buildpkg
@@ -0,0 +1,191 @@
+##!/bin/bash
+# 
+#  Script to auto-generate executable packages from stap scripts
+#
+
+prog=stap-buildpkg
+
+echo_usage () {
+  echo $"Usage: $prog scriptfile [options] [stap-options]"
+  echo $"Options:"
+  echo $"       -n pkg_name   		: specify a package name"
+  echo $"       -v pkg_version		: specify a package version"
+  echo $"       -c pkg_config  		: provide config parameters through a file"
+  echo $"       -p post_process script   : post processing script"
+  echo $"       -h help_config		: help text for config parameters"
+  echo $"       All additonal options will be passed to the stap command"
+}
+
+parse_args() {
+while [ -n "$1" ]; do
+     case "$1" in
+        -n)                    PKG_NAME="$2"
+				shift 2
+				echo $PKG_NAME
+                                ;;
+        -v)			PKG_VERSION=$2
+                                shift 2
+				echo $PKG_VERSION
+                                ;;
+        -c) 		        PKG_INPUT_CONFIG=$2
+				echo $PKG_INPUT_CONFIG
+				shift 2
+                                ;;
+        -p)			PKG_POST_SCRIPT=$2
+				echo $PKG_POST_SCRIPT
+				shift 2
+                                ;;
+        -H) 		        PKG_HELP_CONFIG=$2
+				echo $PKG_HELP_CONFIG
+                                shift 2
+                                ;;
+        *)                     STAP_OPTIONS=$@ 
+				echo $STAP_OPTIONS
+                                shift $#
+esac
+done
+}
+
+validate_args() {
+ if test -z $PKG_NAME; then
+   PKG_NAME=`basename $script .stp`
+ fi
+ if test -z $PKG_VERSION; then
+   PKG_VERSION=1.0
+ fi
+ if test -z $PKG_RELEASE; then
+   PKG_RELEASE="0"
+ fi
+ PKG_ARCH=`uname -i`
+}
+
+module_gen() {
+stap -p4 -m $PKG_NAME.ko $script $STAP_OPTIONS
+if [ -f $PKG_NAME.ko ]
+then
+        echo " Module $PKG_NAME.ko generated successfully ";
+else
+        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
+        rm -rf $TEMPDIR
+        exit;
+fi
+}
+
+create_wrappers() {
+echo "Creating install wrapper from template..";
+echo "s/TEMPLATE_PKG_NAME/$PKG_NAME/g" > $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_VERSION/$PKG_VERSION/g" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_RELEASE/$PKG_RELEASE/" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_ARCH/$PKG_ARCH/g" >> $TEMPDIR/sd-installer-patterns
+
+sed -f $TEMPDIR/sd-installer-patterns template.buildpkg > $TEMPDIR/wrapper
+csplit -s $TEMPDIR/wrapper '/extractor template/'
+mv xx00 ${PKG_NAME}_install
+mv xx01 ${PKG_NAME}_binextractor
+
+chmod a+x ${PKG_NAME}_install
+rm $TEMPDIR/sd-installer-patterns
+rm $TEMPDIR/wrapper
+}
+
+copy_files_to_temp() {
+
+runtime=systemtap-runtime
+if test -z $(rpm -qa $runtime); then
+ runtime=systemtap 
+fi
+
+echo "using systemtap runtime executables from $runtime rpm"
+
+cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
+cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.
+
+echo "Copying Systemtap module.."
+mv $PKG_NAME.ko $TEMPDIR/ 
+echo "Copying config file.."
+echo "time=2m" > $TEMPDIR/$PKG_NAME.config
+if [ -f $PKG_INPUT_CONFIG ] && [ "$PKG_INPUT_CONFIG" != "" ]
+then 
+   cat $PKG_INPUT_CONFIG >> $TEMPDIR/$PKG_NAME.config
+fi
+# Copying help file, or adding an empty file if none is provided.
+if [ -f $PKG_HELP_CONFIG ] && [ "$PKG_HELP_CONFIG" != "" ]
+then
+        cp $PKG_HELP_CONFIG $TEMPDIR/$PKG_NAME.help
+else
+        echo " " > $TEMPDIR/$PKG_NAME.help
+fi
+echo "Copying the post processing script"
+# copying the post processing script, or adding an empty file if none is provided.
+if [ -f $PKG_POST_SCRIPT ] && [ "$PKG_POST_SCRIPT" != "" ]
+then
+    cp $PKG_POST_SCRIPT  $TEMPDIR/$PKG_NAME.post
+else
+   echo " " > $TEMPDIR/$PKG_NAME.post
+fi
+chmod +x $TEMPDIR/$PKG_NAME.post
+mv ${PKG_NAME}_install $TEMPDIR/
+}
+
+#------------------------------------------------------------------
+# Mainline script
+#------------------------------------------------------------------
+script=$1
+echo $script
+shift 1
+
+if test -z $script; then
+ echo "Please provide a script name"
+ echo_usage
+ exit
+fi
+
+if [ ! -f "$script" ]; then
+ echo "Script file does not exist"
+ echo "Please provide a valid script name"
+ echo_usage
+ exit
+fi
+
+ 
+#OPTS=`getopt -s bash -u -o 'n:v:R:c:p:h:' -- $@`
+
+#parse arguments
+parse_args $@
+validate_args
+
+# generate a tmp directory to store all components:remember to clean it up !
+TEMPDIR=/tmp/$PKG_NAME-$PKG_VERSION;
+mkdir  $TEMPDIR
+
+# Generate Stap module
+echo "Generating Stap module..";
+module_gen
+
+#generate installer(wrapper) scripts
+create_wrappers
+
+#copy systemtap-runtime files
+copy_files_to_temp
+
+MY_PWD=`pwd`
+cd $TEMPDIR ; tar -cjf $PKG_NAME-$PKG_VERSION.tar.bz2 *
+cd $MY_PWD;
+
+echo "Building the self-extracting binary.."
+mkdir $TEMPDIR/$PKG_NAME
+mv $TEMPDIR/${PKG_NAME}_install $TEMPDIR/$PKG_NAME/
+tar -xjf $TEMPDIR/$PKG_NAME-$PKG_VERSION.tar.bz2 -C $TEMPDIR/$PKG_NAME/
+mv  ${PKG_NAME}_binextractor $TEMPDIR/${PKG_NAME}_binextractor 2>/dev/null
+
+MY_PWD=`pwd`
+cd $TEMPDIR; 
+tar -czf - $PKG_NAME/ >> ${PKG_NAME}_binextractor
+cd $MY_PWD;
+mv $TEMPDIR/${PKG_NAME}_binextractor ./$PKG_NAME-$PKG_VERSION
+echo "The installer is at `pwd`/$PKG_NAME-$PKG_VERSION"
+chmod +x $PKG_NAME-$PKG_VERSION
+
+#Cleanup the mess in temp dir
+rm -rf $TEMPDIR
+exit
diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
new file mode 100644
index 0000000..07d7926
--- /dev/null
+++ b/scripts/build_pkg/template.buildpkg
@@ -0,0 +1,142 @@
+#!/bin/bash
+RUN_DIR=`pwd`
+
+function run()
+{
+echo "Gathering data... Please wait"
+# Log file appears as systemtap_tcpipstat.out
+TS=`date "+%Y%m%d-%H:%M:%S"`
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 &
+stappid=$!
+sleep $time
+echo "Completed"
+mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+kill -SIGINT $stappid
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function start()
+{
+ # check to see if tapset is installed
+echo "Gathering data..."
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out 2>&1 &
+echo "Run TEMPLATE_PKG_NAME --stop to stop the script and process output"
+}
+
+function stop()
+{
+TS=`date "+%Y%m%d-%H:%M:%S"`
+echo "Stopping systemtap script"
+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
+echo "Done"
+mv $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function helptext()
+{
+ echo " "
+ echo "USAGE: TEMPLATE_PKG_NAME [options] [parameters]"
+ echo " "
+ echo "Options: "
+ echo " --run -r	Runs the scripts for x minutes where x can be passed as a parameter using time=[x]. "
+ echo "		The default value is 10 minutes. Post processing is performed after the script completes."
+ echo " --start -s	Runs the script as a background process"
+ echo " --stop -x	Stops the script and performs post processing"
+ echo " --all		Runs the script and processes the output"
+ echo " --help		Displays this usage text"
+ echo " "
+ echo "Parameters: "
+ echo " "
+ echo " time=[x]	x is in minutes. Runs the script for x minutes. valid for --run(-r)"
+ echo "		--start(-s) or --all(-a) options only"
+ 	cat $RUN_DIR/TEMPLATE_PKG_NAME.help
+ echo ""
+}
+
+function parseparameters()
+{
+for i in $@
+do
+        tmpa=${1%%=*}
+        tmpb=${1#$tmpa=}
+        export $tmpa=$tmpb
+        shift
+done
+}
+
+
+option=$1
+shift
+
+
+  FILE=$RUN_DIR/TEMPLATE_PKG_NAME.config
+   # make sure file exist and readable
+   if [ ! -f $FILE ]; then
+  	echo "$FILE : does not exist"
+   elif [ ! -r $FILE ]; then
+  	echo "$FILE: cannot read"
+   fi
+
+
+
+   exec 3<&0
+   exec 0<$FILE
+   while read line
+	do
+        tmpa=${line%%=*}
+        tmpb=${line#$tmpa=}
+        declare $tmpa=$tmpb
+	done
+   exec 0<&3
+
+   for i in $@
+	do
+	        tmpa=${1%%=*}
+        	tmpb=${1#$tmpa=}
+	        export $tmpa=$tmpb
+        	shift
+	done
+
+
+case $option in
+	-r | --run ) 	 			
+				run
+				exit
+				;;
+        -s | --start )         
+				start
+				exit
+                                ;;
+        -h | --help )           helptext
+                                exit
+                                ;;
+        -x | --stop )		stop
+				exit
+				;;
+	-a | --all )		run
+				exit
+				;;
+        * )                     helptext
+                                exit 1
+esac
+
+## extractor template
+#!/bin/sh -e
+sed -e '1,/^exit$/d' "$0" | tar -xzf - 
+cd TEMPLATE_PKG_NAME 
+ ./TEMPLATE_PKG_NAME_install $@
+cd ..
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME.*
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME_install
+exit

Re: [RFC] Framework for easy distribution of SystemTap scripts (V2)

[Frank Ch. Eigler]

Hi -

On Fri, Apr 09, 2010 at 09:48:53PM +0530, Anithra P Janakiraman wrote:
> [...]
> Thanks for the review! I've attached a V2 of the patch.

Thanks.

> >>+++ b/scripts/build_pkg/README.stap-buildpkg
> >Could this be a normal man page (stap-buildpkg.1) ?
> 
> We have placed the script in the scripts directory, We dont want to 
> install the script, so didn't see the use for a man page.

Why *not* install it?  How are people supposed to get a hold of your
script if it doesn't get installed?


> >Why does there need to be a separate 'version'?

> To be able to distinguish between packages built for different versions 
> of the same script. We initially had a 'release' too :)

To the extent that all these names are free-form, a user could encode
whatever versioning info he needs right into the main module/package
name, but whatever.


> >>+A post processing script can be provided using the -p option.
> >
> >You should specify what exactly this means.  A '/bin/sh -c' command
> >line to pipe stdout through?
> 
> The post processing script can do anything the administrator creating 
> the package wishes to do, maybe even fwd the output to a support team, 
> or provide suggestions based on the output. Have added an explanation.

The key part is to say that for -p SCRIPT, SCRIPT is actually a file name,
whose contents will be copied into the new package, and which will be run
via a pipe to consume the stdout of stdrun.

> >>+        *)                     STAP_OPTIONS=$@
> >>+				echo $STAP_OPTIONS
> >
> >You'll need to watch the quoting in these $foo variables.
> >They could contain spaces etc.
> 
> Didn't understand the comment. stap-buildpkg script.stp -n "package 
> name" -v 1  would work.

Yes, but you need to check your quotation throughout.  In some cases,
there is "$foo", and in other cases, only $foo.  The difference can
matter in some contexts.  So I'm just saying to check and/or test with
all possible spacey arguments.


> >FWIW, you could include this template within the main buildpkg script, as 
> >in
> >
> >echo>  $PKG<<  'END'
> >function run()
> >....
> >END
> >
> 
> True. This is for simplicity of design. It is intuitively a template and 
> a separate file, so we decided to keep it like that esp since we had no 
> intentions of installing the build-pkg script.

I guess simplicity is in the eye of the beholder.  You are having to
edit the thing with sed and concatenate things before & after: this
too seems clumsy.

From just eyeballing the new version of the scripts, it's not clear
how this template file is located.  (If it were installed, you could
autoconf and @prefix@-inform it.)


> >>+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs 
> >>kill -SIGINT
> >
> >You wouldn't need this is you stored $! after the "staprun ...&" a few 
> >lines ago.
> 
> Yes. We had done it this way keeping the older versions of SystemTap in 
> mind, the $! returns pid of 'staprun' in older versions. I've changed 
> the template

(There still appears some ps -ef | grepping in the new version; probably
all of that is unnecessary, if you save child pids properly.)


> >Why does this template need to exist?  The framework maker script
> >could transcribe all of the options right into the output script.
> 
> We want to provide a set of 'default' options.  [...]

But one can provide defaults in lots of ways.  If you believe time-limited
staprun execution is a common and basic option, you can make that an option
provided first class within the stap-buildpkg script, and override it with
a first-class command line options.

To have a separate configuration file for this only, you'd need to
argue why some options have to be treated differently from others.

- FChE

Re: [RFC] Framework for easy distribution of SystemTap scripts (V3)

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 4718 bytes --]

Hi Frank,

Thanks again, attaching a V3

Regards,
Anithra.


On 04/09/2010 10:09 PM, Frank Ch. Eigler wrote:


>>
>> We have placed the script in the scripts directory, We dont want to
>> install the script, so didn't see the use for a man page.
>
> Why *not* install it?  How are people supposed to get a hold of your
> script if it doesn't get installed?

This script will be used by a very select set of users - say admins who 
wish to debug a customer machine, dont think it would be right to 
install it as part of the systemtap/systemtap-runtime rpms as it is not 
going to be used by the 'regular' stap user. It can be obtained from the 
source.


>>>> +A post processing script can be provided using the -p option.
>>>
>>> You should specify what exactly this means.  A '/bin/sh -c' command
>>> line to pipe stdout through?
>>
>> The post processing script can do anything the administrator creating
>> the package wishes to do, maybe even fwd the output to a support team,
>> or provide suggestions based on the output. Have added an explanation.
>
> The key part is to say that for -p SCRIPT, SCRIPT is actually a file name,
> whose contents will be copied into the new package, and which will be run
> via a pipe to consume the stdout of stdrun.

Done. Changed!

>
>>>> +        *)                     STAP_OPTIONS=$@
>>>> +				echo $STAP_OPTIONS
>>>
>>> You'll need to watch the quoting in these $foo variables.
>>> They could contain spaces etc.
>>
>> Didn't understand the comment. stap-buildpkg script.stp -n "package
>> name" -v 1  would work.
>
> Yes, but you need to check your quotation throughout.  In some cases,
> there is "$foo", and in other cases, only $foo.  The difference can
> matter in some contexts.  So I'm just saying to check and/or test with
> all possible spacey arguments.

Done.

>
>
>>> FWIW, you could include this template within the main buildpkg script, as
>>> in
>>>
>>> echo>   $PKG<<   'END'
>>> function run()
>>> ....
>>> END
>>>
>>
>> True. This is for simplicity of design. It is intuitively a template and
>> a separate file, so we decided to keep it like that esp since we had no
>> intentions of installing the build-pkg script.
>
> I guess simplicity is in the eye of the beholder.  You are having to
> edit the thing with sed and concatenate things before&  after: this
> too seems clumsy.

I agree that the sed part may not look very neat, but it is intuitive - 
we replace the place-holders in the template with their corresponding
values. I think it makes the package easier to understand. We can still 
change if you think it is absolutely necessary :)

>
>  From just eyeballing the new version of the scripts, it's not clear
> how this template file is located.  (If it were installed, you could
> autoconf and @prefix@-inform it.)
>

Assuming that the template file will be located in the same directory as 
the stap-buildpkg script, along with the README. Currently placed in the 
scripts dir.

>
>>>> +ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs
>>>> kill -SIGINT
>>>
>>> You wouldn't need this is you stored $! after the "staprun ...&" a few
>>> lines ago.
>>
>> Yes. We had done it this way keeping the older versions of SystemTap in
>> mind, the $! returns pid of 'staprun' in older versions. I've changed
>> the template
>
> (There still appears some ps -ef | grepping in the new version; probably
> all of that is unnecessary, if you save child pids properly.)

With the -start option, staprun executes in the background while the 
script exits. To stop users need to run the package with -stop, it would 
not be possible to store the child pids in this scenario.

>
>
>>> Why does this template need to exist?  The framework maker script
>>> could transcribe all of the options right into the output script.
>>
>> We want to provide a set of 'default' options.  [...]
>
> But one can provide defaults in lots of ways.  If you believe time-limited
> staprun execution is a common and basic option, you can make that an option
> provided first class within the stap-buildpkg script, and override it with
> a first-class command line options.
>
> To have a separate configuration file for this only, you'd need to
> argue why some options have to be treated differently from others.

The problem here is that we are not aware of the options or the no of 
options before hand. This would vary from script to script. The only 
option that we know would be always needed is 'time'. The config file is 
not used by the framework at all, it is needed only by the resulting 
package. Moreover each config option would be associated with a help 
text that can be appended to the resulting package' help text.

Regards,
Anithra.


[-- Attachment #2: distributionframework_v3.patch --]
[-- Type: text/plain, Size: 12597 bytes --]

diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
new file mode 100644
index 0000000..443f841
--- /dev/null
+++ b/scripts/build_pkg/README.stap-buildpkg
@@ -0,0 +1,70 @@
+SystemTap distribution framework
+
+This is free software.
+See the COPYING file for redistribution/modification terms.
+
+Introduction:
+
+The SystemTap distribution framework aims to:
+
+1. packages a set of systemtap scripts in a format that circumvents as much as
+possible, external dependencies like kernel-debuginfo.  
+2.  generate a package that will self-extract,  run the script, and post
+process the output with minimal intervention from  users.
+
+Usage: 
+
+stap-buildpkg scriptfile [options] [stap-options]
+Options:
+       -n pkg_name              : specify a package name
+       -v pkg_version           : specify a package version
+       -p script 		: post processing script
+       -c pkg_config            : provide config parameters through a file
+       -h help_config           : help text for config parameters
+       All additonal options will be passed to the stap command
+
+pkg_name and pkg_version will be used to set the name and version of the
+resulting executable. For the -p script option, script is  a file name, whose
+contents will be copied into the new package. It will be executed post
+completion of the staprun command.  The post processing script can be used to
+process the output any which way the user choses. This could include filtering
+the output,offering suggestions based on the output data, or even forwarding the
+output to a support team.  
+An optional config file that contains post_processing script specific config
+options and corresponding help entries can also be specified using the -c and
+-h options respectively.For eg: If a script expects the pid and ip address as
+options the pkg_config file could contain the following: 
+pid=5632
+ip=127.0.0.1 
+Note: These options are the 'default' options and can be overridden through
+the command line when the resulting package is executed. These options will be
+passed to post_processing script to filter the output and not to the
+stap/staprun command.  The help_config contains help entries for any
+additional parameters. Eg: 
+ipaddress=[ip] ip is a standard IPv4 address.
+pid=[pid] pid of the process to be monitored.  
+The help text would be appended to the usage/help instructions of the
+resulting package.  
+The resulting package is a self-extracting binary. When invoked it would untar
+itself, execute the script(by invoking staprun) and post process.
+
+The usage for the the resulting package:
+
+<package_name> [options] [parameters]
+
+Options:
+   * --run -r   Runs the scripts
+                for x minutes where x can be passed as a parameter 	
+		in the form time=[x]. The default value is 10 minutes. 
+		Post processing is performed after the script completes.
+   * --start -s Invokes the script as a background process.
+   * --stop -x  Stops the script and performs post processing.
+   * --all      Same as --run
+   * --help     Displays this usage text.
+
+Parameters:
+
+   * time=[x]   x is in minutes. Runs the script for x minutes. valid
+                for --run(-r) o --start(-s) or --all(-a) options only
+   Any other parameter specified in the help_config.
+
diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
new file mode 100755
index 0000000..2eb70f3
--- /dev/null
+++ b/scripts/build_pkg/stap-buildpkg
@@ -0,0 +1,191 @@
+##!/bin/bash
+# 
+#  Script to auto-generate executable packages from stap scripts
+#
+
+prog=stap-buildpkg
+
+echo_usage () {
+  echo $"Usage: $prog scriptfile [options] [stap-options]"
+  echo $"Options:"
+  echo $"       -n pkg_name   		: specify a package name"
+  echo $"       -v pkg_version		: specify a package version"
+  echo $"       -c pkg_config  		: provide config parameters through a file"
+  echo $"       -p post_process script   : post processing script"
+  echo $"       -h help_config		: help text for config parameters"
+  echo $"       All additonal options will be passed to the stap command"
+}
+
+parse_args() {
+while [ -n "$1" ]; do
+     case $1 in
+        -n)                    PKG_NAME=$2
+				shift 2
+				echo $PKG_NAME
+                                ;;
+        -v)			PKG_VERSION=$2
+                                shift 2
+				echo $PKG_VERSION
+                                ;;
+        -c) 		        PKG_INPUT_CONFIG=$2
+				echo $PKG_INPUT_CONFIG
+				shift 2
+                                ;;
+        -p)			PKG_POST_SCRIPT=$2
+				echo $PKG_POST_SCRIPT
+				shift 2
+                                ;;
+        -h) 		        PKG_HELP_CONFIG=$2
+				echo $PKG_HELP_CONFIG
+                                shift 2
+                                ;;
+        *)                     STAP_OPTIONS=$@ 
+				echo $STAP_OPTIONS
+                                shift $#
+esac
+done
+}
+
+validate_args() {
+ if test -z $PKG_NAME; then
+   PKG_NAME=`basename $script .stp`
+ fi
+ if test -z $PKG_VERSION; then
+   PKG_VERSION=1.0
+ fi
+ if test -z $PKG_RELEASE; then
+   PKG_RELEASE="0"
+ fi
+ PKG_ARCH=`uname -i`
+}
+
+module_gen() {
+stap -p4 -m $PKG_NAME.ko $script $STAP_OPTIONS
+if [ -f $PKG_NAME.ko ]
+then
+        echo " Module $PKG_NAME.ko generated successfully ";
+else
+        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
+        rm -rf $TEMPDIR
+        exit;
+fi
+}
+
+create_wrappers() {
+echo "Creating install wrapper from template..";
+echo "s/TEMPLATE_PKG_NAME/$PKG_NAME/g" > $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_VERSION/$PKG_VERSION/g" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_RELEASE/$PKG_RELEASE/" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_ARCH/$PKG_ARCH/g" >> $TEMPDIR/sd-installer-patterns
+
+sed -f $TEMPDIR/sd-installer-patterns template.buildpkg > $TEMPDIR/wrapper
+csplit -s $TEMPDIR/wrapper '/extractor template/'
+mv xx00 ${PKG_NAME}_install
+mv xx01 ${PKG_NAME}_binextractor
+
+chmod a+x ${PKG_NAME}_install
+rm $TEMPDIR/sd-installer-patterns
+rm $TEMPDIR/wrapper
+}
+
+copy_files_to_temp() {
+
+runtime=systemtap-runtime
+if test -z $(rpm -qa $runtime); then
+ runtime=systemtap 
+fi
+
+echo "using systemtap runtime executables from $runtime rpm"
+
+cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
+cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.
+
+echo "Copying Systemtap module.."
+mv $PKG_NAME.ko $TEMPDIR/ 
+echo "Copying config file.."
+echo "time=2m" > $TEMPDIR/$PKG_NAME.config
+if [ -f $PKG_INPUT_CONFIG ] && [ "$PKG_INPUT_CONFIG" != "" ]
+then 
+   cat $PKG_INPUT_CONFIG >> $TEMPDIR/$PKG_NAME.config
+fi
+# Copying help file, or adding an empty file if none is provided.
+if [ -f $PKG_HELP_CONFIG ] && [ "$PKG_HELP_CONFIG" != "" ]
+then
+        cp $PKG_HELP_CONFIG $TEMPDIR/$PKG_NAME.help
+else
+        echo " " > $TEMPDIR/$PKG_NAME.help
+fi
+echo "Copying the post processing script"
+# copying the post processing script, or adding an empty file if none is provided.
+if [ -f $PKG_POST_SCRIPT ] && [ "$PKG_POST_SCRIPT" != "" ]
+then
+    cp $PKG_POST_SCRIPT  $TEMPDIR/$PKG_NAME.post
+else
+   echo " " > $TEMPDIR/$PKG_NAME.post
+fi
+chmod +x $TEMPDIR/$PKG_NAME.post
+mv ${PKG_NAME}_install $TEMPDIR/
+}
+
+#------------------------------------------------------------------
+# Mainline script
+#------------------------------------------------------------------
+script=$1
+echo $script
+shift 1
+
+if test -z $script; then
+ echo "Please provide a script name"
+ echo_usage
+ exit
+fi
+
+if [ ! -f "$script" ]; then
+ echo "Script file does not exist"
+ echo "Please provide a valid script name"
+ echo_usage
+ exit
+fi
+
+ 
+#OPTS=`getopt -s bash -u -o 'n:v:R:c:p:h:' -- $@`
+
+#parse arguments
+parse_args $@
+validate_args
+
+# generate a tmp directory to store all components:remember to clean it up !
+TEMPDIR=/tmp/$PKG_NAME-$PKG_VERSION;
+mkdir  $TEMPDIR
+
+# Generate Stap module
+echo "Generating Stap module..";
+#module_gen
+
+#generate installer(wrapper) scripts
+#create_wrappers
+
+#copy systemtap-runtime files
+#copy_files_to_temp
+
+MY_PWD=`pwd`
+cd $TEMPDIR ; tar -cjf $PKG_NAME-$PKG_VERSION.tar.bz2 *
+cd $MY_PWD;
+
+echo "Building the self-extracting binary.."
+mkdir $TEMPDIR/$PKG_NAME
+mv $TEMPDIR/${PKG_NAME}_install $TEMPDIR/$PKG_NAME/
+tar -xjf $TEMPDIR/$PKG_NAME-$PKG_VERSION.tar.bz2 -C $TEMPDIR/$PKG_NAME/
+mv  ${PKG_NAME}_binextractor $TEMPDIR/${PKG_NAME}_binextractor 2>/dev/null
+
+MY_PWD=`pwd`
+cd $TEMPDIR; 
+tar -czf - $PKG_NAME/ >> ${PKG_NAME}_binextractor
+cd $MY_PWD;
+mv $TEMPDIR/${PKG_NAME}_binextractor ./$PKG_NAME-$PKG_VERSION
+echo "The installer is at `pwd`/$PKG_NAME-$PKG_VERSION"
+chmod +x $PKG_NAME-$PKG_VERSION
+
+#Cleanup the mess in temp dir
+rm -rf $TEMPDIR
+exit
diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
new file mode 100644
index 0000000..07d7926
--- /dev/null
+++ b/scripts/build_pkg/template.buildpkg
@@ -0,0 +1,142 @@
+#!/bin/bash
+RUN_DIR=`pwd`
+
+function run()
+{
+echo "Gathering data... Please wait"
+# Log file appears as systemtap_tcpipstat.out
+TS=`date "+%Y%m%d-%H:%M:%S"`
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 &
+stappid=$!
+sleep $time
+echo "Completed"
+mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+kill -SIGINT $stappid
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function start()
+{
+ # check to see if tapset is installed
+echo "Gathering data..."
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out 2>&1 &
+echo "Run TEMPLATE_PKG_NAME --stop to stop the script and process output"
+}
+
+function stop()
+{
+TS=`date "+%Y%m%d-%H:%M:%S"`
+echo "Stopping systemtap script"
+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
+echo "Done"
+mv $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function helptext()
+{
+ echo " "
+ echo "USAGE: TEMPLATE_PKG_NAME [options] [parameters]"
+ echo " "
+ echo "Options: "
+ echo " --run -r	Runs the scripts for x minutes where x can be passed as a parameter using time=[x]. "
+ echo "		The default value is 10 minutes. Post processing is performed after the script completes."
+ echo " --start -s	Runs the script as a background process"
+ echo " --stop -x	Stops the script and performs post processing"
+ echo " --all		Runs the script and processes the output"
+ echo " --help		Displays this usage text"
+ echo " "
+ echo "Parameters: "
+ echo " "
+ echo " time=[x]	x is in minutes. Runs the script for x minutes. valid for --run(-r)"
+ echo "		--start(-s) or --all(-a) options only"
+ 	cat $RUN_DIR/TEMPLATE_PKG_NAME.help
+ echo ""
+}
+
+function parseparameters()
+{
+for i in $@
+do
+        tmpa=${1%%=*}
+        tmpb=${1#$tmpa=}
+        export $tmpa=$tmpb
+        shift
+done
+}
+
+
+option=$1
+shift
+
+
+  FILE=$RUN_DIR/TEMPLATE_PKG_NAME.config
+   # make sure file exist and readable
+   if [ ! -f $FILE ]; then
+  	echo "$FILE : does not exist"
+   elif [ ! -r $FILE ]; then
+  	echo "$FILE: cannot read"
+   fi
+
+
+
+   exec 3<&0
+   exec 0<$FILE
+   while read line
+	do
+        tmpa=${line%%=*}
+        tmpb=${line#$tmpa=}
+        declare $tmpa=$tmpb
+	done
+   exec 0<&3
+
+   for i in $@
+	do
+	        tmpa=${1%%=*}
+        	tmpb=${1#$tmpa=}
+	        export $tmpa=$tmpb
+        	shift
+	done
+
+
+case $option in
+	-r | --run ) 	 			
+				run
+				exit
+				;;
+        -s | --start )         
+				start
+				exit
+                                ;;
+        -h | --help )           helptext
+                                exit
+                                ;;
+        -x | --stop )		stop
+				exit
+				;;
+	-a | --all )		run
+				exit
+				;;
+        * )                     helptext
+                                exit 1
+esac
+
+## extractor template
+#!/bin/sh -e
+sed -e '1,/^exit$/d' "$0" | tar -xzf - 
+cd TEMPLATE_PKG_NAME 
+ ./TEMPLATE_PKG_NAME_install $@
+cd ..
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME.*
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME_install
+exit

Re: [RFC] Framework for easy distribution of SystemTap scripts (V3)

[Frank Ch. Eigler]

Hi -


> >Why *not* install it?  How are people supposed to get a hold of your
> >script if it doesn't get installed?
> 
> This script will be used by a very select set of users - say admins who 
> wish to debug a customer machine, don't think it would be right to 
> install it as part of the systemtap/systemtap-runtime rpms as it is not 
> going to be used by the 'regular' stap user. It can be obtained from the 
> source.

Well, if you are targeting this to such a specific subset of users,
then perhaps there is little need to include the script within the
systemtap source tree.  It could be as easily downloaded from the wiki
for example, no?


> >(There still appears some ps -ef | grepping in the new version; probably
> >all of that is unnecessary, if you save child pids properly.)
> 
> With the -start option, staprun executes in the background while the 
> script exits. To stop users need to run the package with -stop, it would 
> not be possible to store the child pids in this scenario.

(If you used the normal flight-recorder mode startup/shutdown options,
the pids wouldn't be needed, since the module name is used as the
key.)


> [...]
> >To have a separate configuration file for this only, you'd need to
> >argue why some options have to be treated differently from others.
> 
> The problem here is that we are not aware of the options or the no of 
> options before hand. This would vary from script to script.

I guess the help text is not specific as to what these option strings
actually do.  They appear to be made available to the hand-coded
template file as shell variables, which happens to use $time.  How
else are these options conceivably used?  Not staprun module-options;
not apparently post-processing-script environment variables.


- FChE

Re: [RFC] Framework for easy distribution of SystemTap scripts (V4)

[Anithra P Janakiraman]

[-- Attachment #1: Type: text/plain, Size: 3146 bytes --]

On 04/14/2010 02:26 AM, Frank Ch. Eigler wrote:
> Hi -
>
>
>>> Why *not* install it?  How are people supposed to get a hold of your
>>> script if it doesn't get installed?
>>
>> This script will be used by a very select set of users - say admins who
>> wish to debug a customer machine, don't think it would be right to
>> install it as part of the systemtap/systemtap-runtime rpms as it is not
>> going to be used by the 'regular' stap user. It can be obtained from the
>> source.
>
> Well, if you are targeting this to such a specific subset of users,
> then perhaps there is little need to include the script within the
> systemtap source tree.  It could be as easily downloaded from the wiki
> for example, no?

That should be ok too..
>
>
>>> (There still appears some ps -ef | grepping in the new version; probably
>>> all of that is unnecessary, if you save child pids properly.)
>>
>> With the -start option, staprun executes in the background while the
>> script exits. To stop users need to run the package with -stop, it would
>> not be possible to store the child pids in this scenario.
>
> (If you used the normal flight-recorder mode startup/shutdown options,
> the pids wouldn't be needed, since the module name is used as the
> key.)
>

We are assuming that SystemTap might not be installed on the machine 
where the package will be executed. We package the bare minimal 
(staprun/stapio) needed to run the module. The initscripts might not be 
available.

>
>> [...]
>>> To have a separate configuration file for this only, you'd need to
>>> argue why some options have to be treated differently from others.
>>
>> The problem here is that we are not aware of the options or the no of
>> options before hand. This would vary from script to script.
>
> I guess the help text is not specific as to what these option strings
> actually do.  They appear to be made available to the hand-coded
> template file as shell variables, which happens to use $time.  How
> else are these options conceivably used?  Not staprun module-options;
> not apparently post-processing-script environment variables.

The config file and parameters should have been passed to the 
post-processing script. I had completely overlooked that. Thanks for 
pointing it out. Attached the modified template.
The time options is used by the wrapper/template script and the rest are 
passed to the post-processing script.

The help text would be specific to what the options strings do. For eg:
if the config file had  a default:
ipaddress=127.0.0.1

The help text would contain:
ipaddress=[ip] ip is a standard IPv4 address. The output will be 	
	       filtered for this address

The resulting package would have the following --help

Options:
            options explained
Parameters:
     time=[x]   x is in minutes. Runs the script for x minutes. valid	
		for --run(-r) o --start(-s) or --all(-a) options only
     ipaddress=[ip] ip is a standard IPv4 address. The output will be
	      filtered for this address


If an ipaddress is specified at the command line when executing the 
module it would be passed to the post-processing script.

Thanks,
Anithra

[-- Attachment #2: distributionframework_v4.patch --]
[-- Type: text/plain, Size: 12292 bytes --]

diff --git a/scripts/build_pkg/README.stap-buildpkg b/scripts/build_pkg/README.stap-buildpkg
new file mode 100644
index 0000000..443f841
--- /dev/null
+++ b/scripts/build_pkg/README.stap-buildpkg
@@ -0,0 +1,70 @@
+SystemTap distribution framework
+
+This is free software.
+See the COPYING file for redistribution/modification terms.
+
+Introduction:
+
+The SystemTap distribution framework aims to:
+
+1. packages a set of systemtap scripts in a format that circumvents as much as
+possible, external dependencies like kernel-debuginfo.  
+2.  generate a package that will self-extract,  run the script, and post
+process the output with minimal intervention from  users.
+
+Usage: 
+
+stap-buildpkg scriptfile [options] [stap-options]
+Options:
+       -n pkg_name              : specify a package name
+       -v pkg_version           : specify a package version
+       -p script 		: post processing script
+       -c pkg_config            : provide config parameters through a file
+       -h help_config           : help text for config parameters
+       All additonal options will be passed to the stap command
+
+pkg_name and pkg_version will be used to set the name and version of the
+resulting executable. For the -p script option, script is  a file name, whose
+contents will be copied into the new package. It will be executed post
+completion of the staprun command.  The post processing script can be used to
+process the output any which way the user choses. This could include filtering
+the output,offering suggestions based on the output data, or even forwarding the
+output to a support team.  
+An optional config file that contains post_processing script specific config
+options and corresponding help entries can also be specified using the -c and
+-h options respectively.For eg: If a script expects the pid and ip address as
+options the pkg_config file could contain the following: 
+pid=5632
+ip=127.0.0.1 
+Note: These options are the 'default' options and can be overridden through
+the command line when the resulting package is executed. These options will be
+passed to post_processing script to filter the output and not to the
+stap/staprun command.  The help_config contains help entries for any
+additional parameters. Eg: 
+ipaddress=[ip] ip is a standard IPv4 address.
+pid=[pid] pid of the process to be monitored.  
+The help text would be appended to the usage/help instructions of the
+resulting package.  
+The resulting package is a self-extracting binary. When invoked it would untar
+itself, execute the script(by invoking staprun) and post process.
+
+The usage for the the resulting package:
+
+<package_name> [options] [parameters]
+
+Options:
+   * --run -r   Runs the scripts
+                for x minutes where x can be passed as a parameter 	
+		in the form time=[x]. The default value is 10 minutes. 
+		Post processing is performed after the script completes.
+   * --start -s Invokes the script as a background process.
+   * --stop -x  Stops the script and performs post processing.
+   * --all      Same as --run
+   * --help     Displays this usage text.
+
+Parameters:
+
+   * time=[x]   x is in minutes. Runs the script for x minutes. valid
+                for --run(-r) o --start(-s) or --all(-a) options only
+   Any other parameter specified in the help_config.
+
diff --git a/scripts/build_pkg/stap-buildpkg b/scripts/build_pkg/stap-buildpkg
new file mode 100755
index 0000000..61ba597
--- /dev/null
+++ b/scripts/build_pkg/stap-buildpkg
@@ -0,0 +1,192 @@
+##!/bin/bash
+# 
+#  Script to auto-generate executable packages from stap scripts
+#
+
+prog=stap-buildpkg
+
+echo_usage () {
+  echo $"Usage: $prog scriptfile [options] [stap-options]"
+  echo $"Options:"
+  echo $"       -n pkg_name   		: specify a package name"
+  echo $"       -v pkg_version		: specify a package version"
+  echo $"       -c pkg_config  		: provide config parameters through a file"
+  echo $"       -p post_process script   : post processing script"
+  echo $"       -h help_config		: help text for config parameters"
+  echo $"       All additonal options will be passed to the stap command"
+}
+
+parse_args() {
+while [ -n "$1" ]; do
+     case $1 in
+        -n)                    PKG_NAME=$2
+				shift 2
+				echo $PKG_NAME
+                                ;;
+        -v)			PKG_VERSION=$2
+                                shift 2
+				echo $PKG_VERSION
+                                ;;
+        -c) 		        PKG_INPUT_CONFIG=$2
+				echo $PKG_INPUT_CONFIG
+				shift 2
+                                ;;
+        -p)			PKG_POST_SCRIPT=$2
+				echo $PKG_POST_SCRIPT
+				shift 2
+                                ;;
+        -h) 		        PKG_HELP_CONFIG=$2
+				echo $PKG_HELP_CONFIG
+                                shift 2
+                                ;;
+        *)                     STAP_OPTIONS=$@ 
+				echo $STAP_OPTIONS
+                                shift $#
+esac
+done
+}
+
+validate_args() {
+ if test -z $PKG_NAME; then
+   PKG_NAME=`basename $script .stp`
+ fi
+ if test -z $PKG_VERSION; then
+   PKG_VERSION=1.0
+ fi
+ if test -z $PKG_RELEASE; then
+   PKG_RELEASE="0"
+ fi
+ PKG_ARCH=`uname -i`
+}
+
+module_gen() {
+stap -p4 -m $PKG_NAME.ko $script $STAP_OPTIONS
+if [ -f $PKG_NAME.ko ]
+then
+        echo " Module $PKG_NAME.ko generated successfully ";
+else
+        echo " Module $PKG_NAME.ko generation failed. Exiting now... "
+        rm -rf $TEMPDIR
+        exit;
+fi
+}
+
+create_wrappers() {
+echo "Creating install wrapper from template..";
+echo "s/TEMPLATE_PKG_NAME/$PKG_NAME/g" > $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_VERSION/$PKG_VERSION/g" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_PKG_RELEASE/$PKG_RELEASE/" >> $TEMPDIR/sd-installer-patterns
+echo "s/TEMPLATE_ARCH/$PKG_ARCH/g" >> $TEMPDIR/sd-installer-patterns
+
+sed -f $TEMPDIR/sd-installer-patterns template.buildpkg > $TEMPDIR/wrapper
+csplit -s $TEMPDIR/wrapper '/extractor template/'
+mv xx00 ${PKG_NAME}_install
+mv xx01 ${PKG_NAME}_binextractor
+
+chmod a+x ${PKG_NAME}_install
+rm $TEMPDIR/sd-installer-patterns
+rm $TEMPDIR/wrapper
+}
+
+copy_files_to_temp() {
+
+runtime=systemtap-runtime
+if test -z $(rpm -qa $runtime); then
+ runtime=systemtap 
+fi
+
+echo "using systemtap runtime executables from $runtime rpm"
+
+cp `rpm -ql $runtime | grep staprun | grep -v staprun.8.gz` $TEMPDIR/.
+cp `rpm -ql $runtime | grep stapio` $TEMPDIR/.
+
+echo "Copying Systemtap module.."
+mv $PKG_NAME.ko $TEMPDIR/ 
+echo "Copying config file.."
+if [ -f $PKG_INPUT_CONFIG ] && [ "$PKG_INPUT_CONFIG" != "" ]
+then 
+   cp $PKG_INPUT_CONFIG  $TEMPDIR/$PKG_NAME.config
+else
+   echo " " > $TEMPDIR/$PKG_NAME.config
+fi
+# Copying help file, or adding an empty file if none is provided.
+if [ -f $PKG_HELP_CONFIG ] && [ "$PKG_HELP_CONFIG" != "" ]
+then
+        cp $PKG_HELP_CONFIG $TEMPDIR/$PKG_NAME.help
+else
+        echo " " > $TEMPDIR/$PKG_NAME.help
+fi
+echo "Copying the post processing script"
+# copying the post processing script, or adding an empty file if none is provided.
+if [ -f $PKG_POST_SCRIPT ] && [ "$PKG_POST_SCRIPT" != "" ]
+then
+    cp $PKG_POST_SCRIPT  $TEMPDIR/$PKG_NAME.post
+else
+   echo " " > $TEMPDIR/$PKG_NAME.post
+fi
+chmod +x $TEMPDIR/$PKG_NAME.post
+mv ${PKG_NAME}_install $TEMPDIR/
+}
+
+#------------------------------------------------------------------
+# Mainline script
+#------------------------------------------------------------------
+script=$1
+echo $script
+shift 1
+
+if test -z $script; then
+ echo "Please provide a script name"
+ echo_usage
+ exit
+fi
+
+if [ ! -f "$script" ]; then
+ echo "Script file does not exist"
+ echo "Please provide a valid script name"
+ echo_usage
+ exit
+fi
+
+ 
+#OPTS=`getopt -s bash -u -o 'n:v:R:c:p:h:' -- $@`
+
+#parse arguments
+parse_args $@
+validate_args
+
+# generate a tmp directory to store all components:remember to clean it up !
+TEMPDIR=/tmp/$PKG_NAME-$PKG_VERSION;
+mkdir  $TEMPDIR
+
+# Generate Stap module
+echo "Generating Stap module..";
+module_gen
+
+#generate installer(wrapper) scripts
+create_wrappers
+
+#copy systemtap-runtime files
+copy_files_to_temp
+
+MY_PWD=`pwd`
+cd $TEMPDIR ; tar -cjf $PKG_NAME-$PKG_VERSION.tar.bz2 *
+cd $MY_PWD;
+
+echo "Building the self-extracting binary.."
+mkdir $TEMPDIR/$PKG_NAME
+mv $TEMPDIR/${PKG_NAME}_install $TEMPDIR/$PKG_NAME/
+tar -xjf $TEMPDIR/$PKG_NAME-$PKG_VERSION.tar.bz2 -C $TEMPDIR/$PKG_NAME/
+mv  ${PKG_NAME}_binextractor $TEMPDIR/${PKG_NAME}_binextractor 2>/dev/null
+
+MY_PWD=`pwd`
+cd $TEMPDIR; 
+tar -czf - $PKG_NAME/ >> ${PKG_NAME}_binextractor
+cd $MY_PWD;
+mv $TEMPDIR/${PKG_NAME}_binextractor ./$PKG_NAME-$PKG_VERSION
+echo "The installer is at `pwd`/$PKG_NAME-$PKG_VERSION"
+chmod +x $PKG_NAME-$PKG_VERSION
+
+#Cleanup the mess in temp dir
+rm -rf $TEMPDIR
+exit
diff --git a/scripts/build_pkg/template.buildpkg b/scripts/build_pkg/template.buildpkg
new file mode 100644
index 0000000..0bb790e
--- /dev/null
+++ b/scripts/build_pkg/template.buildpkg
@@ -0,0 +1,117 @@
+#!/bin/bash
+RUN_DIR=`pwd`
+time=2m
+
+function run()
+{
+echo "Gathering data... Please wait"
+# Log file appears as systemtap_tcpipstat.out
+TS=`date "+%Y%m%d-%H:%M:%S"`
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME.out 2>&1 &
+stappid=$!
+sleep $time
+echo "Completed"
+mv $RUN_DIR/TEMPLATE_PKG_NAME.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+kill -SIGINT $stappid
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out $RUN_DIR/TEMPLATE_PKG_NAME.config $postoptions
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function start()
+{
+ # check to see if tapset is installed
+echo "Gathering data..."
+export SYSTEMTAP_STAPIO=$RUN_DIR/stapio
+$RUN_DIR/staprun $RUN_DIR/TEMPLATE_PKG_NAME.ko > $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out 2>&1 &
+echo "Run TEMPLATE_PKG_NAME --stop to stop the script and process output"
+}
+
+function stop()
+{
+TS=`date "+%Y%m%d-%H:%M:%S"`
+echo "Stopping systemtap script"
+ps -ef | grep TEMPLATE_PKG_NAME | grep stapio | awk '{print $2}' | xargs kill -SIGINT
+echo "Done"
+mv $RUN_DIR/TEMPLATE_PKG_NAME_tmp.out $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out
+echo "Output file TEMPLATE_PKG_NAME_${TS}.out is in $RUN_DIR directory"
+echo "Postprocessing..."
+$RUN_DIR/TEMPLATE_PKG_NAME.post $RUN_DIR/TEMPLATE_PKG_NAME_${TS}.out $RUN_DIR/TEMPLATE_PKG_NAME.config $postoptions
+rm $RUN_DIR/staprun
+rm $RUN_DIR/stapio
+}
+
+function helptext()
+{
+ echo " "
+ echo "USAGE: TEMPLATE_PKG_NAME [options] [parameters]"
+ echo " "
+ echo "Options: "
+ echo " --run -r	Runs the scripts for x minutes where x can be passed as a parameter using time=[x]. "
+ echo "		The default value is 10 minutes. Post processing is performed after the script completes."
+ echo " --start -s	Runs the script as a background process"
+ echo " --stop -x	Stops the script and performs post processing"
+ echo " --all		Runs the script and processes the output"
+ echo " --help		Displays this usage text"
+ echo " "
+ echo "Parameters: "
+ echo " "
+ echo " time=[x]	x is in minutes. Runs the script for x minutes. valid for --run(-r)"
+ echo "		--start(-s) or --all(-a) options only"
+ 	cat $RUN_DIR/TEMPLATE_PKG_NAME.help
+ echo ""
+}
+
+
+option=$1
+shift
+
+
+   postoptions=$@
+
+   for i in $@
+	do
+	        tmpa=${1%%=*}
+                if [ "$tmpa" == "time" ]
+                then
+        	tmpb=${1#$tmpa=}
+	        export $tmpa=$tmpb
+                break
+                fi
+        	shift
+	done
+
+case $option in
+	-r | --run ) 	 			
+				run
+				exit
+				;;
+        -s | --start )         
+				start
+				exit
+                                ;;
+        -h | --help )           helptext
+                                exit
+                                ;;
+        -x | --stop )		stop
+				exit
+				;;
+	-a | --all )		run
+				exit
+				;;
+        * )                     helptext
+                                exit 1
+esac
+
+## extractor template
+#!/bin/sh -e
+sed -e '1,/^exit$/d' "$0" | tar -xzf - 
+cd TEMPLATE_PKG_NAME 
+ ./TEMPLATE_PKG_NAME_install $@
+cd ..
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME.*
+rm -f TEMPLATE_PKG_NAME/TEMPLATE_PKG_NAME_install
+exit

Re: [RFC] Framework for easy distribution of SystemTap scripts (V4)

[Frank Ch. Eigler]

Hi -

On Wed, Apr 14, 2010 at 09:48:21PM +0530, Anithra P Janakiraman wrote:
> [...]
> >Well, if you are targeting this to such a specific subset of users,
> >then perhaps there is little need to include the script within the
> >systemtap source tree.  It could be as easily downloaded from the wiki
> >for example, no?
> 
> That should be ok too..

OK, in that case, feel free to post it to the wiki as/when you like.


> >>With the -start option, staprun executes in the background while the
> >>script exits. To stop users need to run the package with -stop, it would
> >>not be possible to store the child pids in this scenario.
> >
> >(If you used the normal flight-recorder mode startup/shutdown options,
> >the pids wouldn't be needed, since the module name is used as the
> >key.)
> 
> We are assuming that SystemTap might not be installed on the machine 
> where the package will be executed. We package the bare minimal 
> (staprun/stapio) needed to run the module. The initscripts might not be 
> available.

I wasn't talking about initscripts, but about stap's -F and staprun's
-L/-A options.


> >>[...]  The config file and parameters should have been passed to
> the post-processing script. I had completely overlooked that.  [...]

OK.  It almost sounds like what you'd need is another post-processing
option, where the user specifies a command line (rather than a file
name).  Then the 'ipaddress' sorts of filtering would be simply:

    stap-makepkg --filter-output 'grep 127.0.0.1' .....

with no post-processing script/file.


- FChE