Trying to get an embedded C function to use file I/O

Trying to get an embedded C function to use file I/O

[Jeff Johnston]

I'm trying to use regular C I/O in an embedded C function for SystemTap. 
  When I try and do:

%{
#include <stdio.h>
%}

I get an error that SystemTap cannot find stdio.h.  I am using -g for 
guru mode.

Is there some form of restriction on C stdio?

I want to use it because I ultimately need to create multiple files from 
trace data and -o does not allow me to do this.

-- Jeff J.

Re: Trying to get an embedded C function to use file I/O

[Eric Wong]

Jeff Johnston <jjohnstn@redhat.com> wrote:
> I'm trying to use regular C I/O in an embedded C function for
> SystemTap.  When I try and do:
> 
> %{
> #include <stdio.h>
> %}
> 
> I get an error that SystemTap cannot find stdio.h.  I am using -g
> for guru mode.
> 
> Is there some form of restriction on C stdio?

I think guru mode runs the code in kernel space.  I tried using
strftime() the other day and could not.  Maybe dyninst lets one use
glibc, I haven't looked into it more...

(disclaimer: I'm no systemtap expert)

Re: Trying to get an embedded C function to use file I/O

[Frank Ch. Eigler]

jjohnstn wrote:

> [...]
> Is there some form of restriction on C stdio?

There are several restrictions on embedded-C code; you ran into one
associated with the default kernel-module based backend.

> I want to use it because I ultimately need to create multiple files
> from trace data and -o does not allow me to do this.

Can you elaborate?  Until systemtap grows a good native ability, could
you output into a pipe and let a userspace filter separate the items?

- FChE

Re: Trying to get an embedded C function to use file I/O

[Jeff Johnston]

On 08/17/2013 07:48 PM, Frank Ch. Eigler wrote:
>
> jjohnstn wrote:
>
>> [...]
>> Is there some form of restriction on C stdio?
>
> There are several restrictions on embedded-C code; you ran into one
> associated with the default kernel-module based backend.
>
>> I want to use it because I ultimately need to create multiple files
>> from trace data and -o does not allow me to do this.
>
> Can you elaborate?  Until systemtap grows a good native ability, could
> you output into a pipe and let a userspace filter separate the items?
>
> - FChE
>

I am trying to use SystemTap to replace the LTTng lttng-modules package 
which requires building and installation of kernel modules.  The user of 
LTTng uses the lttng-tools package to run the lttng command to create a 
session trace, select events and channels, start a trace, stop a trace, 
and end a session.

The LTTng kernel side of the trace ends up with multiple CTF format 
files: a metadata file which describes the data format and a number of 
binary files (essentially a permutation of CPUs and channels (usually 
just channel 0).

I had thought to have the ability for someone to do this manually using 
a SystemTap script and providing key functions (e.g. 
ctf_output_metadata(), ctf_init_headers(), ctf_syscall_event()).

With the restriction, I guess the toolset could be rewritten (or a new 
toolset introduced) to use SystemTap under the covers and so a filter is 
certainly feasible.

-- Jeff J.

Re: Trying to get an embedded C function to use file I/O

[Josh Stone]

On 08/16/2013 11:17 AM, Eric Wong wrote:
> I think guru mode runs the code in kernel space.  I tried using
> strftime() the other day and could not.  Maybe dyninst lets one use
> glibc, I haven't looked into it more...

Correct - the entire script runs in kernel mode, so any embedded-C you
add can only use kernel headers and functions.  This is not directly an
artifact of guru, really -- it's just that guru is required to let you
write embedded-c at all.  For your own good. :)

And you're also correct that --runtime=dyninst can use glibc, because in
this mode the script will be running as a plain shared library.  In
theory, you could use *any* header and library on your system, although
I don't think we currently give any way to add compiler flags.  You also
have to be aware of the process space though, e.g. a file you open() in
one process isn't going to be open in all involved processes.

Re: Trying to get an embedded C function to use file I/O

[Frank Ch. Eigler]

jjohnstn wrote:

> [...]  The LTTng kernel side of the trace ends up with multiple CTF
> format files: a metadata file which describes the data format and a
> number of binary files (essentially a permutation of CPUs and
> channels (usually just channel 0). [...]

Is lttng restricted to having only a single 'channel' or dataset per
file?  If not, why not 

(a) have one monster CTF file that includes cross-cpu cross-channel
    cross-everything data?
or
(b) use stap -b (bulk) mode to produce per-cpu files, each of which
    could be converted to CTF in userspace (perhaps requiring nothing
    but removal of the stap timestamping/framing metadata in those
    bulk-mode files; see stap-merge.c).

- FChE

Re: Trying to get an embedded C function to use file I/O

[Jeff Johnston]

Thanks Frank.  I think option b sounds most promising as they
separate per-cpu data and so I think I'll let SystemTap do that work.

-- Jeff J.

----- Original Message -----
From: "Frank Ch. Eigler" <fche@redhat.com>
To: "Jeff Johnston" <jjohnstn@redhat.com>
Cc: systemtap@sourceware.org
Sent: Monday, August 19, 2013 6:02:12 PM
Subject: Re: Trying to get an embedded C function to use file I/O


jjohnstn wrote:

> [...]  The LTTng kernel side of the trace ends up with multiple CTF
> format files: a metadata file which describes the data format and a
> number of binary files (essentially a permutation of CPUs and
> channels (usually just channel 0). [...]

Is lttng restricted to having only a single 'channel' or dataset per
file?  If not, why not 

(a) have one monster CTF file that includes cross-cpu cross-channel
    cross-everything data?
or
(b) use stap -b (bulk) mode to produce per-cpu files, each of which
    could be converted to CTF in userspace (perhaps requiring nothing
    but removal of the stap timestamping/framing metadata in those
    bulk-mode files; see stap-merge.c).

- FChE