Systemtap filghtrecorder over network

Systemtap filghtrecorder over network

[Jelle Smet]

Hi list,

I was wondering whether the flight recorder mode has besides a "to 
file" and "in memory" function also a way to transmit its output over 
network?

I read in the following document a reference that there's working 
pending related to this:

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&ved=0CB0QFjAA&url=http%3A%2F%2Fevents.linuxfoundation.org%2Fslides%2F2010%2Flinuxcon_japan%2Flinuxcon_jp2010_hiramatsu.pdf&ei=e2KEUJPsJtGT0QXB9ICABQ&usg=AFQjCNHjavabllXapGoDsx3TwnozOhAcSg&sig2=yen154eYwD_KDQACKeEf2w

see page 20.

Thanks,

-- 

Jelle Smet
http://smetj.net

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: Systemtap filghtrecorder over network

[Frank Ch. Eigler]

Jelle Smet <systemtap@smetj.net> writes:

> [...]
> I was wondering whether the flight recorder mode has besides a "to
> file" and "in memory" function also a way to transmit its output over
> network?

"over the network" sounds like ordinary remote invocation, such as with 

% stap --remote=USER@HOST foo.stp

which uses ssh and a remote systemtap-runtime to run foo.stp and transport
the output back.


- FChE

Re: Systemtap filghtrecorder over network

[Jelle Smet]

On 22.10.2012 16:06, fche@redhat.com wrote:

Yeah, I see, but that's not what I was looking for.
This approach is difficult to scale when dealing with many nodes.
It would be practical to let stap run on a machine dumping its data
over an UDP socket just like one would pipe its stdout through
netcat.  Having this functionality built in however would be nice.

Cheers,

Jelle

> Jelle Smet <systemtap@smetj.net> writes:
>
>> [...] I was wondering whether the flight recorder mode has besides a
>> "to file" and "in memory" function also a way to transmit its output
>> over network?
>
> "over the network" sounds like ordinary remote invocation, such as 
> with
>
> % stap --remote=USER@HOST foo.stp
>
> which uses ssh and a remote systemtap-runtime to run foo.stp and
> transport
> the output back.
>
> - FChE

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: Systemtap filghtrecorder over network

[Frank Ch. Eigler]

Hi -

> [...]
> It would be practical to let stap run on a machine dumping its data
> over an UDP socket just like one would pipe its stdout through
> netcat.  Having this functionality built in however would be nice.

OK, what would you imagine, other than:

   stap FOO.stp | nc -u LOGHOST LOGPORT

(Automated remote invocation is trickier because it involves
authentication/authorization and lots of other setup.)

- FChE

Re: Systemtap filghtrecorder over network

[Jelle Smet]

stap -F -u LOGHOST -p LOGPORT FOO.stp

Remote invocation is indeed out of scope, just the ability to have
native network transport in stap itself.


- Jelle

On 22.10.2012 16:37, Frank Ch. Eigler wrote:

> Hi -
>
>> [...] It would be practical to let stap run on a machine dumping its
>> data over an UDP socket just like one would pipe its stdout through
>> netcat. Having this functionality built in however would be nice.
>
> OK, what would you imagine, other than:
>
> stap FOO.stp | nc -u LOGHOST LOGPORT
>
> (Automated remote invocation is trickier because it involves
> authentication/authorization and lots of other setup.)
>
> - FChE

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Re: Systemtap filghtrecorder over network

[Frank Ch. Eigler]

Hi -

>> stap FOO.stp | nc -u LOGHOST LOGPORT

> stap -F -u LOGHOST -p LOGPORT FOO.stp

To the extent the latter is syntactic sugar for the former, its better
place could be a shell script wrapper.  I wonder if I'm missing some
more convincing reason to include support right within stap.  If you
can think of some, please feel free to open an enhancement request in
our bugzilla.

- FChE

Re: Systemtap filghtrecorder over network

[Jelle Smet]

Hi,

There's a performance gain when writing directly to a socket using 
proper buffering compared to redirecting stdout to another process in 
order to send stap output over network. This small poc is in Python but 
should be relevant nevertheless: https://gist.github.com/3935398
Besides that it's from an sysadmin point of view cleaner to have 
network functionality built in?  One less dependency, native support, no 
wrapper scripts required.
Writing output to a file could also be done by using a STDOUT redirect 
to a file, yet the functionality is built in, which is the cleanest 
approach.

The motivation for being able to transmit data over network using a 
simple network protocol is that it would make life easier to centralize
a large amount of metrics from a large amount of hosts.
It's also not required to foresee any local disk space nor memory to 
store data, which could be handy on VM environments with a higher guest 
density.
I would *personally* find UDP more than sufficient, it would also be 
less intrusive or blocking for stap when the remote destination isn't 
available.

I've submitted a feature request here 
http://sources.redhat.com/bugzilla/show_bug.cgi?id=14755 as you 
suggested.

Thanks,

Jelle


On 22.10.2012 20:42, fche@redhat.com wrote:

> Hi -
>
>>> stap FOO.stp | nc -u LOGHOST LOGPORT
>
>> stap -F -u LOGHOST -p LOGPORT FOO.stp
>
> To the extent the latter is syntactic sugar for the former, its 
> better
> place could be a shell script wrapper. I wonder if I'm missing some
> more convincing reason to include support right within stap. If you
> can think of some, please feel free to open an enhancement request in
> our bugzilla.
>
> - FChE

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.