Is there a roadmap for the STAP development?

Is there a roadmap for the STAP development?

[Arkady]

Hi,

I am considering some patches for SystemTap - mostly fixing minor
inconveniences. Obviously I would like to align the efforts and,
hopefully merge the patches into the master. Is there a document
describing the SystemTap development roadmap?

Examples of things I would like to extend

* Add build only flag in the stap utility which does not require root privileges
* Add RPM spec file supporting automatic build of the kernel driver
after kernel updates
* In the _stp_reserve_bytes() return indication that the trace buffer
is overwritten
* Add support for zero copy data transfer

Etc

Thanks

Re: Is there a roadmap for the STAP development?

[David Smith]

On 02/14/2017 12:51 AM, Arkady wrote:
> Hi,
> 
> I am considering some patches for SystemTap - mostly fixing minor
> inconveniences. Obviously I would like to align the efforts and,
> hopefully merge the patches into the master. Is there a document
> describing the SystemTap development roadmap?

Hmm, not that I know of. Some items of future development are tracked as
enhancements in sourceware's bugzilla.

> Examples of things I would like to extend
> 
> * Add build only flag in the stap utility which does not require root privileges

Different from '-p4'?

> * Add RPM spec file supporting automatic build of the kernel driver
> after kernel updates

What do you mean by "the kernel driver"?

> * In the _stp_reserve_bytes() return indication that the trace buffer
> is overwritten
> * Add support for zero copy data transfer

You'd have to flesh those last two items out a bit for me to understand
what you are looking to do.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Is there a roadmap for the STAP development?

[Arkady]

On Tue, Feb 14, 2017 at 6:13 PM, David Smith <dsmith@redhat.com> wrote:
> On 02/14/2017 12:51 AM, Arkady wrote:
>> Hi,
>>
>> I am considering some patches for SystemTap - mostly fixing minor
>> inconveniences. Obviously I would like to align the efforts and,
>> hopefully merge the patches into the master. Is there a document
>> describing the SystemTap development roadmap?
>
> Hmm, not that I know of. Some items of future development are tracked as
> enhancements in sourceware's bugzilla.
>
>> Examples of things I would like to extend
>>
>> * Add build only flag in the stap utility which does not require root privileges
>
> Different from '-p4'?

Yes, -p4 does the trick. I missed it.

>
>> * Add RPM spec file supporting automatic build of the kernel driver
>> after kernel updates
>
> What do you mean by "the kernel driver"?

I need to build the module automatically every time the kernel
updates. I am preparing the RPM/DEB which does the trick, but to have
a generic example in /docs/examples would be nice or even a script
which generates such RPM based on the stap command line options.

>
>> * In the _stp_reserve_bytes() return indication that the trace buffer
>> is overwritten
>> * Add support for zero copy data transfer
>
> You'd have to flesh those last two items out a bit for me to understand
> what you are looking to do.

When I send the events from the stap module to the user space I am doing
event_type* event = _stp_reserve_bytes(sizeof(event_type));
event->field1 = value1;
.....
I do not handle case when _stp_reserve_bytes() starts to overwrite the
data in the trace buffer. It would be cool to have some feedback that
buffer is full or is getting full. In some situations it is better to
drop the event and avoid overwriting the buffer.

In another approach I am planning to use the shared memory for sending
events from the kernel to the user space in one specific case - very
frequent system calls. Similarly to transport.c with zero copy API.



>
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)

Re: Is there a roadmap for the STAP development?

[David Smith]

On 02/14/2017 12:00 PM, Arkady wrote:
> On Tue, Feb 14, 2017 at 6:13 PM, David Smith <dsmith@redhat.com> wrote:

... stuff deleted ...

>>> * Add RPM spec file supporting automatic build of the kernel driver
>>> after kernel updates
>>
>> What do you mean by "the kernel driver"?
> 
> I need to build the module automatically every time the kernel
> updates. I am preparing the RPM/DEB which does the trick, but to have
> a generic example in /docs/examples would be nice or even a script
> which generates such RPM based on the stap command line options.

Hmm, interesting. This sounds a bit like the kmod system that rebuilds
external kernel modules when the kernel gets updated.

>>> * In the _stp_reserve_bytes() return indication that the trace buffer
>>> is overwritten
>>> * Add support for zero copy data transfer
>>
>> You'd have to flesh those last two items out a bit for me to understand
>> what you are looking to do.
> 
> When I send the events from the stap module to the user space I am doing
> event_type* event = _stp_reserve_bytes(sizeof(event_type));
> event->field1 = value1;
> .....
> I do not handle case when _stp_reserve_bytes() starts to overwrite the
> data in the trace buffer. It would be cool to have some feedback that
> buffer is full or is getting full. In some situations it is better to
> drop the event and avoid overwriting the buffer.

As a side note I'll state that you seem to be depending on internal
systemtap interfaces here by using _stp_reserve_bytes(). In general we
try to keep "external" interfaces, like tapsets, compatible between
systemtap versions. There is no such guarantee for internal interfaces.

(That being said, I know of no plans to change _stp_reserve_bytes().)

> In another approach I am planning to use the shared memory for sending
> events from the kernel to the user space in one specific case - very
> frequent system calls. Similarly to transport.c with zero copy API.

Interesting. Yep, that code would be interesting to see.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Is there a roadmap for the STAP development?

[Arkady]

On Thu, Feb 16, 2017 at 9:38 PM, David Smith <dsmith@redhat.com> wrote:
> On 02/14/2017 12:00 PM, Arkady wrote:
>> On Tue, Feb 14, 2017 at 6:13 PM, David Smith <dsmith@redhat.com> wrote:
>
> ... stuff deleted ...
>
>>>> * Add RPM spec file supporting automatic build of the kernel driver
>>>> after kernel updates
>>>
>>> What do you mean by "the kernel driver"?
>>
>> I need to build the module automatically every time the kernel
>> updates. I am preparing the RPM/DEB which does the trick, but to have
>> a generic example in /docs/examples would be nice or even a script
>> which generates such RPM based on the stap command line options.
>
> Hmm, interesting. This sounds a bit like the kmod system that rebuilds
> external kernel modules when the kernel gets updated.
>
> ... stuff deleted ...
>> In another approach I am planning to use the shared memory for sending
>> events from the kernel to the user space in one specific case - very
>> frequent system calls. Similarly to transport.c with zero copy API.
>
> Interesting. Yep, that code would be interesting to see.

Here is an example of zero copy support
https://github.com/larytet/YALAS/blob/master/src/shm_fifo.h#L40
and a macro which allocates the data
https://github.com/larytet/YALAS/blob/master/src/driver.h#L284

Arkady.
>
> --
> David Smith
> dsmith@redhat.com
> Red Hat
> http://www.redhat.com
> 256.217.0141 (direct)
> 256.837.0057 (fax)