Re: Fwd: systemtap global var lead to high cpu

[Kun <mingkunone@qq.com>]

Hi，
How to embed your module into systemtap？
it prompts some error when i run dup_probe.stp，
semantic error：while resolving probe point: identifier ‘kernel’ at kprocess.stp:29:25
    Source: probe kprocess.creat = kernel.function(“copy_process”).return
    Thank you




------------------ Original ------------------
From: Arkady <arkady.miasnikov@gmail.com&gt;
Date: Fri,Jul 24,2020 11:49 AM
To: Kun <mingkunone@qq.com&gt;
Cc: Serhei Makarov <smakarov@redhat.com&gt;, systemtap <systemtap@sourceware.org&gt;
Subject: Re: Fwd: systemtap global var lead to high cpu



Hi Kun,

I avoid locks by using lockless data structures. For example, I have
my own hashtables instead of associative arrays.
You can check this code for inspiration
https://github.com/larytet/lockfree_hashtable

If the performance is paramount for your task you can switch to 100%
handwritten C code.

Arkady,

On Fri, Jul 24, 2020 at 5:01 AM Kun via Systemtap
<systemtap@sourceware.org&gt; wrote:
&gt;
&gt; My global var saved is string type, not number, so I can not use aggregate.
&gt; FChE will fixed this bug,
&gt; https://sourceware.org/bugzilla/show_bug.cgi?id=26296
&gt; Hoping to fixed soon.
&gt;
&gt;
&gt;
&gt;
&gt; ------------------ Original ------------------
&gt; From: Serhei Makarov <smakarov@redhat.com&amp;gt;
&gt; Date: Thu,Jul 23,2020 10:22 PM
&gt; To: systemtap <systemtap@sourceware.org&amp;gt;, mingkunone <mingkunone@qq.com&amp;gt;
&gt; Subject: Re: Fwd: systemtap global var lead to high cpu
&gt;
&gt;
&gt;
&gt; Forwarding your question to systemtap@sourceware.org in case other people have suggestions.
&gt;
&gt;
&gt;
&gt; In general, the locks protect concurrent modifications from interfering with each other when different processes trigger the same probe.
&gt;
&gt;
&gt; Depending on what&amp;nbsp; you want to do with the iphdr value, you may be able to reduce contention by using statistical aggregates
&gt; (which do not require locking). For example, aaa[iphdr] <<< some_statistic; in a later probe iterate through aaa. A lot of the SystemTap example scripts use this type of structure, for example: https://sourceware.org/systemtap/examples/network/netfilter_summary.stp
&gt;
&gt;
&gt; Otherwise, every tcp_ack() for every packet on your system will try to grab the same lock. The resulting CPU load is unsurprising to me.
&gt;
&gt;
&gt; Hope this information is helpful; if not, someone else may have a better suggestion.
&gt;
&gt;
&gt; All the best,
&gt; &amp;nbsp;&amp;nbsp;&amp;nbsp; &amp;nbsp; Serhei
&gt;
&gt;
&gt; ---------- Forwarded message ---------
&gt; From: Kun <mingkunone@qq.com&amp;gt;
&gt; Date: Wed, Jul 22, 2020 at 11:24 PM
&gt; Subject: systemtap global var lead to high cpu
&gt; To: smakarov <smakarov@redhat.com&amp;gt;
&gt;
&gt;
&gt;
&gt; Hi,
&gt; &amp;nbsp; &amp;nbsp; I have a problem of system tap which
&gt; &amp;nbsp;Using global var.
&gt; &amp;nbsp; &amp;nbsp; A simple demo as following,
&gt; Global aaa
&gt; Probe kernel.function(“tcp_ack”){
&gt; &amp;nbsp; &amp;nbsp; Iphdr = __get_skb_iphdr($skb)
&gt; &amp;nbsp; &amp;nbsp; If(iphdr == 0){
&gt; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; aaa=iphdr
&gt; &amp;nbsp; &amp;nbsp; }
&gt;
&gt;
&gt; Then our env have a 10Gbps flow, and our cpu is nearly 100%.
&gt;
&gt;
&gt; Analysising c code, I find this is because of a lock as following:
&gt; Static void probe_6330()
&gt; {
&gt; If(sta_lock_probe(lock, ARRAY_SIZE(locks))
&gt; &amp;nbsp; &amp;nbsp; Return;
&gt; If(l-&amp;gt;iphdr == 0){
&gt; &amp;nbsp; &amp;nbsp; Global(s_global_aaa)=l-&amp;gt;l_iphdr;
&gt; }
&gt;
&gt;
&gt; My question is that aaa should be protected in “if”,
&gt; Why is the lock directly at the function entry?