From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x343.google.com (mail-ot1-x343.google.com [IPv6:2607:f8b0:4864:20::343]) by sourceware.org (Postfix) with ESMTPS id 1B22C3857C41 for ; Sat, 25 Jul 2020 05:25:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1B22C3857C41 Received: by mail-ot1-x343.google.com with SMTP id d4so8630613otk.2 for ; Fri, 24 Jul 2020 22:25:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=pM6Z5JI/hMxpBzQYfgr3PYhogkK3kM7SCUQiLbeQ8Wg=; b=go1zEZ+48jWnhkCeaGHmLVVaWukP5NjFsN7zNOu+OuObSzHof9eCOPWdIoUeVj+Yb6 +qu8QefXO43oy0xl7LRH60Sp35Erqw7Cmv2ZLlplQj0wq2FjvEOrVc+h2A1v5rsEKQB8 Qd0v3ZrGcouV6AXFYFnOsZK1Qkh67ZiX+TJtCqXYJCCJf9WPkmIRmgTW5LiOQL7Nn+MC O8R58ajOO0aqCoeLdnowVxIqx2ao/vB83LF9ksXt0ExJYRBjYAdm0p8U4uW3m7IQOt4n 9At5Q9gTsu/zkvMT8Py4oRqQC2kecJnEwh5VRT3PD0OXIfQ/XK9KdHF9rhwuDdO2demb Bocw== X-Gm-Message-State: AOAM531+DDrvLuYZHHLoc06DzybzoOtsAOhzAuA84wPGtkoyJErmeMhU axzipng0E4pX1WFuiETibX/AjuA9X0L8FUxkzD4= X-Google-Smtp-Source: ABdhPJzjBj+r0bS64g6afEZb5z72ewulsIffdDUuLJ3JpBq8Oms7Xzbb5kT85mS4AmJ5e/JJgaNMBQHU7WD7as31Z1k= X-Received: by 2002:a05:6830:15c3:: with SMTP id j3mr11459612otr.2.1595654729381; Fri, 24 Jul 2020 22:25:29 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Arkady Date: Sat, 25 Jul 2020 08:25:17 +0300 Message-ID: Subject: Re: Fwd: systemtap global var lead to high cpu To: Kun Cc: Serhei Makarov , systemtap Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: systemtap@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Systemtap mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jul 2020 05:25:34 -0000 On Sat, Jul 25, 2020 at 5:45 AM Kun wrote: > > Hi=EF=BC=8C > How to embed your module into systemtap=EF=BC=9F Try this https://sourceware.org/systemtap/tutorial/4_Tapsets.html#SECTION00053000000= 000000000 https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/ht= ml/systemtap_language_reference/ch03s06 > it prompts some error when i run dup_probe.stp=EF=BC=8C > semantic error=EF=BC=9Awhile resolving probe point: identifier =E2=80=98k= ernel=E2=80=99 at kprocess.stp:29:25 > Source: probe kprocess.creat =3D kernel.function(=E2=80=9Ccopy_proces= s=E2=80=9D).return Please send the whole probe > Thank you > > > > ------------------ Original ------------------ > From: Arkady > Date: Fri,Jul 24,2020 11:49 AM > To: Kun > Cc: Serhei Makarov , systemtap > Subject: Re: Fwd: systemtap global var lead to high cpu > > Hi Kun, > > I avoid locks by using lockless data structures. For example, I have > my own hashtables instead of associative arrays. > You can check this code for inspiration > https://github.com/larytet/lockfree_hashtable > > If the performance is paramount for your task you can switch to 100% > handwritten C code. > > Arkady, > > On Fri, Jul 24, 2020 at 5:01 AM Kun via Systemtap > wrote: > > > > My global var saved is string type, not number, so I can not use aggreg= ate. > > FChE will fixed this bug, > > https://sourceware.org/bugzilla/show_bug.cgi?id=3D26296 > > Hoping to fixed soon. > > > > > > > > > > ------------------ Original ------------------ > > From: Serhei Makarov > Date: Thu,Jul 23,2020 10:22 PM > > To: systemtap > Subject: Re: Fwd: systemtap global var lead to high cpu > > > > > > > > Forwarding your question to systemtap@sourceware.org in case other peop= le have suggestions. > > > > > > > > In general, the locks protect concurrent modifications from interfering= with each other when different processes trigger the same probe. > > > > > > Depending on what  you want to do with the iphdr value, you may be= able to reduce contention by using statistical aggregates > > (which do not require locking). For example, aaa[iphdr] <<< some_statis= tic; in a later probe iterate through aaa. A lot of the SystemTap example s= cripts use this type of structure, for example: https://sourceware.org/syst= emtap/examples/network/netfilter_summary.stp > > > > > > Otherwise, every tcp_ack() for every packet on your system will try to = grab the same lock. The resulting CPU load is unsurprising to me. > > > > > > Hope this information is helpful; if not, someone else may have a bette= r suggestion. > > > > > > All the best, > >       Serhei > > > > > > ---------- Forwarded message --------- > > From: Kun > Date: Wed, Jul 22, 2020 at 11:24 PM > > Subject: systemtap global var lead to high cpu > > To: smakarov > > > > > > > Hi, > >     I have a problem of system tap which > >  Using global var. > >     A simple demo as following, > > Global aaa > > Probe kernel.function(=E2=80=9Ctcp_ack=E2=80=9D){ > >     Iphdr =3D __get_skb_iphdr($skb) > >     If(iphdr =3D=3D 0){ > >         aaa=3Diphdr > >     } > > > > > > Then our env have a 10Gbps flow, and our cpu is nearly 100%. > > > > > > Analysising c code, I find this is because of a lock as following: > > Static void probe_6330() > > { > > If(sta_lock_probe(lock, ARRAY_SIZE(locks)) > >     Return; > > If(l->iphdr =3D=3D 0){ > >     Global(s_global_aaa)=3Dl->l_iphdr; > > } > > > > > > My question is that aaa should be protected in =E2=80=9Cif=E2=80=9D, > > Why is the lock directly at the function entry?