From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by sourceware.org (Postfix) with ESMTP id 9601D3857C5A for ; Thu, 23 Jul 2020 14:22:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 9601D3857C5A Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-318-d7U3vUi3NMyF_8_PUG7nFA-1; Thu, 23 Jul 2020 10:22:27 -0400 X-MC-Unique: d7U3vUi3NMyF_8_PUG7nFA-1 Received: by mail-wm1-f71.google.com with SMTP id r2so2412035wmh.1 for ; Thu, 23 Jul 2020 07:22:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=JAfhM29FVyLQDLvLTPc2IGXsrORHJG+OHx+lb/lu52c=; b=EZ+FVuihhx+GcoGS34dWdqdMTlnflRBkNG1uwvkQYIiDcWF4vI6qeRFZFloJAlCwDW 9nAFDsvbLH6jOajt6SdOOE5EJ1BDmNZ7DOyANRBsM0/ZGpKp0tLjllneSsmXQfYLlcRz BXqS1ki34hp8TDgZxEGPkRiBHPRfd3qME+XI8UXiwTkt9vYwzr6MOisJt7GqwDxauHMd coNCMmkYYhNo+baoFaMv9eiLgrXxrEAeeBBsQDpQERKRjPeGy0nFlMBIW0IvsJa+ksn1 GuMI5cepfbEiOsVXPffBOQ4W3yZv6Nq0kxgPOYeSDbdc+a5/39bTB0qLQBC8zLGOd/by b4Mw== X-Gm-Message-State: AOAM53051ro0gPafcVXg3uHaTfgIEnyuD9WNyu45z5bRijpt75yPSsvl rdqwUg+h+mc0D0Y1LKlGZZROm4HKW8jMTxdmR9a8WTp9VuAs4G4/JbjEp5htCJ6TQzEYkc/NYNx mqSEVNx2Z3f7maQWwhRQd863eNuY3CUwPojs= X-Received: by 2002:a05:600c:2511:: with SMTP id d17mr4489343wma.127.1595514145850; Thu, 23 Jul 2020 07:22:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwy7RcvvaNnE80u8Zyn9SIxGbHlo91CDC0ZZEB9LvgpWEKT6d3Odp1BNxwiq01RGCk6mYt6QnuVXs3YOcV2ZzY= X-Received: by 2002:a05:600c:2511:: with SMTP id d17mr4489325wma.127.1595514145576; Thu, 23 Jul 2020 07:22:25 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Serhei Makarov Date: Thu, 23 Jul 2020 10:22:14 -0400 Message-ID: Subject: Fwd: systemtap global var lead to high cpu To: systemtap@sourceware.org, mingkunone@qq.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: systemtap@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Systemtap mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jul 2020 14:22:34 -0000 Forwarding your question to systemtap@sourceware.org in case other people have suggestions. In general, the locks protect concurrent modifications from interfering with each other when different processes trigger the same probe. Depending on what you want to do with the iphdr value, you may be able to reduce contention by using statistical aggregates (which do not require locking). For example, aaa[iphdr] <<< some_statistic; in a later probe iterate through aaa. A lot of the SystemTap example scripts use this type of structure, for example: https://sourceware.org/systemtap/examples/network/netfilter_summary.stp Otherwise, every tcp_ack() for every packet on your system will try to grab the same lock. The resulting CPU load is unsurprising to me. Hope this information is helpful; if not, someone else may have a better suggestion. All the best, Serhei ---------- Forwarded message --------- From: Kun Date: Wed, Jul 22, 2020 at 11:24 PM Subject: systemtap global var lead to high cpu To: smakarov Hi, I have a problem of system tap which Using global var. A simple demo as following, Global aaa Probe kernel.function(=E2=80=9Ctcp_ack=E2=80=9D){ Iphdr =3D __get_skb_iphdr($skb) If(iphdr =3D=3D 0){ aaa=3Diphdr } Then our env have a 10Gbps flow, and our cpu is nearly 100%. Analysising c code, I find this is because of a lock as following: Static void probe_6330() { If(sta_lock_probe(lock, ARRAY_SIZE(locks)) Return; If(l->iphdr =3D=3D 0){ Global(s_global_aaa)=3Dl->l_iphdr; } My question is that aaa should be protected in =E2=80=9Cif=E2=80=9D, Why is the lock directly at the function entry? ------------------------------