From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 27143 invoked by alias); 22 Jul 2014 01:32:35 -0000 Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org Received: (qmail 27088 invoked by uid 89); 22 Jul 2014 01:32:31 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-5.1 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx4-phx2.redhat.com Received: from mx4-phx2.redhat.com (HELO mx4-phx2.redhat.com) (209.132.183.25) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 22 Jul 2014 01:32:28 +0000 Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s6M1WM8V021667; Mon, 21 Jul 2014 21:32:22 -0400 Date: Tue, 22 Jul 2014 01:32:00 -0000 From: Nathan Scott Reply-To: Nathan Scott To: David Smith Cc: Systemtap List , pcp Message-ID: <861139755.14608867.1405992742567.JavaMail.zimbra@redhat.com> In-Reply-To: <53C83CB9.3020808@redhat.com> References: <53C83CB9.3020808@redhat.com> Subject: Re: [pcp] systemtap/pcp integration MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_14608865_1230919135.1405992742565" X-SW-Source: 2014-q3/txt/msg00061.txt.bz2 ------=_Part_14608865_1230919135.1405992742565 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-length: 2458 Hi David, ----- Original Message ----- > [...] > Note that systemtap will create a file called 'mmv' in > /proc/systemtap/{MODULE_NAME}. I've just been using pcp's 'mmvdump' > utility to dump the contents of the /proc/systemtap/{MODULE_NAME}/mmv > file. Currently the pcp mmv pmda only looks in one place for mmv files, > but it might be possible to create a symbolic link to systemtap's mmv > file to make it happy. (OOC, what's {MODULE_NAME} in this context?) A symlink would sorta work but it feels like a bit of a workaround - the PMDA is written to be able to detect arrival/departure of new MMV files based on changes in a directory (and the location of that directory is parameterised via /etc/pcp.conf variables). I'd not recommend trying to find it within a stap script ... I imagine it will be cleaner if we go for separate user/kernel locations for MMV files. Attached patch (lightly tested) implements the PCP side of things with that in mind - with this patch (and making the kernel code manage the lifecycle of separate /proc/mmv/* entries), things should begin to work out-of-the-box (the MMV PMDA is already default-enabled in the default pmcd config file, so everything else is in place for you). It also occurs to me that there's not really anything systemtap-specific about the kernel MMV instrumentation you've done, or is there? A device driver author might want to instrument her code, for example, without needing a stap install/script...? It would be worth thinking about if the MMV bits in your script could become a more general kernel module for others to use too (in addition to stap, of course!). The extremely lightweight nature of the interface is such that it can be permanently enabled (that's certainly how the userspace MMV is used). Nice work anyway ... I'm looking forward to seeing the code in wider use. Oh, to use the attached patch with PCP tools - apply to a "dev" PCP tree (git.performancecopilot.org/pcp), build, cd src/pmdas/mmv/src, and ... $ pminfo -L -Kclear -Kadd,70,./pmda_mmv.so,mmv_init mmv ... will serve as a quick sanity test that the metrics are functioning. You can also use gdb on /usr/bin/pminfo to inspect behaviour of the PMDA. Once thats going, installing it properly ('make install') and restarting the pmcd service should get you a fully-functional PCP setup with your kernel code firing on all cylinders (pmchart, pmie, everything will work at that point). cheers. -- Nathan ------=_Part_14608865_1230919135.1405992742565 Content-Type: text/x-patch; name=proc-mmv.patch Content-Disposition: attachment; filename=proc-mmv.patch Content-Transfer-Encoding: base64 Content-length: 7508 ZGlmZiAtLWdpdCBhL3NyYy9wbWRhcy9tbXYvc3JjL21tdi5jIGIvc3JjL3Bt ZGFzL21tdi9zcmMvbW12LmMKaW5kZXggZjNlZjI2Zi4uOTJjZDRlMSAxMDA2 NDQKLS0tIGEvc3JjL3BtZGFzL21tdi9zcmMvbW12LmMKKysrIGIvc3JjL3Bt ZGFzL21tdi9zcmMvbW12LmMKQEAgLTQyLDE1ICs0MiwyMSBAQCBzdGF0aWMg aW50IGluY250OwogCiBzdGF0aWMgaW50IHJlbG9hZDsKIHN0YXRpYyBfX3Bt bnNUcmVlICogcG1uczsKLXN0YXRpYyBpbnQgc3RhdHNkaXJfY29kZTsJCS8q IGxhc3Qgc3RhdHNkaXIgc3RhdCBjb2RlICovCi1zdGF0aWMgdGltZV90IHN0 YXRzZGlyX3RzOwkJLyogbGFzdCBzdGF0c2RpciB0aW1lc3RhbXAgKi8KIHN0 YXRpYyBjaGFyICogcHJlZml4ID0gIm1tdiI7CiAKIHN0YXRpYyBjaGFyICog cGNwdG1wZGlyOwkJLyogcHJvYmFibHkgL3Zhci90bXAgKi8KIHN0YXRpYyBj aGFyICogcGNwdmFyZGlyOwkJLyogcHJvYmFibHkgL3Zhci9wY3AgKi8KIHN0 YXRpYyBjaGFyICogcGNwcG1kYXNkaXI7CQkvKiBwcm9iYWJseSAvdmFyL3Bj cC9wbWRhcyAqLwogc3RhdGljIGNoYXIgcG1uc2RpcltNQVhQQVRITEVOXTsJ LyogcGNwdmFyZGlyL3BtbnMgKi8KLXN0YXRpYyBjaGFyIHN0YXRzZGlyW01B WFBBVEhMRU5dOwkvKiBwY3B0bXBkaXIvPHByZWZpeD4gKi8KKwordHlwZWRl ZiBzdHJ1Y3QgeworICAgIGNoYXIJcGF0aFtNQVhQQVRITEVOXTsKKyAgICBp bnQJCXN0YXR1czsJCQkvKiBsYXN0IHN0YXRzZGlyIHN0YXQgcmV0dXJuIGNv ZGUgKi8KKyAgICB0aW1lX3QJdGltZXN0YW1wOwkJLyogbGFzdCBzdGF0c2Rp ciBtb2RpZnkgdGltZXN0YW1wICovCit9IHN0YXRzZGlyX3Q7CisKK3N0YXRp YyBzdGF0c2Rpcl90IHNkaXJsaXN0WzJdOwkJLyogdXNlcnNwYWNlIGFuZCBr ZXJuZWwgTU1WIGZpbGVzICovCisjZGVmaW5lIFNESVJDT1VOVCAoc2l6ZW9m KHNkaXJsaXN0KS9zaXplb2Yoc2Rpcmxpc3RbMF0pKQogCiB0eXBlZGVmIHN0 cnVjdCB7CiAgICAgY2hhciAqCW5hbWU7CQkJLyogc3RyZHVwIGNsaWVudCBu YW1lICovCkBAIC0zNjQsMTMgKzM3MCw0NCBAQCBjcmVhdGVfaW5kb20ocG1k YUV4dCAqcG1kYSwgc3RhdHNfdCAqcywgbW12X2Rpc2tfaW5kb21fdCAqaWQs IHBtSW5Eb20gaW5kb20pCiAgICAgcmV0dXJuIDA7CiB9CiAKK3N0YXRpYyBp bnQKK2NyZWF0ZV9jbGllbnRfc3RhdGRpcihzdGF0c2Rpcl90ICpzdGF0c2Rp cikKK3sKKyAgICBzdHJ1Y3QgZGlyZW50ICoqZmlsZXM7CisgICAgaW50IG5l ZWRfcmVsb2FkID0gMDsKKyAgICBpbnQgaSwgbnVtOworCisgICAgbnVtID0g c2NhbmRpcihzdGF0c2Rpci0+cGF0aCwgJmZpbGVzLCBOVUxMLCBOVUxMKTsK KyAgICBmb3IgKGkgPSAwOyBpIDwgbnVtOyBpKyspIHsKKwlzdHJ1Y3Qgc3Rh dCBzdGF0YnVmOworCWNoYXIgcGF0aFtNQVhQQVRITEVOXTsKKwljaGFyICpj bGllbnQ7CisKKwlpZiAoZmlsZXNbaV0tPmRfbmFtZVswXSA9PSAnLicpCisJ ICAgIGNvbnRpbnVlOworCisJY2xpZW50ID0gZmlsZXNbaV0tPmRfbmFtZTsK KwlzcHJpbnRmKHBhdGgsICIlcyVjJXMiLCBzdGF0c2Rpci0+cGF0aCwgX19w bVBhdGhTZXBhcmF0b3IoKSwgY2xpZW50KTsKKworCWlmIChzdGF0KHBhdGgs ICZzdGF0YnVmKSA+PSAwICYmIFNfSVNSRUcoc3RhdGJ1Zi5zdF9tb2RlKSkK KwkgICAgaWYgKGNyZWF0ZV9jbGllbnRfc3RhdChjbGllbnQsIHBhdGgsIHN0 YXRidWYuc3Rfc2l6ZSkgPT0gLUVBR0FJTikKKwkJbmVlZF9yZWxvYWQgPSAx OworICAgIH0KKworICAgIGZvciAoaSA9IDA7IGkgPCBudW07IGkrKykKKwlm cmVlKGZpbGVzW2ldKTsKKyAgICBpZiAobnVtID4gMCkKKwlmcmVlKGZpbGVz KTsKKworICAgIHJldHVybiBuZWVkX3JlbG9hZDsKK30KKwogc3RhdGljIHZv aWQKIG1hcF9zdGF0cyhwbWRhRXh0ICpwbWRhKQogewotICAgIHN0cnVjdCBk aXJlbnQgKipmaWxlczsKICAgICBjaGFyIG5hbWVbNjRdOwogICAgIGludCBu ZWVkX3JlbG9hZCA9IDA7Ci0gICAgaW50IGksIGosIGssIHN0cywgbnVtOwor ICAgIGludCBpLCBqLCBrLCBzdHM7CiAKICAgICBpZiAocG1ucykKIAlfX3Bt RnJlZVBNTlMocG1ucyk7CkBAIC00MDcsMjcgKzQ0NCw4IEBAIG1hcF9zdGF0 cyhwbWRhRXh0ICpwbWRhKQogCXNjbnQgPSAwOwogICAgIH0KIAotICAgIG51 bSA9IHNjYW5kaXIoc3RhdHNkaXIsICZmaWxlcywgTlVMTCwgTlVMTCk7Ci0g ICAgZm9yIChpID0gMDsgaSA8IG51bTsgaSsrKSB7Ci0Jc3RydWN0IHN0YXQg c3RhdGJ1ZjsKLQljaGFyIHBhdGhbTUFYUEFUSExFTl07Ci0JY2hhciAqY2xp ZW50OwotCi0JaWYgKGZpbGVzW2ldLT5kX25hbWVbMF0gPT0gJy4nKQotCSAg ICBjb250aW51ZTsKLQotCWNsaWVudCA9IGZpbGVzW2ldLT5kX25hbWU7Ci0J c3ByaW50ZihwYXRoLCAiJXMlYyVzIiwgc3RhdHNkaXIsIF9fcG1QYXRoU2Vw YXJhdG9yKCksIGNsaWVudCk7Ci0KLQlpZiAoc3RhdChwYXRoLCAmc3RhdGJ1 ZikgPj0gMCAmJiBTX0lTUkVHKHN0YXRidWYuc3RfbW9kZSkpCi0JICAgIGlm IChjcmVhdGVfY2xpZW50X3N0YXQoY2xpZW50LCBwYXRoLCBzdGF0YnVmLnN0 X3NpemUpID09IC1FQUdBSU4pCi0JCW5lZWRfcmVsb2FkID0gMTsKLSAgICB9 Ci0KLSAgICBmb3IgKGkgPSAwOyBpIDwgbnVtOyBpKyspCi0JZnJlZShmaWxl c1tpXSk7Ci0gICAgaWYgKG51bSA+IDApCi0JZnJlZShmaWxlcyk7CisgICAg Zm9yIChpID0gMDsgaSA8IFNESVJDT1VOVDsgaSsrKQorCW5lZWRfcmVsb2Fk ICs9IGNyZWF0ZV9jbGllbnRfc3RhdGRpcigmc2Rpcmxpc3RbaV0pOwogCiAg ICAgZm9yIChpID0gMDsgc2xpc3QgJiYgaSA8IHNjbnQ7IGkrKykgewogCXN0 YXRzX3QgKiBzID0gc2xpc3QgKyBpOwpAQCAtNjA5LDExICs2MjcsMzggQEAg bW12X2ZldGNoQ2FsbEJhY2socG1kYU1ldHJpYyAqbWRlc2MsIHVuc2lnbmVk IGludCBpbnN0LCBwbUF0b21WYWx1ZSAqYXRvbSkKICAgICByZXR1cm4gMDsK IH0KIAorc3RhdGljIGludAorY2hlY2tfdGltZV9zdGF0c2RpcihzdGF0c2Rp cl90ICpzdGF0c2RpcikKK3sKKyAgICBzdHJ1Y3Qgc3RhdCBzOworICAgIGlu dCBuZWVkX3JlbG9hZCA9IDA7CisKKyAgICAvKgorICAgICAqIGNoZWNrIGlm IHRoZSBkaXJlY3RvcnkgaGFzIGJlZW4gbW9kaWZpZWQsIHJlbG9hZCBpZiBz bzsKKyAgICAgKiBub3RlIG1vZGlmaWNhdGlvbiBtYXkgaW52b2x2ZSByZW1v dmFsIG9yIG5ld2x5IGFwcGVhcmVkLAorICAgICAqIGEgY2hhbmdlIGluIHBl cm1pc3Npb25zIGZyb20gYWNjZXNzaWJsZSB0byBub3QgKG9yIHZpY2UtCisg ICAgICogdmVyc2EpLCBhbmQgc28gb24uCisgICAgICovCisgICAgaWYgKHN0 YXQoc3RhdHNkaXItPnBhdGgsICZzKSA+PSAwKSB7CisJaWYgKHMuc3RfbXRp bWUgIT0gc3RhdHNkaXItPnRpbWVzdGFtcCkgeworCSAgICBuZWVkX3JlbG9h ZCsrOworCSAgICBzdGF0c2Rpci0+c3RhdHVzID0gMDsKKwkgICAgc3RhdHNk aXItPnRpbWVzdGFtcCA9IHMuc3RfbXRpbWU7CisJfQorICAgIH0gZWxzZSB7 CisJaWYgKHN0YXRzZGlyLT5zdGF0dXMgIT0gb3NlcnJvcigpKSB7CisJICAg IHN0YXRzZGlyLT5zdGF0dXMgPSBvc2Vycm9yKCk7CisJICAgIHN0YXRzZGly LT50aW1lc3RhbXAgPSAwOworCSAgICBuZWVkX3JlbG9hZCsrOworCX0KKyAg ICB9CisgICAgcmV0dXJuIG5lZWRfcmVsb2FkOworfQorCiBzdGF0aWMgdm9p ZAogbW12X3JlbG9hZF9tYXliZShwbWRhRXh0ICpwbWRhKQogewogICAgIGlu dCBpOwotICAgIHN0cnVjdCBzdGF0IHM7CiAgICAgaW50IG5lZWRfcmVsb2Fk ID0gcmVsb2FkOwogCiAgICAgLyogY2hlY2sgaWYgZ2VuZXJhdGlvbiBudW1i ZXJzIGNoYW5nZWQgb3IgbW9uaXRvcmVkIHByb2Nlc3MgZXhpdGVkICovCkBA IC02MjksMjYgKzY3NCw4IEBAIG1tdl9yZWxvYWRfbWF5YmUocG1kYUV4dCAq cG1kYSkKIAl9CiAgICAgfQogCi0gICAgLyoKLSAgICAgKiBjaGVjayBpZiB0 aGUgZGlyZWN0b3J5IGhhcyBiZWVuIG1vZGlmaWVkLCByZWxvYWQgaWYgc287 Ci0gICAgICogbm90ZSBtb2RpZmljYXRpb24gbWF5IGludm9sdmUgcmVtb3Zh bCBvciBuZXdseSBhcHBlYXJlZCwKLSAgICAgKiBhIGNoYW5nZSBpbiBwZXJt aXNzaW9ucyBmcm9tIGFjY2Vzc2libGUgdG8gbm90IChvciB2aWNlLQotICAg ICAqIHZlcnNhKSwgYW5kIHNvIG9uLgotICAgICAqLwotICAgIGlmIChzdGF0 KHN0YXRzZGlyLCAmcykgPj0gMCkgewotCWlmIChzLnN0X210aW1lICE9IHN0 YXRzZGlyX3RzKSB7Ci0JICAgIG5lZWRfcmVsb2FkKys7Ci0JICAgIHN0YXRz ZGlyX2NvZGUgPSAwOwotCSAgICBzdGF0c2Rpcl90cyA9IHMuc3RfbXRpbWU7 Ci0JfQotICAgIH0gZWxzZSB7Ci0JaSA9IG9zZXJyb3IoKTsKLQlpZiAoc3Rh dHNkaXJfY29kZSAhPSBpKSB7Ci0JICAgIHN0YXRzZGlyX2NvZGUgPSBpOwot CSAgICBzdGF0c2Rpcl90cyA9IDA7Ci0JICAgIG5lZWRfcmVsb2FkKys7Ci0J fQotICAgIH0KKyAgICBmb3IgKGkgPSAwOyBpIDwgU0RJUkNPVU5UOyBpKysp CisJbmVlZF9yZWxvYWQgKz0gY2hlY2tfdGltZV9zdGF0c2Rpcigmc2Rpcmxp c3RbaV0pOwogCiAgICAgaWYgKG5lZWRfcmVsb2FkKSB7CiAJaWYgKHBtRGVi dWcgJiBEQkdfVFJBQ0VfQVBQTDApCkBAIC04MDgsNiArODM1LDcgQEAgbW12 X2luaXQocG1kYUludGVyZmFjZSAqZHApCiB7CiAgICAgaW50CW07CiAgICAg aW50IHNlcCA9IF9fcG1QYXRoU2VwYXJhdG9yKCk7CisgICAgY2hhciAqcHJv Y2RpcjsKIAogICAgIGlmIChpc0RTTykgewogCXBtZGFEU08oZHAsIFBNREFf SU5URVJGQUNFXzQsICJtbXYiLCBOVUxMKTsKQEAgLTgxOCwxMiArODQ2LDE1 IEBAIG1tdl9pbml0KHBtZGFJbnRlcmZhY2UgKmRwKQogICAgIHBjcHRtcGRp ciA9IHBtR2V0Q29uZmlnKCJQQ1BfVE1QX0RJUiIpOwogICAgIHBjcHZhcmRp ciA9IHBtR2V0Q29uZmlnKCJQQ1BfVkFSX0RJUiIpOwogICAgIHBjcHBtZGFz ZGlyID0gcG1HZXRDb25maWcoIlBDUF9QTURBU19ESVIiKTsKKyAgICBpZiAo KHByb2NkaXIgPSBnZXRlbnYoIlBST0NfU1RBVFNQQVRIIikpID09IE5VTEwp CisJcHJvY2RpciA9ICIvcHJvYyI7CiAKLSAgICBzbnByaW50ZihzdGF0c2Rp ciwgc2l6ZW9mKHN0YXRzZGlyKSwgIiVzJWMlcyIsIHBjcHRtcGRpciwgc2Vw LCBwcmVmaXgpOwogICAgIHNucHJpbnRmKHBtbnNkaXIsIHNpemVvZihwbW5z ZGlyKSwgIiVzJWMiICJwbW5zIiwgcGNwdmFyZGlyLCBzZXApOwotICAgIHN0 YXRzZGlyW3NpemVvZihzdGF0c2RpciktMV0gPSAnXDAnOwogICAgIHBtbnNk aXJbc2l6ZW9mKHBtbnNkaXIpLTFdID0gJ1wwJzsKIAorICAgIHNucHJpbnRm KHNkaXJsaXN0WzBdLnBhdGgsIE1BWFBBVEhMRU4tMSwgIiVzJWMlcyIsIHBj cHRtcGRpciwgc2VwLCBwcmVmaXgpOworICAgIHNucHJpbnRmKHNkaXJsaXN0 WzFdLnBhdGgsIE1BWFBBVEhMRU4tMSwgIiVzJWMlcyIsIHByb2NkaXIsIHNl cCwgcHJlZml4KTsKKwogICAgIC8qIEluaXRpYWxpemUgaW50ZXJuYWwgZGlz cGF0Y2ggdGFibGUgKi8KICAgICBpZiAoZHAtPnN0YXR1cyA9PSAwKSB7CiAJ LyoK ------=_Part_14608865_1230919135.1405992742565--