From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28205 invoked by alias); 17 Oct 2006 08:12:54 -0000 Received: (qmail 28197 invoked by uid 22791); 17 Oct 2006 08:12:54 -0000 X-Spam-Status: No, hits=-1.7 required=5.0 tests=AWL,BAYES_00,SPF_PASS X-Spam-Check-By: sourceware.org Received: from ausmtp06.au.ibm.com (HELO ausmtp06.au.ibm.com) (202.81.18.155) by sourceware.org (qpsmtpd/0.31) with ESMTP; Tue, 17 Oct 2006 08:12:51 +0000 Received: from sd0208e0.au.ibm.com (d23rh904.au.ibm.com [202.81.18.202]) by ausmtp06.au.ibm.com (8.13.8/8.13.6) with ESMTP id k9H8DeKQ8163484 for ; Tue, 17 Oct 2006 18:13:40 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.250.244]) by sd0208e0.au.ibm.com (8.13.6/8.13.6/NCO v8.1.1) with ESMTP id k9H89Akv163718 for ; Tue, 17 Oct 2006 18:16:01 +1000 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id k9H839Au017782 for ; Tue, 17 Oct 2006 18:03:12 +1000 Received: from [127.0.0.1] ([9.181.133.111]) by d23av03.au.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id k9H8346J017613; Tue, 17 Oct 2006 18:03:06 +1000 Message-ID: <45348E36.8040704@cn.ltcfwd.linux.ibm.com> Date: Tue, 17 Oct 2006 08:12:00 -0000 From: "Gui,Jian" User-Agent: Thunderbird 1.5.0.7 (Windows/20060909) MIME-Version: 1.0 To: systemtap@sources.redhat.com CC: Vara Prasad , Li Guanglei Subject: a framework of generating whitelist for safe-mode probes Content-Type: multipart/mixed; boundary="------------040909040001000208060406" X-Virus-Checked: Checked by ClamAV on sourceware.org X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2006-q4/txt/msg00143.txt.bz2 This is a multi-part message in MIME format. --------------040909040001000208060406 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-length: 2410 Hi, Here is an implementation of generating whitelist for safe-mode probes based on the discussions in former thread (http://sourceware.org/ml/systemtap/2006-q3/msg00574.html). Its main idea is: 1) Fetch a group of probe points from probes.pending, probe them and run some workload(e.g. runltp -t 60s) meanwhile. 2) If the probe test ends without crash, those actually triggered probe points are moved into probes.passed and those untriggered are into probes.untriggered; If the probe test crashes the system, it will be resumed automatically after system reboot. Those probe points which have been triggered are also moved into probes.passed, but those untriggered ones are moved into probes.failed. 3) Repeat the above until probes.pending becomes empty, then: Normally, probes.pending is reinitialized from probes.failed (or probes.untriggered if probes.failed is empty) and start the next iteration; But if max iteration limit (e.g. 5) is reached, or probes.pending, probes.failed and probes.untriggered are all empty, stop the whole test. To be able to resume after a crash, this test will register itself as the last system service at the beginning and unregister itself at the end. I also use a script in a remote server to restart the test machine automatically if it is crashed. Usage: runtest whitelist.exp * Please remove "/stp_genwhitelist_running" first if you want to restart the test from the scratch. Rough result on my 2.6.18/ppc64: root:/home/root/testsuite/systemtap.stress>wc -l probes.* 13209 probes.all * all input probe points 2 probes.failed * causing crash and unrecorded in probe.out 1146 probes.passed * whitelist,for those recorded at least once 12061 probes.untriggered * probed without crash but unrecorded Some problems: 1) How to choose workload to trigger as much as many probe points? Now only 9% of all are triggered when using "runltp -t 60s". 2) How to set the proper sizes of each group for different iteration levels? A smaller group size will help find those probe points really crashing the system, but also causes longer running time. After all, there are 13209 probe points for kernel.function("*") and 31132 for kernel.inline("*"). btw: do we need to handle module("*").function("*") ? Any comments? Thanks. -Guijian --------------040909040001000208060406 Content-Type: application/octet-stream; name="genwhitelist.1017.tgz" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="genwhitelist.1017.tgz" Content-length: 6942 H4sIAAn5NEUAA+08a1cbuZL5Ov4VlcaZtRP84pGcISE7DHgy7ALhALl3Z4Hx abdluzf9uv2AMFz/962S1N1SP0hyw8y9e7Z1OGC3pKpSValeUrNg3u3Sjplj R/HgyR/ThthebW/zv9iKf/nn0XB7tD16tbW5tfFkONrYeLn1BLb/IHq0lkSx GQI8CX0/fmjc5/r/j7aFKn/X9+zYD/vR8lFxcPlvbdXJf2Pz1baQ//bmaDR8 ifLf2hxtPIHho1JR0/6fy3/t6WBqe4OpGS1ba601OBYqALdLFi9ZCPgLQub6 MYOYRTFEd1HMXLAjmPm3HpjeDHBWyKbIHrDxZw5xmLB1CBPPs70F+DQIfjne h4iFNyzkWPb94C60F8sYOvtd2BgOX8LhT8f4OAz62I0/F0tEMbcdRqgCM4zB n0vksRmsc8Q0ImQMIn8e35oh6wP86idgmR4naYYqHdrTBElHwnDCANfl+jN7 fkcPEm8m1xez0I0IPn15d/IB3jGPhaaDUE6TqWNbcGRbzIsYdN6dHnVfA7M5 b3A1kY3r21gHhNwxY7jzkxD8IManXcR4hxAcM86H9sXqQ4YPZzC9g3eJDf9h I4PeLBL7f/DDj5bXt6du3/Ldtzj4Ynx+cf7r+cX4eHKydzzeNSLfZRPPdJmh 9h0eyB57pj3fOzg4S3sC7Dk8Hr//cLG7MWy1jvf+6/Tw5N357hZ9PBv/9P79 xfnuCHsCFNskZMg7Fu0OW0K2yoMWGgyUyyWM4Lo181sAkcNYADgXgCZD7xba Ehd+0MmBngWbOBAV5RLa/w499jcYIiAg7nv4R8BQ8NEjZi19MLSOtvptHQpk tvXvBkJhTsTK8Nsd9ikIQYMGL2DUfTS86Vo1DL0Fg3YqA758hQHpDB0W9JyY z5HCkrO0eQCk+WTSYnXrOriDshHWEvcRDsHnvZB3Qc/V5ESqBr2ePdOeHh5A z4domcR862O/66IW93oSYYagyBLJ4cJqMh7XiFwRGLUpbpqP8tvcbmV/8NfM 91irJUR1bH5KrVGKCCdaSzbrw3nsBwF+MFrsExqAYeufbXz/BZrm/7NPfZTY 4+EgJ/+y1v+PRq9GW5n/Hw5x3Gh72Pj/P6etSWeHxiCTPszRnUXmnPWC0J/i DrK93PWi/4JlHAc7g0GE3s5i3PP64WLgOoNs1IB8eu9vmwM3WgyH26+2+svY dRrf/zW+H/fgxPYQgmc6aKxajm/OJo49BSOTFH2lvWq0WhGLAX3D5OzDyQn6 lMnR+C/jI9gWHecXe6eTs/H5h6ML9GYk1L6fxHLW6dn7n8bnk72jI9kX9U3H MdS+0/HJAQLN+gPmzdBia2P2P5ydjU8usjFWEobMKyA53Ts/Hx/kcMwoYjMN zM97h0fKiLmJeqCP+HBycXb47t34TBmWeCjuxYKFNFYu+XTybnzy118OL8ZH h+cXKWPAGERxMFHt3kTGqnJmpmmTyArtIIZ77m0Wjj9FQZDjFFEEYYZnkeym Rn2X/Pk88axO9xrevHkDI96/UiZN2QK3VD7P8RcdI0ML3JtSHMVHG93S/Nh2 Wdh3o8422s+uAgg3Ljk76BB+vm2RotzLCgioU/OO8ezZDJ49i648Yx1+tHxk YIeTTzOvu+tAf8uYUe71dAulKFC9aom8wkRqQlQ622Ppdrtd+o5IK3AIRjz3 lwuWCWPisBvmXMPuLgxXEucanKBf5+zBLXTH4nXIQ50cGloG3+XPUIBmbC2z IIxSENzOk8QLUQRIN+rLtbKgNTiTz8G9i5gzBzPigBwzz30kEL7xzYC2AigA aDRuYBdtDk71YcBia2BjUtWfDUKr7/gWKpESKgl0EwlUYXjGEAdZFtm/Y7yY 71SdLwLzISKxTYdG5tuYwFBaFgFzg/iOaPaQhwv7RokZibyJmDPBOQoRa3CO O0JuZV00xJoRrTBa+reY5kneIyYpoFYGWpvXKlHLjRyGkjZ5ISQQtSRIYuEC IsDUFIWNplbYrZQ+9olZELrQm2d8Sa1UW7dItcOFLWpXmJW2ZoxksOmQYIM7 VQxF1Dmq2E9wH/5jqPjijXOu2Sa5uWgJRZvFmY2R7IpHydn+ODbvcJemO8R0 0BzM7nCX+JiuA+WBYDnM5Ik5KobHLBZFZngnDJwZTs0Fm1i+4zALU429o7Px 3sGvk/2zvfNfNNrOmOXf0OYRW43vDpejxg0XYbCdGipJpDCsqRqRMiMWDyeR tQxQ/dUuqDIDqu63BBEV4OyIK5zQqxlpaLs8ykgz2PtRuoEEwBP/FlmF49A+ 6Y6OAFfvRSn4a+MzezYdV7Fv95fM+pgVXG4ZigVpJzvisU9UVKHwjGINsXeV mRSnlQnlGz0bJd1WKTRQwNiehXqCWsQzRlm3EXucMHBDGsswzESibnOaFBNi hRX7XFEZDW6KckYGpELeRg64xi+85amwviaVr6qy8vwPFfSTwkzHdu1452Hc 1NS8E62iut10NPtC0QoMTBWnQP2b3QrydcT1uiRMBbFgWFyxEGjI7JIvyBQk 020RWhGFnu/1dK1JW72VLQ0tG0gZyn3RxBlzWFxcYjU5ummtGSllUkEOKlyB IrHpd79kg+t6UC8ixcJ/s5yU2PYPFFaVT/oaiSnzv0RsDw2vkJ1K3SMKsEIq qSvAbDFkub2IuFXWq0jFqbqp4Fi0b2tAXpDs543pJDwMrvBhZHI1Gwu6OaX2 qN60vPpqz0pIqz0q6uStzg58jJF+kpfvBCdWkMXx6GozTS7qPkaUiYM5MtUv MX6Rc7JChZ5g1oV3pehvibCg51UuoBhCvq0HE6PZRDAvLkVVswzsxei6Ah5m DkEOxL3hD0o7TYuxcOmk7Bi5SSRKVGVIphz4YFpxgh4+7+NdMz9Tg+xZurHc gLbujAft+TSIEoviwXniOHdGK5OUiOlIL+PEQ/47NQElkhqELDBx2wgd/hTn 5JSCy5P3Iq7kCSLs+94CZZs4crMRADMM7RsCjGERa1Fq5mLQCZTUEO/8EANX mSNQRSjP6WTu1kqX+xnQuJOz+I8XzKuRrKsYeEwkUjaOnArKvK78pUkJtGsL FBoUtXLDVSgnIssZv6WcXV3/TetKj1NjfLj+uzHc3BoW678vNzeb+u+f0dCC sDm5TKHSSYAqn9aeLDZLMPmEhOJ02wPteKCp5X5NLZd8Z2rxcKfS10J2DSOR JH84H0/O9o/e7+8dpY/Ox2d/Odwfi2O5YiFAlCxlXUsWLI3iqT6XguzkcqCy K3fSoe+IHN6kxEUmLR0bZYBfxbfNLkLIapISCh9y/sMPvxM9GTE4smXNoM28 m87pXw+6LQTCXYumOvB9WmkloElaZ53Y3iS0RIHMUKG8hnowpKZFXwf3K71m W+HK02puodabVQotqhvCPYZTXsl0hys49Ein1dKhcCE/i5wKXWHVzDxGClmc hJ5SaCN2cFcc+DY6egraMNSSaPgIWbN4eknlibRHzSsIBPrc0KyaLdd1f0l7 iWen3iLGcJxPqExRihShtiBTwGjnD9fBSCGUpj5IyUpZueQ0Ch43phwpejLG kFDpzgMYRxXR7ORdGEzSdYWmRyfrCuFD2Ny47vf7+VE875z4H9Vh1wVGSjW/ xGjBxVW3S4cC6txspnakAO2Mbvw2ZR4us4e0SQA9H7KTGBkcF1ISoVIZtSJf 31HAepilEPeMRyCdBx38DD0l8C1C6uP2VobwMCP9mnjSXaDCG2WiOL3oP4z8 IIBvVhIgpb0zNjcTR9msERp8uOR0LDCOBGNHGbZj5JX02Jxeg2HPdp7N1CEG KF9a+S6Dtvp8pdvUo8OT/wSDgw4tXqK/LBB4jU/J0LVVM2woi8kyzntSXeJB uiJSNu77KI0xp/ihnQ7J9U1Z9q0FPUcdYzybGcDh5/dC7i9tb+6TMCJUD955 Dd9/D23+sbyTUyaoYNT9J/uHmoD0Sn/RntaFrnWLrhuvbbqcDWRmHpjDmVJ5 JKFbOpVNlcMF26oPNyoMYsrIh1E/zNpCFnR/L0v0k7RunuVFqxLPs0Tgy1yb dvRQ0aEUzvQOtTTzmaq2RFuuatfn41pKU+8PK3Aq88oIaXy7wMyiAHPlKpD0 9i3UlBMry0RfAqeqvrV6PPbI5Hzqow/XyyE8os4cC1A2rlcM/4To5puX9wjR kDABkTjWnGkRgaZJlUoi/KAyRZGssqFqNeRLtPGLkFVUt2srl58FVldzXVVo 6Ocl8Lno7WuUQCpAaibLh0lf54JkmVXMFHW6irjxxeg6MzXQzsaj5YeSKRNs VcbU+qfqUC4HMvz85GqWaEf0dSnO3tFRzvxBEoUDnk/xbDCiOxq9YIOCUBFV wkcWeszp010PynU7V8bzK6N7vzKoYOkGJMwcnAjKfivMMdKBSs2UTsbzeZRq 9myONRp019uD7uB1NPixT6jaA/w1MNSZ+pK/QQtyno8e5HmGLzMW94rmE9cm IYswElTCulIepWYS5Jvg738vnhgrkOQYVc2yWCFXAVLjuR1SYV/osMQpnhll 4RkaHS9Gr4SCrxEgfkZfDehHfV5P7otsVuUklFv9tOwOFNTnZW0BqE0orh/m K7+TVHL5BY5Jl3j4bzMMij8y8lIJ7UFFdxVwbUMTyEoW2iUQXv5dUh3bcfxb Xtxe4u9oHaYJf80Bg1W6RrNEZ+uIc3OLDtBvealawOAo06ssOJgAyK5qP6yQ Q074jEVFL1zjhtWJtpfrcafLq3jKyUyNkkm1H2YPEBjpbWr2U1IqPG/aVXK9 Uo4iFTSeK6w36Co4h19yvgoxI62jcLSW+qnP0Vrlo+QALUIXxM0yU5Dl3Iol yNJUOntzZ1KxHrCxAoMjr4jRFGl11V2AtlYl5KlUiTYOv1bJKR3r3acXKVAn +HpSVvKCiR+bTn76l9ugbLhM/KNbm6OT5jXnML8bQ6Bm9o0d+SGMhsph5v1G qVfr3ix0b+vdW+XZWv92ebran1UOCiGQft3D9m5Mx54Vz1F35GKLx7caPSVN W+W8lVIQ3BVngBm7B20JQrKXx4HqhDcFG1aCONLwFVJOGpEpRfEYqOQclUqy 5jOVMkbVc6qIqM/12rLWU1e+zax5W61mo/0e1QRWhfo1eesilQ+EVGq526ie zH/zKMWM08MEbRhuWha6dhTxM0L+vqpezZCFhLbKJtU2a3ciSrxcVQDTroZq xNTBrZ1QXKS42aGt7+G5ldTrvWhEPuorq4cJX68BtUcBmMCo5bl+rmF5TTGr r96rVjp/qlhpJw4GdLQRByWZMCuJRcEqnVlOgc/TcxDTg6OLU6BL+ZBVl7NF tfOKbwwvh1G1Cq/JN+f4kTRM7bgOmRj2++//rWEy53Qk9ZIOUUuJg1J0qrr0 LBj1Xd3WV+A8LXNH2wIZmvLBdDlc/1LTo+SO9TpW3mVauuHxjCM/0x48DYwq TUrp12v390BvvglE90fv9w4od5j8/OFk/+Lw/QkYxgrMcBGBlhPwiiMBo2uo qNGOQ1XxSXANT3dxCpUbn17qPTg/vw0uTrmYfO2OI38NBYG8LsRuFFekZzP3 ldHIzQraRKvwSeyGLHdg3no8yJB8C3g1svVdj14toLoRahX0QhThb6dmFF1B 53K0sbl13d25/O0qvH5+hQukh8PeD9cvuoQ1/RzdRdlnDNqcK3Cj/lV45VXG egi7LbBPEGtntC7Cxi5c6e7ZuIrVcRvpOP3xZvXjrfSxoXBTrnuS3lRa1S4e tnfy40/UjXQ1Oqx0wxfWiJGfcTUcbhoPIE6fZvh/KDB/+/G4ThwHI9cv/kaH dvHntVIN+o6oMCLmmkgrmuow9MMdA5dOZzsaGMt3A1tcsTFoYrqUiqHS1ENH jiGxKIyi2ZRa3JdpzWZif9eoOEXkIb7IUkybn/X8s+9X/Ks37f4PXUg4Hj86 jofv/ww3Ry/T//8x3NjexuejzdHmVnP/589ov7CQX83BcMama4EuphnybYM5 LMSNS7J7FS+Huv4svYvZyl7SoVLIzI6shEfSVI6gY2h+QYeq4a3OP/ruaLff ah1iUOTSi2T2jJlI9U4LRl34mdHWN2ER+mgb0nc3gOfXkbjcot8vXU8vPS6Z S8clLWm/xX3GWz/8SFFdh/UX4hZMHMgorgsuMz1eAEFqYKMLh+KykXKJktF7 X5hVL8n68dL/Og6hyoO4p+ncQXadmyPWqKXjGrqASOlr7Gd0i/s6dLIjQKlX wjkQmqfOUAZIn1emVJxLyItfnPnrVNq6tR1HXL2NElfSaCax76IiWJx+EXfK t+HEy/Z9uOCEaYtBRqFgluYNEyRMGfPyxXOaMW32axcsqm1ixS0ZH2WzfTrs rmaXuChBEtrswhkLmCneEDSndLUTgdhO8cbxlKEDY/LdmXX+HxV2OMoTukTh 0JXX8os26vX9maZo8tUKDqGTv6Sj3eSveA+DY+9yOefvNXIg+ptAUqQ/Jfz/ rVS81wIdz7+F7a6gktdD+N22wj7I9S+jIjs91KkVsnJS/kSxH+TE5W9eIs8v fFIeniigRIQSSY0x8/2AdIkLXaRs2a1aO5bvXLZq3rkEKUn+Aq0n0jDt5m8K gQ9r4TLJaggto2KwmV2C8+jCm/jnGOL/0khqM7Zz8lxkne2xlq7/2SuV8mQP cXyIzAXboRp05VU16nhOW/DOTzA08eJKbMXXVlHnHP5elriUzHn9wNvLojK/ zg20oC+RL1jZKG4MkAQEXqyWZ8Xldww4lerro1H+vqhmHqRvIN3QmCMKBrSS 4vmE8dzoNvFY05rWtKY1rWlNa1rTmta0pjWtaU1rWtOa1rSmNa1pTWta05rW tKY1rWlNa1rTmta0pjWtaU1rWtOa9g3tfwFysv8rAHgAAA== --------------040909040001000208060406--