From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 9919 invoked by alias); 27 Oct 2008 17:02:57 -0000 Received: (qmail 9890 invoked by uid 22791); 27 Oct 2008 17:02:54 -0000 X-Spam-Status: No, hits=-1.2 required=5.0 tests=AWL,BAYES_00,UNPARSEABLE_RELAY X-Spam-Check-By: sourceware.org Received: from rcsinet12.oracle.com (HELO rgminet12.oracle.com) (148.87.113.124) by sourceware.org (qpsmtpd/0.31) with ESMTP; Mon, 27 Oct 2008 17:02:12 +0000 Received: from rgminet15.oracle.com (mail-router.oracle.com [148.87.113.117]) by rgminet12.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id m9RH1ldn032050 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 27 Oct 2008 17:01:48 GMT Received: from acsmt706.oracle.com (acsmt706.oracle.com [141.146.40.84]) by rgminet15.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id m9RH1k4R019123; Mon, 27 Oct 2008 17:01:47 GMT Received: from localhost (/66.92.236.138) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 27 Oct 2008 17:01:43 +0000 Date: Mon, 27 Oct 2008 17:02:00 -0000 From: Kris Van Hees To: Mark Wielaard Cc: wenji.huang@oracle.com, SystemTAP , "ZANNONI,ELENA" , "VAN HEES,KRIS" Subject: Re: Labrat reports for 2.6.27-RC snapshot tests Message-ID: <20081027170140.GO20331@oracle.com> References: <490172AA.2000605@oracle.com> <1224848066.3418.20.camel@dijkstra.wildebeest.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1224848066.3418.20.camel@dijkstra.wildebeest.org> User-Agent: Mutt/1.5.16 (2007-06-09) X-Source-IP: acsmt706.oracle.com [141.146.40.84] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090207.4905F3FA.0388:SCFSTAT928724,ss=1,fgs=0 X-IsSubscribed: yes Mailing-List: contact systemtap-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: systemtap-owner@sourceware.org X-SW-Source: 2008-q4/txt/msg00213.txt.bz2 On Fri, Oct 24, 2008 at 01:34:26PM +0200, Mark Wielaard wrote: << snip>> > There is some noise in the output because the script seems to not > understand some of the error messages when things fail. e.g. > > FAIL -> N/A test - systemtap.samples/gtod.exp(gtod (0)) > [...] > N/A -> PASS test - systemtap.samples/gtod.exp(gtod (300)) > > It would be useful to keep either the kernel version or the systemtap > snapshot version constant between runs. It looks like currently both are > updated at the same time. Although it isn't completely clear, the > version is always reported as 0.7.1, it would be good to mention the > last git commit (as in stap -V). This is actually not something that can easily be solved without hardcoding test-specific logic to strip out dynamic content in these messages. Since labrat is not a project-specific reporting system, implementing such hardcoded rules would be less desirable. Also note that wiring this logic in a way that is not test specific is somewhat dangerous because you may end stripping out important information from the messages unless people are extremely careful in how they write the pass/fail messages. If there is a very strict convention (one that gets enforced) on how dynamic data is included in these messages, I can definitely look at implementing the logic to strip it out based on the established conventions. Kris