From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 23748 invoked by alias); 22 Mar 2006 11:12:27 -0000 Received: (qmail 23739 invoked by uid 22791); 22 Mar 2006 11:12:27 -0000 X-Spam-Check-By: sourceware.org Received: from outmail128168.authsmtp.com (HELO outmail128168.authsmtp.com) (62.13.128.168) by sourceware.org (qpsmtpd/0.31) with ESMTP; Wed, 22 Mar 2006 11:12:26 +0000 Received: from [192.168.1.100] (cpc4-hem14-0-0-cust839.lutn.cable.ntl.com [86.5.111.72]) (authenticated bits=0) by squirrel.dmpriest.net.uk (8.13.3/8.13.3/Kp) with ESMTP id k2MBC8qr006697; Wed, 22 Mar 2006 11:12:08 GMT (envelope-from david.gilbert@object-refinery.com) Message-ID: <44213403.3020307@object-refinery.com> Date: Wed, 22 Mar 2006 11:12:00 -0000 From: David Gilbert User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051026) MIME-Version: 1.0 To: Bryce McKinlay CC: Anthony Balkissoon , classpath@gnu.org, mauve-discuss@sources.redhat.com Subject: Re: Mauve wishlist References: <1142613140.3805.20.camel@rh-ibm-t41> <1142873502.3112.16.camel@tony.toronto.redhat.com> <4420336F.4070602@object-refinery.com> <4420874C.2090800@redhat.com> In-Reply-To: <4420874C.2090800@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Server-Quench: b40b2920-b994-11da-b770-001185d377ca X-Authentic-SMTP: 61633132333134.squirrel.dmpriest.net.uk:1.47/Kp X-Powered-By: AuthSMTP - http://www.authsmtp.com - Authenticated SMTP Mail Relay X-Report-SPAM: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-Virus-Status: No virus detected - but ensure you scan with your own anti-virus system! X-IsSubscribed: yes Mailing-List: contact mauve-discuss-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: mauve-discuss-owner@sourceware.org X-SW-Source: 2006-q1/txt/msg00071.txt.bz2 Bryce McKinlay wrote: > > It is not reasonable to expect test case developers ensure that all > tests "run linearly". Exceptions can potentially be thrown at any > time, so to ensure linearity, every check() call would need wrapped > with a try/catch. > If there is a linear sequence of checks in a test() method, and an (unexpected) exception causes the test() method to exit early (after, say, 3 checks out of 10), I don't consider that to be non-linear. If we are comparing run A to run B, and 10 checks complete in run A, but only 3 checks complete in run B, we can safely assume that checks 4 to 10 were not completed in run B, and report that. The majority of test() methods in Mauve are written that way, so I don't think it is an unreasonable requirement, especially if it means we can develop better comparison/regression reporting on top of the existing TestHarness. Regards, Dave