Re: Mauve wishlist

[David Gilbert <david.gilbert@object-refinery.com>]

Bryce McKinlay wrote:

>
> It is not reasonable to expect test case developers ensure that all 
> tests "run linearly". Exceptions can potentially be thrown at any 
> time, so to ensure linearity, every check() call would need wrapped 
> with a try/catch.
>
If there is a linear sequence of checks in a test() method, and an 
(unexpected) exception causes the test() method to exit early (after, 
say, 3 checks out of 10), I don't consider that to be non-linear.  If we 
are comparing run A to run B, and 10 checks complete in run A, but only 
3 checks complete in run B, we can safely assume that checks 4 to 10 
were not completed in run B, and report that.

The majority of test() methods in Mauve are written that way, so I don't 
think it is an unreasonable requirement, especially if it means we can 
develop better comparison/regression reporting on top of the existing 
TestHarness.

Regards,

Dave