Re: [Kissme-general] Re: Should I or not submit changes?

[Dalibor Topic <robilad@yahoo.com>]

--- Mark Wielaard <mark@klomp.org> wrote:
> Hi,
> 
> On Wed, 2002-07-17 at 16:13, Andrew Haley wrote:
> > If mauve wasn't such a pain to use and hack on I
> certainly would have
> > contributed more to it.  Dejagnu, although
> horrible in its own special
> > way, at least allows individual tests to be
> developed and used without
> > needing the framework.
> 
> I was thinking of having very simple support for
> "output comparison"
> tests. You write a bla/test.java class with a normal
> static main method
> that just prints to System.out.println and there is
> a bla/test.out file
> that contains the correct output of the test. The
> only drawback is that
> when people explicitly use Testlet.check() it is
> much more clear what it
> is they are precisely testing.

Kaffe's regression testing suite relies mostly on
output comparison. While it's a great way to create
tests in a minute, it's not that great in the end.

Sometimes the output depends on the locale you are in,
so people get erraneous failures. Sometimes people
will include toString() output in the expected
results, which is quite painful when you test for
exceptions. Unless you want to match Sun's messages
literally, of course.

In my opinion, explicit check methods that verify
single properties are more useful, as it is easier to
extract context information about failures, most
importantly which test failed. That is really hard if
something messes up your tests output beyound the
region matching capabilities of diff.

best regards,

dalibor topic

__________________________________________________
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
http://autos.yahoo.com