Re: Autotesting (was Re: RFC: Increment and Decrement)

[Jim Kingdon <kingdon@panix.com>]

> I do intend to look more closely at auto-testing at some point. It has
> been on my list to do for some time. (Incidentally, I have used it
> before: to verify the packed boolean tables implementation.)

Oh yes, I remember.  That's why I thought I might con you into writing
more ;-).

> I thought that I would probably have to figure out how to instantiate
> a Lisp stream buffer [. . . etc etc etc and so on]

Ah yes, but unfortunately for your excuse, this part was written years
ago by Stan.  There's a function in lisp.c called
read_form_from_string which has all I needed to write a simple test
(which I have now checked in).

> might have to do a full init on the Xconq engine to make sure that
> various crucial data structures and variables got initialized.... IOW,
> potentially a pain.

I'm not sure it affects the lisp.c code, but in general, yeah, these
kinds of global "crucial data structures and variables" are the bane
of unit testing.  There are basically two solutions: (1) make the code
more modular, so it is easier to just set up the part you are testing,
or (2) as part of the test code, have a bunch of helper methods which
do various kinds of setup.  Most systems I know which are more fully
unit tested do some of each.

As for where autotest.c stands, well, it hasn't really tackled these
issues.  It does a fairly normal xconq setup (using auto.g), and then
it runs the tests in order (without resetting anything in between).
Generally speaking, I had in mind moving towards a situation where
each unit test blows away any state which might confuse it, and then
sets up what it needs to run.  I could elaborate, but for now I'll
just say that (a) to really get this working smoothly won't happen
overnight, but (b) it is usually feasible to do it one step at a time,
starting with those tests, or styles of tests, which are easy to write
now and working up to the tests which are now hard.