GDL Notice: Arithmetic Operators / Quasi-Formulae

GDL Notice: Arithmetic Operators / Quasi-Formulae

[Eric McDonald]

Hello Xconq Game Designers,

  I just added the four basic arithmetic operators to GDL. They are:
"+", "-", "*", and "/" (all without the quotes). I think that when they
are used in conjunction with defined symbols in GDL, they can provide
much of the power and flexibility you might expect from having formulae
in GDL.

  Here are some rather unsophisticated examples of their usage that I
hacked into Wreckreation for testing:

(define VILLAGE_STORAGE_MULT 5)
(define HUMAN_BURDEN_CAP 40)
(table unit-storage-x add
    (village food 200)
    (village food (* HUMAN_BURDEN_CAP VILLAGE_STORAGE_MULT))
    (village food (/ (* HUMAN_BURDEN_CAP VILLAGE_STORAGE_MULT) 1))
    (village (food orc-food) ((+ 50 (- 200 50)) (/ 0 100)))
    (village (food orc-food) (+ 50 (150 50)))
    (village (food orc-food) (+ (150 50) 50))
    (village (food orc-food) (+ (50 50) (150 50)))
)

  The first 3 "(village food ..." declarations are all equivalent to one
another. The last 4 declarations are also all equivalent to one another,
and demonstrate the new operator's powerful ability to manipulate lists
and mixtures of numbers and lists. In all of the above declarations, the
gist is that the storage capacity of a 'village' for 'food' is always
200, despite the different methods of calculating it. In the last 4
declarations, all of the storage capacity for 'orc-food' is always 100,
despite the different methods of calculating it.

  Some things to note:
(1) I still consider this experimental. Although my testing did not
reveal any bugs (__well, it did, but I fixed them :-), there may be some
lurking in the code that handles the operators. Please kick the tires
some and tell me what you think about it.
(2) Note that the notation is postfix. So, operator first, then the
operands.
(3) There is no limit on the number of operands that can be supplied to
the operators.
(4) Division by zero will produce a warning if you attempt it. This is
to say that only the first operand (the dividend) of a division operator
can be 0; all others (the divisors) must be non-zero, and return an
empty result.
(5) List lengths must match, if lists are used. Failure to match list
lengths will produce an warning, and return an empty result.
(6) As can be seen above, you can nest operators. The innermost (most
deeply buried in parentheses) operations are performed first.
(7) Precedence is enforced by the parentheses; there is no natural
precedence order. Expressions are, of course, read from left to right.
(8) I will document this fairly soon. I want to make sure bugs are
ironed out, produce some neater examples, and add a bell and a whistle
before making this feature official.
(9) The "bell" (not yet added) is "(- n)", which will decrement n by 1.
(10) The "whistle" (not yet added) is "(+ n)", which will increment n by
1.
(11) Xconq does not support rational numbers (fractions), and so results
between 0 and 1 will be 0. This is standard integer arithmetic.
(12) I may, at some point, add a modulo operator ("%") for extracting
remainders of divisions.
(13) Lists and individual numbers can be mixed freely. Again, the only
constraint is that the lengths of all the lists must match.
(14) Doing math on dice specs is not recommended with this set of
operators. They treat dice specs like normal integers (> 16384), and
will likely screw with your dice specs in ways that you might not
imagine.

  Enjoy,
   Eric

Newest code crashes Tcl/Tk

[Brian Dunn]

Something in the last two days' worth of changes has caused an XWindows lock-up.  The last time this happened it was on a Tcl/Tk call.  The problem disappeared when some bug fix in the Mac Tcl code related to flickering / fast refreshes was fixed.  The problem has now reappered for the first time since then, which was months ago.  I played many games successfully since then, and suddenly it has locked up again.

Note that when I recompile XConq on my Linux machine, many files in the mac directory are in fact compiled, for some reason.


Brian

Re: Newest code crashes Tcl/Tk

[Eric McDonald]

Hi Brian,

On Mon, 21 Jun 2004, Brian Dunn wrote:

> Something in the last two days' worth of changes has caused an 
>XWindows lock-up.  The last time this happened it was on a Tcl/Tk 
>call.  The problem disappeared when some bug fix in the Mac Tcl 
>code related to flickering / fast refreshes was fixed.  The 
>problem has now reappered for the first time since then, which 
>was months ago.  I played many games successfully since then, and 
>suddenly it has locked up again.

Can you provide a backtrace of the call stack for us?

Also, does this happen with all games, all the time? And when does 
it occur? Is it apparently random, does it happen in conjunction 
with a specific action, or does it happen at the beginning of the 
game?

> Note that when I recompile XConq on my Linux machine, many 
>files in the mac directory are in fact compiled, for some reason.

This seems odd. I don't recall that happening on my machine. Can 
you send me your Makefiles from the top-level dir and the tcltk 
dir under it?

Thanks for the bug report,
   Eric

Re: GDL Notice: Arithmetic Operators / Quasi-Formulae

[Eric McDonald]

On Mon, 21 Jun 2004, Eric McDonald wrote:

> (2) Note that the notation is postfix. So, operator first, then the
> operands.

The notation is prefix, not postfix.

Also, there might be a bug when doing (/ x (y z)) or (- x (y z)). 
I will verify this tonight; the thought just occurred to me while 
I was daydreaming about the code at "work"....

Eric

Re: Newest code crashes Tcl/Tk

[Hans Ronne]

>Something in the last two days' worth of changes has caused an XWindows
>lock-up.  The last time this happened it was on a Tcl/Tk call.  The
>problem disappeared when some bug fix in the Mac Tcl code related to
>flickering / fast refreshes was fixed.  The problem has now reappered for
>the first time since then, which was months ago.  I played many games
>successfully since then, and suddenly it has locked up again.

There was a recent change to the tcltk interface code which reduced buzzing
during autoscrolling. It should not cause any lockups, but then we never
understood why you had these Debian-specific lockups previously, or why
they disappeared when redrawing of the world map was fixed back in March.

Since you are using the CVS sources, perhaps you could make a couple of
checkouts with different dates, to verify that this change is responsible?
If that is the case, a checkout with -D 2004-06-20 should work while -D
2004-06-21 should freeze (note that the dates in the ChangeLog do not
always agree with the checkout dates).

It would also be useful, as Eric said, to know a little more about your
freezes: if they happen all the time or only when you do certain things etc.

>Note that when I recompile XConq on my Linux machine, many files in the
>mac directory are in fact compiled, for some reason.

This does not make sense to me. There is no makefile in the mac directory,
so there is no way these files could compile under Linux.

Hans

Re: GDL Notice: Arithmetic Operators / Quasi-Formulae

[Eric McDonald]

Hello Xconq Game Designers,

On Mon, 2004-06-21 at 00:46, Eric McDonald wrote:

>     (village (food orc-food) ((+ 50 (- 200 50)) (/ 0 100)))

>  In the last 4
> declarations, all of the storage capacity for 'orc-food' is always 100,
> despite the different methods of calculating it.

Not quite true. The first of the last four (included above) actually
yields an 'orc-food' of 0, not 100.

>   Some things to note:
> (1) I still consider this experimental. Although my testing did not
> reveal any bugs (__well, it did, but I fixed them :-), there may be some
> lurking in the code that handles the operators. Please kick the tires
> some and tell me what you think about it.

I fixed the problem that I mentioned earlier.
  (- x (y z))
actually was fine, but:
  (- (y z) x)
and:
  (- (a b) (c d))
were not. But, they are now fixed.
Unless someone finds a bug with this stuff, I am going to consider it
official.

> (3) There is no limit on the number of operands that can be supplied to
> the operators.

For example:
(village food (+ 50 50 75 25))   ; evaluates to 200
(define HUMAN_BURDEN_CAP (/ 400 5 2 1 1))       ; evaluates to 40

> (9) The "bell" (not yet added) is "(- n)", which will decrement n by 1.
> (10) The "whistle" (not yet added) is "(+ n)", which will increment n by
> 1.

These have now been added and work fine. Here is a frivolous example:
(define VILLAGE_STORAGE_MULT (+ (- 5)))         ; evaluates to 5
   ; because 5 - 1 = 4, and then 4 + 1 = 5

> (13) Lists and individual numbers can be mixed freely. Again, the only
> constraint is that the lengths of all the lists must match.

For example:
(village (food orc-food) (/ (1000 500) 5))	; evaluates to (200 100)
(village (food orc-food) (/ (1000 1000) (5 0.10)))
   ; evaluates to (200 100)

Note that 0.10 is treated as 10 by Xconq internally. 10 = 0.10 = 10% in
the world of Xconq. Similarly, 400 = 4.00 = 400%. My point is that the
GDL math expressions can contain these kinds of numbers (decimals and
percents); just be aware of what internal values they really represent.

> (14) Doing math on dice specs is not recommended with this set of
> operators. They treat dice specs like normal integers (> 16384), and
> will likely screw with your dice specs in ways that you might not
> imagine.

Documentation is forthcoming. I will start with extremely simple
examples and progress to advanced ones. In the advanced examples, I will
mention how to safely manipulate dice specs using arithmetic expressions
(involving multiples of certain powers of 2).

  Enjoy,
    Eric

Re: GDL Notice: Arithmetic Operators / Quasi-Formulae

[Jim Kingdon]

> These have now been added and work fine. Here is a frivolous example:
> (define VILLAGE_STORAGE_MULT (+ (- 5)))         ; evaluates to 5
>    ; because 5 - 1 = 4, and then 4 + 1 = 5

Hmm.  I didn't notice this the first time through, but this doesn't
read so well.  (- 5) looks like -5.

I suppose

(++ 5)    =>    6

might be better, although of course it isn't like the C ++ because it
just returns the incremented value, rather than assigning.

There's also

(inc 5)

Or of course going back to (+ 5 1) if all of these choices seem
problematic.

Re: GDL Notice: Arithmetic Operators / Quasi-Formulae

[Juergen Ruehle]

Jim Kingdon writes:
 > > These have now been added and work fine. Here is a frivolous example:
 > > (define VILLAGE_STORAGE_MULT (+ (- 5)))         ; evaluates to 5
 > >    ; because 5 - 1 = 4, and then 4 + 1 = 5
 > 
 > Hmm.  I didn't notice this the first time through, but this doesn't
 > read so well.  (- 5) looks like -5.
 > 
 > I suppose
 > 
 > (++ 5)    =>    6
 > 
 > might be better, although of course it isn't like the C ++ because it
 > just returns the incremented value, rather than assigning.
 > 
 > There's also
 > 
 > (inc 5)
 > 
 > Or of course going back to (+ 5 1) if all of these choices seem
 > problematic.

Actually I would prefer the (Common) LISP version:

  (1+ 5) => 6
  (1- 5) => 6

while

  (+ 5) => 5
  (- 5) => -5

This avoids overloading of + and -, but that's just IMO.

Re: GDL Notice: Arithmetic Operators / Quasi-Formulae

[Eric McDonald]

Hi Jim,

On Tue, 2004-06-22 at 00:19, Jim Kingdon wrote:

> I suppose
> 
> (++ 5)    =>    6

This can be done and seems reasonable. Another nested test would have to
be hacked into the tokenizer, though.

> might be better, although of course it isn't like the C ++ because it
> just returns the incremented value, rather than assigning.

True. But, I think that after a C programmer, unfamiliar with Lisp, has
read that 'remove' and 'append' don't really assign anything either,
he/she is going to get the hint that assigning isn't the Lisp/GDL way
(aside from 'define' and 'set').

> (inc 5)

Also sounds reasonable. This would save us from any further tokenizer
hacking.

> Or of course going back to (+ 5 1) if all of these choices seem
> problematic.

Yeah, but the whole point was to try to make some shorthand notation for
increment and decrement.

  Thanks for the feedback,
     Eric

RFC: Increment and Decrement (was Re: GDL Notice: Arithmetic Operators / Quasi-Formulae)

[Eric McDonald]

Hi Juergen,

On Tue, 2004-06-22 at 03:50, Juergen Ruehle wrote:

> Actually I would prefer the (Common) LISP version:

I guess I should have checked to see what Common Lisp did instead of
writing some code first.

>   (1+ 5) => 6
>   (1- 5) => 6

This seems fairly reasonable. However, GDL has dice specs which look
like mmdnn+oo. Doing something like "1+" looks like part of a dice spec.
This is not a problem from the tokenization point of view; it would
require another tokenizer hack, but that is manageable. I am more
concerned about confusion on the part of GDL users.

>   (+ 5) => 5
>   (- 5) => -5

I agree. One of the concerns that was in the back of mind was how to
easily negate a number. I think I will change the behavior of unary "+"
and "-" to what you just suggested.

> This avoids overloading of + and -, but that's just IMO.

Thanks for the feedback.

So, I guess the choices for increment and decrement are:
(++ n) (-- n)
(inc n) (dec n)
(1+ n) (1- n)

Does anyone have any preferences (aside from those already stated)?

  Regards,
    Eric

Re: RFC: Increment and Decrement (was Re: GDL Notice: Arithmetic Operators / Quasi-Formulae)

[Jim Kingdon]

> So, I guess the choices for increment and decrement are:
> (++ n) (-- n)
> (inc n) (dec n)
> (1+ n) (1- n)

I'd say "1+" and "1-".  Dice specs contain a "d", so they don't look
all *that* confusable to me.

Re: RFC: Increment and Decrement (was Re: GDL Notice: Arithmetic Operators / Quasi-Formulae)

[Eric McDonald]

On Tue, 2004-06-22 at 08:50, Jim Kingdon wrote:
> > So, I guess the choices for increment and decrement are:
> > (++ n) (-- n)
> > (inc n) (dec n)
> > (1+ n) (1- n)
> 
> I'd say "1+" and "1-".  Dice specs contain a "d", so they don't look
> all *that* confusable to me.

I will add "1+" and "1-" at a later point. They need to be defined as
new operators, and it seems that they might work a bit differently than
the basic four.

With "+", the lengths of the list operands must be the same: 
  (+ 1 (2 3) (4 5))
But, it would seem reasonable that this constraint need not apply with
"1+" and "1-":
  (1+ (1 2 3) (4 5))
since each operand is operated on independently of the others. The
current structure of 'eval_arithmetic_expression' does not allow this
behavior to be easily integrated; I would have to write a new logic
branch in the function to deal with this.

For the time being, I will leave the increment and decrement
functionality out.

I did address Juergen's suggestion about reflection and negation:
 (+ 5)
now evaluates to 5, and 
 (- 5)
now evaluates to -5.

I also added the seemingly useless feature of "+" returning the additive
identity (0) and "*" returning the multiplicative identity (1), to
comply with Common Lisp.

As far as the dice spec argument goes, I admit that is was lame. I was
in a hurry to get out the door this morning when I wrote it. The real
reason for my reluctance with that notation is because I didn't want to
hack the "el cheapo" Lisp reader (as it is referred to in the comments
introducing it) again, if I didn't have to. As it turns out, my concern
was somewhat justified, after I made the horrific discovery this evening
that I had broken negative number tokenization a few days ago. I have
now fixed this, _but it shows that even the simplest hacks can produce
big bugs.

Eric

Re: RFC: Increment and Decrement (was Re: GDL Notice: Arithmetic Operators / Quasi-Formulae)

[Jim Kingdon]

> As it turns out, my concern was somewhat justified, after I made the
> horrific discovery this evening that I had broken negative number
> tokenization a few days ago. I have now fixed this, _but it shows that
> even the simplest hacks can produce big bugs.

That's what you get for not writing any tests :-).

Unlike many parts of xconq, writing autotest-style tests for GDL
parsing is really easy.

I suppose if I feel like writing some xconq tests, I could start with
these, rather than proceeding directly to the nastier, harder to test
things like "make sure the AI knows how to use a transport".

RE: RFC: Increment and Decrement (was Re: GDL Notice: ArithmeticOperators / Quasi-Formulae)

[Erik Jessen]

I sure like getting the basics well-tested.
It's amazing how many simple bugs are found, even in stable code that's
been in use for years.

Erik

-----Original Message-----
From: xconq7-owner@sources.redhat.com
[mailto:xconq7-owner@sources.redhat.com] On Behalf Of Jim Kingdon
Sent: Tuesday, June 22, 2004 11:02 PM
To: mcdonald@phy.cmich.edu
Cc: xconq7@sources.redhat.com
Subject: Re: RFC: Increment and Decrement (was Re: GDL Notice:
ArithmeticOperators / Quasi-Formulae)


> As it turns out, my concern was somewhat justified, after I made the 
> horrific discovery this evening that I had broken negative number 
> tokenization a few days ago. I have now fixed this, _but it shows that

> even the simplest hacks can produce big bugs.

That's what you get for not writing any tests :-).

Unlike many parts of xconq, writing autotest-style tests for GDL parsing
is really easy.

I suppose if I feel like writing some xconq tests, I could start with
these, rather than proceeding directly to the nastier, harder to test
things like "make sure the AI knows how to use a transport".

Autotesting (was Re: RFC: Increment and Decrement)

[Eric McDonald]

On Wed, 2004-06-23 at 00:02, Jim Kingdon wrote:
> > As it turns out, my concern was somewhat justified, after I made the
> > horrific discovery this evening that I had broken negative number
> > tokenization a few days ago. I have now fixed this, _but it shows that
> > even the simplest hacks can produce big bugs.
> 
> That's what you get for not writing any tests :-).

Yeah, yeah. ;-)

Actually, I thought I had tested the negative number stuff, but it must
have never gotten past the intention stage.

> Unlike many parts of xconq, writing autotest-style tests for GDL
> parsing is really easy.

I had considered this, but I wasn't sure how much work would be
involved. I thought that I would probably have to figure out how to
instantiate a Lisp stream buffer (which probably isn't all that
involved, but it was one more thing to worry about), tell it to read
from a specific game module, and then look up the initialized data. I
also thought that I might not be able to just test the aspects of GDL I
was interested in, but might have to do a full init on the Xconq engine
to make sure that various crucial data structures and variables got
initialized.... IOW, potentially a pain.

Considering all that, I thought it would be easier to simply do testing
within an existing game module (wreckreation.g).

> I suppose if I feel like writing some xconq tests, I could start with
> these, rather than proceeding directly to the nastier, harder to test
> things like "make sure the AI knows how to use a transport".

I do intend to look more closely at auto-testing at some point. It has
been on my list to do for some time. (Incidentally, I have used it
before: to verify the packed boolean tables implementation.)

Eric

Re: Newest code crashes Tcl/Tk

[Brian Dunn]

> Since you are using the CVS sources, perhaps you could make a couple of
> checkouts with different dates, to verify that this change is responsible?
> If that is the case, a checkout with -D 2004-06-20 should work while -D
> 2004-06-21 should freeze (note that the dates in the ChangeLog do not
> always agree with the checkout dates).

Going backwards, the 6/12 (not 6/21) build seems to work fine, and anything after can lock up the machine while clicking on the world map to zoom to new locations.  This includes after the game is quit but you may continue to look around.


>> Note that when I recompile XConq on my Linux machine, many files in the
>> mac directory are in fact compiled, for some reason.

I take that back. No .o files appear in the mac directory.


Brian

Re: Newest code crashes Tcl/Tk

[Hans Ronne]

>> Since you are using the CVS sources, perhaps you could make a couple of
>> checkouts with different dates, to verify that this change is responsible?
>> If that is the case, a checkout with -D 2004-06-20 should work while -D
>> 2004-06-21 should freeze (note that the dates in the ChangeLog do not
>> always agree with the checkout dates).
>
>Going backwards, the 6/12 (not 6/21) build seems to work fine, and
>anything after can lock up the machine while clicking on the world map to
>zoom to new locations.  This includes after the game is quit but you may
>continue to look around.

Thanks! Is that the -D 2004-06-12 checkout, that is the one that contains
entries from 6/11 but not 6/12? In that case, I am surprised. Most of the
code that was checked in on 6/12 were fixes to the Mac interface or to the
Windows app, neither of which could explain your problem.

What remains are two things. One is Eric's fix to change_unit_type, which
is unlikely to be the cause of the freezes. The other is the following
changes relating to acp-independent units:

	Fix update_unit_display buzzing for acp-independent units
	that were waiting for a build task.
	* plan.c (set_waiting_for_tasks): Return without calling
	update_unit_display if the flag already is set.

	Fix so that acp-independent building may resume in the same
	turn if we change the build task and thus may have enough
	material to build on the new unit type.
	* task.c (set_build_task): Call set_unit_buildingdone to reset
	the buildingdone flag to FALSE.
	(push_build_task): Call set_unit_buildingdone to reset the
	buildingdone flag to FALSE.
	(resume_build_task): Call set_unit_buildingdone to reset the
	buildingdone flag to FALSE.

Does the game where you see freezes have any acp-independent units?

Hans

Re: Newest code crashes Tcl/Tk

[Brian Dunn]

> Thanks! Is that the -D 2004-06-12 checkout, that is the one that contains
> entries from 6/11 but not 6/12?

I used 'cvs update -D 2004-06-12' then ./configure, then make, then make install, and the ChangeLog's first entry is:

2004-06-12  Eric McDonald  <mcdonald@phy.cmich.edu>

         Kill any changed type which has a side which is not compatible
         with the transport it is in.
         * unit.c (change_unit_type): If the changing unit is in a
         transport and is changing side, then see if the transport trusts
         the new side. If not, then kill the unit.

The next entry is from 6-11.  There may have been other 6-12 changes but they did not show up in the checkout you get with cvs update -D 2004-06-12.

The game is wreckreation, which in this old version is a simpler game with 'good, bad, ugly'.  The games which locked the machine in later versions was also wreckreation.  When the machine locks, it totally freezes and requires a hardware reset.

Re: Newest code crashes Tcl/Tk

[Eric McDonald]

Hi Brian,

On Wed, 2004-06-23 at 21:32, Brian Dunn wrote:

> 2004-06-12  Eric McDonald  <mcdonald@phy.cmich.edu>

I guess I get to look forward to another wave of spam now. :-)

> The game is wreckreation, which in this old version is a simpler game with 'good, bad, ugly'.  
> The games which locked the machine in later versions was also wreckreation.

You have not seen this problem occur with any other game?
Do the freezes happen when an AI is playing, or do they seem to happen
when you are playing?

> When the machine locks, it totally freezes and requires a hardware reset.

On a system with protected memory and whatnot, this is fairly hard to
imagine, unless there are other factors involved such as a flaky video
card or some sort of faulty in-kernel video support. Furthermore, this
does not seem to happen on my development system.

If you are using Linux when this happens, can you give some details
about your kernel: which version (particularly, is it a 2.4 or 2.6),
does it have NPTL support, does it have DRM support? And some other
details: what version of Tcl and is it compiled for multithreading, and
what version of XFree86 and which of its extension modules are loaded?

  Thanks,
    Eric

Re: Newest code crashes Tcl/Tk

[Brian Dunn]

>> The game is wreckreation, which in this old version is a simpler game with
>> 'good, bad, ugly'.  The games which locked the machine in later versions was
>> also wreckreation.

> You have not seen this problem occur with any other game?

I'll try the standard game and see if it shows up.



> Do the freezes happen when an AI is playing, or do they seem to happen
> when you are playing?

I can get it to freeze after I quit and am simply clicking around the map before final game shutdown.




>> When the machine locks, it totally freezes and requires a hardware reset.

> On a system with protected memory and whatnot, this is fairly hard to
> imagine, unless there are other factors involved such as a flaky video
> card or some sort of faulty in-kernel video support. Furthermore, this
> does not seem to happen on my development system.

All other operations seem ok.  No system locks other than Xconq.




> If you are using Linux when this happens, can you give some details
> about your kernel: which version (particularly, is it a 2.4 or 2.6),

Debian Stable/Woody, nothing added, kernel 2.4.18-bf2.4, tcl/tk 8.3.



> does it have NPTL support, does it have DRM support?

neither 'nptl' nor 'drm' shows up with apt-cache search nor with man -k



  And some other
> details: what version of Tcl and is it compiled for multithreading

Unknown about multithreading.  Doesn't show up in apt-cache search


, and
> what version of XFree86

4.01


> and which of its extension modules are loaded?

Quite a list is shown.  Want the whole list?  Anything in particular to look for?

Re: Autotesting (was Re: RFC: Increment and Decrement)

[Jim Kingdon]

> I do intend to look more closely at auto-testing at some point. It has
> been on my list to do for some time. (Incidentally, I have used it
> before: to verify the packed boolean tables implementation.)

Oh yes, I remember.  That's why I thought I might con you into writing
more ;-).

> I thought that I would probably have to figure out how to instantiate
> a Lisp stream buffer [. . . etc etc etc and so on]

Ah yes, but unfortunately for your excuse, this part was written years
ago by Stan.  There's a function in lisp.c called
read_form_from_string which has all I needed to write a simple test
(which I have now checked in).

> might have to do a full init on the Xconq engine to make sure that
> various crucial data structures and variables got initialized.... IOW,
> potentially a pain.

I'm not sure it affects the lisp.c code, but in general, yeah, these
kinds of global "crucial data structures and variables" are the bane
of unit testing.  There are basically two solutions: (1) make the code
more modular, so it is easier to just set up the part you are testing,
or (2) as part of the test code, have a bunch of helper methods which
do various kinds of setup.  Most systems I know which are more fully
unit tested do some of each.

As for where autotest.c stands, well, it hasn't really tackled these
issues.  It does a fairly normal xconq setup (using auto.g), and then
it runs the tests in order (without resetting anything in between).
Generally speaking, I had in mind moving towards a situation where
each unit test blows away any state which might confuse it, and then
sets up what it needs to run.  I could elaborate, but for now I'll
just say that (a) to really get this working smoothly won't happen
overnight, but (b) it is usually feasible to do it one step at a time,
starting with those tests, or styles of tests, which are easy to write
now and working up to the tests which are now hard.

Re: Newest code crashes Tcl/Tk

[Hans Ronne]

>> Thanks! Is that the -D 2004-06-12 checkout, that is the one that contains
>> entries from 6/11 but not 6/12?
>
>I used 'cvs update -D 2004-06-12' then ./configure, then make, then make
>install, and the ChangeLog's first entry is:
>
>2004-06-12  Eric McDonald  <mcdonald@phy.cmich.edu>
>
>         Kill any changed type which has a side which is not compatible
>         with the transport it is in.
>         * unit.c (change_unit_type): If the changing unit is in a
>         transport and is changing side, then see if the transport trusts
>         the new side. If not, then kill the unit.
>
>The next entry is from 6-11.  There may have been other 6-12 changes but
>they did not show up in the checkout you get with cvs update -D 2004-06-12.

That's strange. When I do the same thing, my first entry is:

2004-06-11  Hans Ronne <hronne@comhem.se>

	Hide the unused More Variants button in the Mac interface.
	* macinit.c (create_variants_dialog): Call HideDItem for
	diVariantsMoreVariants.

Not a big difference, but you would expect a checkout with the same date to
be identical irrespective of who makes it.

In any case, this rules out Eric's change_unit_type hack as a possible
source of the problem. What remains is the code that I checked in on 6/12.
To be absolutely clear, what was the first ChangeLog entry in the
subsequent checkout in which the bug first appeared? I'm asking since none
of the code i checked in on 6/12 contains any tcltk interface code.

Hans

Re: Newest code crashes Tcl/Tk

[Brian Dunn]

> Not a big difference, but you would expect a checkout with the same date to
> be identical irrespective of who makes it.

Looks like cvs update -D takes local time zone into account...


> To be absolutely clear, what was the first ChangeLog entry in the
> subsequent checkout in which the bug first appeared? I'm asking since none
> of the code i checked in on 6/12 contains any tcltk interface code.

I'm working forward from 6/12 now...


Brian

Tcl/Tk update

[Brian Dunn]

A few more data points on this lockup problem:

With 6/12 nothing seems to crash.

With 6/20, first entry in the change log being 'More support and fixes for universal ACP and MP arithmetic', the standard game never seems to lock the system, but wreckreation does.

The system locks when clicking in the world map window.  The lock occurs at the mouse click and before the maps update.  This never seems to happen with the standard game, but it does with wreckreation.

There is a 6/19 change which mentions buzzing with auto scrolling code.  This is the only thing I can find in the change log which seems related.

Most of the other changes do not seem to be related, because this lock can occur even after you have quit the game, surrendered, and are simply moving around the map.

One interesting difference between wreckreation and the standard game is the size of the map.  Is there a boundary or array overflow?

In a moment I'll try wreckreation with a smaller map...


Brian

Tck/Tk info

[Brian Dunn]

Ok, the wreckreation game seems to be ok if you set a smaller world size.  I'll try it again a few more times with the latest version, and see how it goes.


Brian

Re: Newest code crashes Tcl/Tk

[Jim Kingdon]

> That's strange. When I do the same thing, my first entry is:

Probably timezones.  I don't remember how CVS handles them in this
context.

Re: Newest code crashes Tcl/Tk

[Hans Ronne]

>> That's strange. When I do the same thing, my first entry is:
>
>Probably timezones.  I don't remember how CVS handles them in this
>context.

Right. But this makes no sense. CVS should use local time or possibly GMT
to date a checkin. The checkin date should not depend on who is checking
out the code. In fact, if this is what is going on, I would call it a bug
in CVS.

Hans

Re: Tcl/Tk update

[Hans Ronne]

>A few more data points on this lockup problem:
>
>With 6/12 nothing seems to crash.
>
>With 6/20, first entry in the change log being 'More support and fixes for
>universal ACP and MP arithmetic', the standard game never seems to lock
>the system, but wreckreation does.
>
>The system locks when clicking in the world map window.  The lock occurs
>at the mouse click and before the maps update.  This never seems to happen
>with the standard game, but it does with wreckreation.
>
>There is a 6/19 change which mentions buzzing with auto scrolling code.
>This is the only thing I can find in the change log which seems related.

This change was my original suspect. Have you confirmed that a 6/19
checkout (which should include code up to 6/18) does not freeze?

Hans

Re: Newest code crashes Tcl/Tk

[Eric McDonald]

On Thu, 24 Jun 2004, Hans Ronne wrote:

> >> That's strange. When I do the same thing, my first entry is:
> >
> >Probably timezones.  I don't remember how CVS handles them in this
> >context.
> 
> Right. But this makes no sense. CVS should use local time or possibly GMT
> to date a checkin. The checkin date should not depend on who is checking
> out the code. In fact, if this is what is going on, I would call it a bug
> in CVS.

Well, if it's a bug, then it is a very well-documented one. :-)
From my CVS manpage:

-D date_spec
  Use the most recent revision no later than date_spec  (a  single
  argument,  date  description  specifying a date in the past).  A
  wide variety of date formats are supported,  in particular  ISO
  ("1972-09-24  20:05")  or  Internet  ("24 Sep 1972 20:05").  The
  date_spec is interpreted as being in the local timezone,  unless
  a   specific   timezone  is  specified.

[snipped, examples below]

  3/31/92 10:00:07 PST
  22:00 GMT

To get everyone on the same page, maybe we should be doing 
checkouts or updates with the date specifier and "0:00 GMT".

Eric

Re: Newest code crashes Tcl/Tk

[Hans Ronne]

>> Right. But this makes no sense. CVS should use local time or possibly GMT
>> to date a checkin. The checkin date should not depend on who is checking
>> out the code. In fact, if this is what is going on, I would call it a bug
>> in CVS.

>Well, if it's a bug, then it is a very well-documented one. :-)
From my CVS manpage:
>
>-D date_spec
>  Use the most recent revision no later than date_spec  (a  single
>  argument,  date  description  specifying a date in the past).  A
>  wide variety of date formats are supported,  in particular  ISO
>  ("1972-09-24  20:05")  or  Internet  ("24 Sep 1972 20:05").  The
>  date_spec is interpreted as being in the local timezone,  unless
>  a   specific   timezone  is  specified.

The key point is the meaning of the word "local". I always read it as local
to the CVS repository, not local to the remote user. The latter just does
not make sense, even if it is how CVS actually works.

Hans

Re: Newest code crashes Tcl/Tk

[Eric McDonald]

Hi Brian,

On Thu, 24 Jun 2004, Brian Dunn wrote:

> I can get it to freeze after I quit and am simply clicking 
>around the map before final game shutdown.

OK, thanks for the clarification.

> > On a system with protected memory and whatnot, this is fairly hard to
> > imagine, unless there are other factors involved such as a flaky video
> > card or some sort of faulty in-kernel video support. Furthermore, this
> > does not seem to happen on my development system.
> 
> All other operations seem ok.  No system locks other than Xconq.

OK.
 
> Debian Stable/Woody, nothing added, kernel 2.4.18-bf2.4, tcl/tk 8.3.

OK.

> > does it have NPTL support, does it have DRM support?
> 
> neither 'nptl' nor 'drm' shows up with apt-cache search nor with man -k

I would not expect them to show up with "man -k". I don't know 
about apt-cache, since it has been a long while since I have used 
Debian.

For DRM, at least, you will probably have to "grep DRM" on your  
kernel's config file. In the case of NPTL, it may simply be a 
matter of seeing whether it is in your libc's full name. However, 
I doubt that NPTL is the issue or that you are using it on your 
system with the older 2.4 kernel.

I am not sure that DRM or other kernel special video support is an 
issue either.

>   And some other
> > details: what version of Tcl and is it compiled for multithreading
> 
> Unknown about multithreading.  Doesn't show up in apt-cache search

You can get that from your 'tclConfig.sh' file, which, on Debian, 
is probably located in '/usr/lib/tcl8.3'.

> , and
> > what version of XFree86
> 
> 4.01
> 
> > and which of its extension modules are loaded?
> 
> Quite a list is shown.  Want the whole list?  Anything in particular to look for?

Nothing in particular at this point. If you want to give the whole 
list (via private email), you can. If you just do an "xset q", it 
should mention which ones are actually loaded by the presently 
running X server.

To be honest, my line of diagnosis was probably a bit off track 
when I mentioned NPTL or DRM. In the case of NPTL, I was concerned 
that maybe multithreaded Tcl was having a bad interaction with it, 
and destabilizing the system somehow. However, this seems 
_extremely_ unlikely.
The other thought was that some sort of in-kernel video support 
was somehow getting abused by Xconq/Tk/X and thereby causing 
problems. However, this also seems unlikely.

I think the best place to be looking is how Xconq/Tk might be 
abusing the X server. One thing that I do wonder about is how hard 
your system was really locked?

If the just X server is partially frozen, then you might be able 
to do a CTRL-ALT-BACKSPACE to kill off the server or a 
CTRL-ALT-F1 to get to tty1 (function keys combinations using F2 
through F6 are probably also available, and should drop you to 
tty's 2 through 6, respectively). If the server is busy choking on 
something, then you might need to wait a minute or two for it to 
honor your request.

If the X server is totally frozen, then, if you have another 
machine handy, you might be able to remote login (say with ssh) 
and "kill -9" the X server out of its miserable existence.

Then again, the X server might be okay, and it might be something 
evil happening with your window manager/desktop environment. In 
that case, maybe you could play with running Xconq under a 
different Window manager (perhaps a simpler one such as FVWM2, or 
even TWM).

In any case, I am more inclined to suspect that the "hard" lock 
only extends as far as the X stuff, and that the OS kernel is 
probably working just fine.

My suggestion would be that we try to verify that either the X 
server or else the window manager is getting hurt, and then start 
looking at various scenarios from that point. Considering that the 
Tcl/Tk interface on Linux/Unix systems already produces a number 
of "BadDrawable" and "BadWindow" errors (which are shown on 
whatever terminal window, 'xconq' is launched from), there is 
already circumstantial evidence to suggest Tkconq is not a 
well-behaved X11 app.

Eric

Re: Autotesting (was Re: RFC: Increment and Decrement)

[Eric McDonald]

On Thu, 24 Jun 2004, Jim Kingdon wrote:

> > I thought that I would probably have to figure out how to instantiate
> > a Lisp stream buffer [. . . etc etc etc and so on]
> 
> Ah yes, but unfortunately for your excuse, this part was written years
> ago by Stan.  There's a function in lisp.c called
> read_form_from_string which has all I needed to write a simple test
> (which I have now checked in).

I see. Thanks for popping the excuse bubble, __I guess. :-)
I will followup with some more tests, hopefully in the next few 
days.

> I'm not sure it affects the lisp.c code, but in general, yeah, these
> kinds of global "crucial data structures and variables" are the bane
> of unit testing.  There are basically two solutions: (1) make the code
> more modular, so it is easier to just set up the part you are testing,
> or (2) as part of the test code, have a bunch of helper methods which
> do various kinds of setup.

I agree with both solutions, and this is pretty much what has been 
lurking in the back of my head, regarding what likely needed to be 
done for better autotesting.

> Generally speaking, I had in mind moving towards a situation where
> each unit test blows away any state which might confuse it, and then
> sets up what it needs to run.

Yes, I completely agree with this goal.

Eric

Re: Tck/Tk info

[Brian Dunn]

> Ok, the wreckreation game seems to be ok if you set a smaller world size.  I'll
> try it again a few more times with the latest version, and see how it goes.


So far no more crashes when I use a smaller world size...

Re: Tck/Tk info

[Hans Ronne]

>> Ok, the wreckreation game seems to be ok if you set a smaller world
>>size.  I'll
>> try it again a few more times with the latest version, and see how it goes.
>
>
>So far no more crashes when I use a smaller world size...

OK. To sort these problems out once and for all: did your crashes start
when the size of the wreckreation map was increased, or because of some
other changes in the CVS? Or to put it differently: if you take the
checkout from 6/12 (which worked) and add a newer wreckreation.g with a big
map, do you then get crashes?

Hans