Re: ACATS legal status cleared by FSF

["Joseph S. Myers" <jsm28@cam.ac.uk>]

On Thu, 6 Dec 2001, Geert Bosch wrote:

> It is virtually impossible for people to "break" these tests, which
> is why I say they are of no value. Even if people *do* manage to break
> them (in the hypothetical case that the maintainers would not catch the
> error before approving), this will not go unnoticed for a long
> time anyway. In the mean time, the *only* programs affected  are 
> programs with fatal errors to start with.

I have not examined these tests, but for C I would consider it valuable
for every constraint from either standard version, and every diagnostic
message that the front end can issue, to have at least one test.  (Ideally
the test suite should aim for high code coverage in the compiler; I
haven't tried running in conjunction with gcov to see how far away from
this we are.)  In general I consider a patch which adds a diagnostic 
without including a test exercising that code path, or adds a language 
feature without proper tests for the associated constraints, to be 
defective.  I get the impression from this discussion that these tests 
represent something similar for Ada - tests of the ways in which code can 
be defective and diagnostics issued for it - and so would be of similar 
value.  It is just as much a fundamental part of avoiding regressions that 
bad code remains diagnosed and the messages do not get worse, as that good 
code continues to compile and code quality does not get worse.

-- 
Joseph S. Myers
jsm28@cam.ac.uk