Re: ACATS legal status cleared by FSF

["Jerry van Dijk" <jvandyk@attglobal.net>]

guerby@acm.org writes:

 > To avoid preprocessing and script-like machinery overhead, I propose
 > that we commit the ACATS sources directly in the form expected by
 > GNAT, so we end up just having a list of main unit names, and so a
 > simple minded loop of "gnatmake x; run x" will be the only thing
 > needed to run. 
 > 
 > [1] Any objection?

I assume we want to run an ACATS subset as a basic sanity check, right ?
But even in that case, it would be advisable to follow the changes made
to the test suite. Your proposal means this has to be done manually by
someone. Who is going to monitor this, and propagate all changes in the
tests ?

 > [2] Do we want to keep some subdirectories, if so what granularity, or
 > all in one dir is okay (something like 4000 files)?

I like the current subdirectory structure, it makes it easier if you want
to check just one test, or a set or related tests.

 > [5] IMHO it is best if the form in which we commit the sources of the
 > ACATS tests is as separated as possible from the testsuite harness
 > technology at first, just sources, README and a list of test names.
 > Okay?

I would a a minium also add the scripts to run all test automatically
and report on them. Also a way to run a single test for debugging.

 > The B tests require a lot of maintainance (hundreds of pages of
 > changes each time you improve a message, split of files with too many
 > errors, etc...), and have no value for the backend, may be someone
 > will volunteer the packaging, but not me. I assume ACT dedicates
 > someone to this task anyway :).

I think it's important to test the whole GNAT system, not just the backend.
The value of the B tests is that it alerts you to unexpected changes. I think
that at least for now they should stay. We can add 'skip tests' for changes
that are to much work for the added value.

 > [12] Do we want to scrap the legalese and replace it by something else?
 > Do we want to identify changes or extension made within the GCC
 > project, and if so how?

I do not think you are allowed to 'scap the legales'.

Basically the question is if you want to see the test suite as an scaled
down ACATS (in which case all changes etc need to be documented), or see
ACATS as a starting point, and develop the suite further, for example
adding regression testing for discovered bugs, etc.

My first impulse is to stick to ACATS, to follow language development,
and add extra tests separately.

Just my opinion, of course.

-- 
--  Jerry van Dijk   | email: jvandyk@attglobal.net
--  Leiden, Holland  | web:   home.trouwweb.nl/Jerry