Re: ACATS B tests

Re: ACATS B tests

[dewar]

<<OK, I give up.  Why is that?  Is it because the error was supposed to be
on the *previous* line?
>>

Nope. Not necessarily, for example there are cases where the ACVC test
expects the error message on the definition instead of the reference
or vice versa, and there are cases where the ACVC expects a cascaded
error where we deliberately suppress it, and there are cases where
GNAT generates cascaded errors where none are expected etc etc.

Re: ACATS B tests

[guerby]

> OK, I give up.  Why is that?  Is it because the error was supposed to be
> on the *previous* line?

My rough estimate is that yes, about half of errors will be generated
not on the line flagged as ERROR, but I haven't looked at the whole B
tests, just enough to realize that it is a huge task.

> BTW, what is your plan with regard to macro substitutions?  I assume you'll
> remove the CE tests that need absolute filenames, right?

I believe the 3 CE tests doing smart things with filenames have been
removed, I have to recheck otherwise I won't bother with them.

I plan to commit with all macro substituted, most macro deal with the
testsuite being tailored to one special compiler, and we have only one
compiler out there :). GNAT handles platform specific stuff using
Standard'Attribute in a uniform source way, no need for target macros
as far as ACATS for GCC is concerned.

-- 
Laurent Guerby <guerby@acm.org>

Re: ACATS B tests

[Richard Kenner]

    In particular, just testing that errors are reported on lines flagged
    ERROR will not work for at least half of the tests, if you bother to
    look at the first B test:

OK, I give up.  Why is that?  Is it because the error was supposed to be
on the *previous* line?

BTW, what is your plan with regard to macro substitutions?  I assume you'll
remove the CE tests that need absolute filenames, right?

ACATS B tests

[guerby]

<http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ACATS/>
will give to anyone access to the ACATS sources from
web, tarballs are at 
<http://www.adaic.org/compilers/acats/2.4/tars/>

I don't remember how the B tests are run at ACT (gnatchop, flags,
multi-file tests if any, big split list, etc...) but I remember it's
much more complicated than executable tests and requires maintainance
of a baseline. In particular, just testing that errors are reported on
lines flagged ERROR will not work for at least half of the tests,
if you bother to look at the first B test:

<http://www.ada-auth.org/cgi-bin/cvsweb.cgi/ACATS/B2/B22001A.TST?rev=1.2>

Commiting the B tests as is in GCC CVS is of zero value given they are
readily available to anyone willing to spend more than one useful
second of actual work on it.

While waiting for someone volunteer to do the B test work instead of
just talking ("talk is cheap" as once said a famous participant of
this list :), I assume it is reasonable to proceed with the executable
part as discussed and so far agreed upon.

I'll upload a prototype tarball this weekend, I have a few hours of
work to do to properly match and package the current upstream ACATS
and write all the documentation I promised :).

-- 
Laurent Guerby <guerby@acm.org>