From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 15997 invoked by alias); 8 Dec 2001 02:57:38 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 15976 invoked from network); 8 Dec 2001 02:57:35 -0000 Received: from unknown (HELO nile.gnat.com) (205.232.38.5) by sources.redhat.com with SMTP; 8 Dec 2001 02:57:35 -0000 Received: by nile.gnat.com (Postfix, from userid 338) id 7B397F28C7; Fri, 7 Dec 2001 21:56:50 -0500 (EST) From: dewar@gnat.com To: dewar@gnat.com, kenner@vlsi1.ultra.nyu.edu, mrs@windriver.com, zack@codesourcery.com Subject: Re: ACATS legal status cleared by FSF Cc: gcc@gcc.gnu.org Message-Id: <20011208025650.7B397F28C7@nile.gnat.com> Date: Fri, 07 Dec 2001 19:12:00 -0000 X-SW-Source: 2001-12/txt/msg00411.txt.bz2 Note that just *reading* B tests to see if the output is correct is a very difficult task, one that only someone with quite a bit of ACVC/ACATS validation experience can do. A formal validation run using these tests often involves several days of painstaking manual work by someone who is an expert in the B tests to assure compliance. Consider various possibilities: 1. You must not make changes to the compiler that change the B tests baselines. This is far too restrictive, it would forbid even fixing a spelling error. In practice fixing a bug in one part of the compiler can often change an error message elsewhere in one of the B tests (by implementing better error recovery). Most often such changes are for the better. 2. If the B test baseline changes, verify the baseline change and adjust the baseline. This would be fine, except that the verification process typically requires someone with very good expertise in Ada semantics, AND very good familiarity with the ACATS test suite. There are not many such people in the world. A huge effort from any Ada vendor goes into making sure that the compiler passes all the B tests. Many have actively questioned the value of this effort in the past, but if you want to formally validate you have no choice. But this does not necessarily mean that these are well chosen tests from the point of view of our use here at gnu.org.