From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 2268 invoked by alias); 9 Dec 2001 23:02:53 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 2246 invoked from network); 9 Dec 2001 23:02:52 -0000 Received: from unknown (HELO nile.gnat.com) (205.232.38.5) by sources.redhat.com with SMTP; 9 Dec 2001 23:02:52 -0000 Received: by nile.gnat.com (Postfix, from userid 338) id BB23CF28F3; Sun, 9 Dec 2001 18:02:07 -0500 (EST) From: dewar@gnat.com To: guerby@acm.org, zack@codesourcery.com Subject: Re: ACATS legal status cleared by FSF Cc: dewar@gnat.com, gcc@gcc.gnu.org, kenner@vlsi1.ultra.nyu.edu, mrs@windriver.com Message-Id: <20011209230207.BB23CF28F3@nile.gnat.com> Date: Sun, 09 Dec 2001 15:06:00 -0000 X-SW-Source: 2001-12/txt/msg00448.txt.bz2 <> Well sure, there have been some cases in which a subtle change to the front end caused an error message to be lost, but it is infrequent. And for sure it has not stopped ACT from doing serious error message work, which has always been a focus for us (getting the best possible error messages), but we often often have cases where we make a simple improvement in an error message, and 90% of the work is checking very carefully through the B-test baselines to ensure that the baselines should indeed be updated and nothing has slipped by. It is certainly not possible to provide a comprehensive test suite for use with the gcc tree. That's because the most valuable tests from the test suites in use are the tests in the Compaq and ACT test suites None of the Compaq suite (called "DEC test suite" informally) are available for use, due to licensing restrictions, and the great majority of the ACT tests are not available, since they are proprietary customer code. So what we are doing at the gcc site is to put a subset of high value tests that are worth the effort running. What I am saying is that the C tests meet this criterion, but the B tests don't. Yes occasionally, a change that someone makes to the system will break a B test, so what? Much more often it will be the case if people make changes to the front end that they break one of the tests in the DEC test suite or ACT test suite. In either case, we here at ACT have to figure out how to repair the problem, and I would guess that the B tests issues will play a very minor role. If someone sees a misspelling in an error message, I am happy for them to just fix it, and do not want to inhibit such a change just because of the effort of updating the B tests. Indeed if someone does update the B test baseline, it would be more work for us to check that they had done this update correctly than to do it ourselves, and that careful check would be required in any case. Remember that the B test baseline is an artifact that is maintained not for testing purposes primarily, but for validation purposes, something we are not interested in for the gcc version per se. Now of course there are changes to error messages that require a huge amount of work in all test suites. A good example is an enhancement request we have logged that suggests a nicer treatment of continuation messages, so that a multi-line message is obviously a multi-line messagre rather than separate messages. This is a fairly simple patch that could be done in half an hour, but the consequences to the base lines of all test suites would be ferocious, so that is why this change is still on the list (there are lots of other things on the list). In fact perhaps we can publish at least some of our list of suggested enhancements so that people can try to do some of them :-) I actually think that by far the most valuable addition to the C tests would be to add some of the tests from the ACT test suite that ACT wrote, and that are therefore potentially available. As soon as we have the tree issues fully under control (most notably the docuemntation is still a real issue), we will send some of these tests along. Robert Dewar And P.S. we are certainly not suggesting hiding the B tests, we are just suggesting not worrying about them too much :-)