From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Stump To: dewar@gnat.com, law@redhat.com Cc: gcc@gcc.gnu.org, kenner@vlsi1.ultra.nyu.edu, rth@cygnus.com Subject: Re: Why not gnat Ada in gcc? Date: Wed, 01 Nov 2000 21:28:00 -0000 Message-id: <200011020528.VAA27440@kankakee.wrs.com> X-SW-Source: 2000-11/msg00108.html > To: dewar@gnat.com, law@redhat.com, mrs@windriver.com > Cc: gcc@gcc.gnu.org, kenner@vlsi1.ultra.nyu.edu, rms@gnu.org, rth@cygnus.com > Date: Wed, 1 Nov 2000 23:39:41 -0500 (EST) > From: dewar@gnat.com (Robert Dewar) > Presumably only Cygnus/Redhat can measure this, since only they have > the internal test suite that would give the data. :-) Nope, on a couple of counts. First, my SOP was to take the time to sanitize the testcase, and fold it into the testsuite. Nearly all of my testcases made the transition from company confidential to public testcase. I don't know exactly how many were lost in the transition to publicness, but my guess is not that many for the g++ testsuite. (<3%?) For the gcc testsuite, quite a bit of the content originated as a public testsuite (c-torture). Second, it can still be measured against the public testsuite, even though some internal testsuites might give slightly better results. Third, the there is a testresults web page that has the last information on it ( http://gcc.gnu.org/testresults/ ), you merely have to mine it out. And last, _you_ have a testing framework that can tell you about the quality of gcc. You merely have to ask it. (Yes, I know how hard this is. I have my own framework as well, complete with ~88 boards), and I don't yet ask it about public sources.) > Well I think in the case of Ada it will be a huge help to make the > official validation (ACATS) tests available and runnable, and that > is one of our projects (these are openly available tests, but > setting them up to run is nowhere near trivial). Should be. For gcc/g++/g77/objc/libstdc++/libio, the goal is make check. That mostly works. It is mostly easy to run. If you engineer them to work from make check, I think they can be made to be trivial. For example, the Fortran testsuite is so trivial to run, that almost no one who runs tests doesn't also run it, even though they aren't typically interested in Fortran. > Unfortunately the more significant test suites (the ACT internal > test suite, and the Digital test suite) are full of highly protected > proprietary code, and cannot be made available for external use. Can you report the results publicly? If so, maybe you can give warm fuzzies, without saying exactly what went wrong. > It will be interesting to see how stable things stay ... It'll be 6 months to condition people to understand the results you feed them, and to condition them to stop breaking it. After the ramp up, people should mostly keep it to the quality you expect. If they don't, let's talk about it again, then.