From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 22705 invoked by alias); 2 Sep 2002 22:42:10 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 22698 invoked from network); 2 Sep 2002 22:42:10 -0000 Received: from unknown (HELO omnigroup.com) (198.151.161.1) by sources.redhat.com with SMTP; 2 Sep 2002 22:42:10 -0000 Received: from seel (seel.omnigroup.com [198.151.161.19]) by omnigroup.com (8.10.2/8.9.1) with ESMTP id g82Mfr311880; Mon, 2 Sep 2002 15:41:53 -0700 (PDT) Date: Mon, 02 Sep 2002 15:42:00 -0000 Subject: Re: [GCC 3.x] Performance testing for QA Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v543) Cc: Peter.Sasi@t-systems.co.hu, aj@suse.de, gcc@gcc.gnu.org To: dewar@gnat.com (Robert Dewar) From: "Timothy J. Wood" In-Reply-To: <20020902214735.1363EF2D4E@nile.gnat.com> Message-Id: <2D929931-BEC5-11D6-9ABD-0003938E4E3C@omnigroup.com> Content-Transfer-Encoding: 7bit X-SW-Source: 2002-09/txt/msg00042.txt.bz2 On Monday, September 2, 2002, at 02:47 PM, Robert Dewar wrote: >> Dear GCC people, >> >> Allow me the humble question to ask you if it would possible to >> employ a simple yet efficient testing method from real life as >> opposed to the SPEC* (artificial) test? > > > Interesting that someone should think that the gcc compiler is an > artificial test (it is one of the SPEC tests :-) This doesn't seem odd at all. A vanishingly small number of people actually build the compiler itself. It makes sense to use code that normal people actually use every day as benchmarks, so things like MP3/Ogg encoders/decoders, DivX encoders/decoders, and possibly rendering glyphs with FreeType could provide another set of useful benchmarks. Clearly a bunch of work would need to be done to isolate the effects of the compiler from the effects of improvements in the algorithms and underlying OS, but more people actually use these applications than the compiler itself. (It's also great to have a bunch of scientific code as benchmarks, but code that Joe Bob might use should make an appearance too). I only reason I think its nice to have the compiler bootstrap as part of the benchmark is to make sure that the compiler is fast. Instead, I think it would be more interesting to see how fast the compiler could build all the other benchmarks -- not how fast it could build itself :) -tim