From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 23745 invoked by alias); 20 Dec 2002 17:32:31 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 23690 invoked from network); 20 Dec 2002 17:32:20 -0000 Received: from unknown (HELO yeah-baby.shagadelic.org) (208.176.2.162) by 209.249.29.67 with SMTP; 20 Dec 2002 17:32:20 -0000 Received: by yeah-baby.shagadelic.org (Postfix, from userid 2158) id 851387B12; Fri, 20 Dec 2002 09:32:08 -0800 (PST) Date: Fri, 20 Dec 2002 12:02:00 -0000 From: Jason R Thorpe To: Volker Reichelt Cc: jakub@redhat.com, gcc@gcc.gnu.org, bangerth@ticam.utexas.edu, janis187@us.ibm.com, ehrhardt@mathematik.uni-ulm.de Subject: Re: A testcase library Message-ID: <20021220173208.GC15004@yeah-baby.shagadelic.org> Mail-Followup-To: Jason R Thorpe , Volker Reichelt , jakub@redhat.com, gcc@gcc.gnu.org, bangerth@ticam.utexas.edu, janis187@us.ibm.com, ehrhardt@mathematik.uni-ulm.de References: <20021220125202.B1310@sunsite.ms.mff.cuni.cz> <200212201313.OAA72149@numa6.igpm.rwth-aachen.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200212201313.OAA72149@numa6.igpm.rwth-aachen.de> User-Agent: Mutt/1.4i Organization: Wasabi Systems, Inc. X-SW-Source: 2002-12/txt/msg01271.txt.bz2 On Fri, Dec 20, 2002 at 02:13:23PM +0100, Volker Reichelt wrote: > ================================================================================= > On Fri, Nov 29, 2002 at 09:47:24AM +0000, Nathan Sidwell wrote: > > Craig Rodrigues wrote: > > > A few questions: > > > (1) Can I commit my testcase to the testsuite, since it is definitely > > > a regression from gcc 3.1 and gcc 3.2? > > No, we only commit test cases when the bug's fixed. The rationale being you > > don't want to add a new failing test case -- that'd confuse people. Marking > > it XFAIL isn't really the right solution, as then when you fix the bug, you > > don't know whether fixing the XFAIL was accidental, and end up adding a new > > test case anyway. Also, in fixing the bug, you (well I at least), can > > construct a better test case having undersytood the failure mode. > ================================================================================= I have to say that I disagree with this policy, esp. if the testcase shows a regression from a previous GCC release. Gaps in the testsuite mean it will take longer for bugs to be fixed, and give people a false sense of confidence in the quality of the compiler when they see results that show few, if any, failures. One example is a PR I filed recently, with test case that causes the compiler to loop forever. The rationale I got for not adding the testcase was "it would make everyone's bootstrap/test cycle 5 minutes longer". My initial thought was "...and that will motivate someone with the knowledge to do so to fix it." (The test case in question shows a regression from a previous release of GCC.) ...the principle is not too different than a suggestion someone made a while ago about making GCC maintainers do bootstraps on 100MHz Pentiums (maybe GCC wouldn't slow down so much over time if they did :-) -- -- Jason R. Thorpe