From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <gcc-return-170127-listarch-gcc=gcc.gnu.org@gcc.gnu.org>
Received: (qmail 19757 invoked by alias); 8 Sep 2011 13:24:05 -0000
Received: (qmail 19745 invoked by uid 22791); 8 Sep 2011 13:24:04 -0000
X-SWARE-Spam-Status: No, hits=-2.3 required=5.0	tests=AWL,BAYES_00,RP_MATCHES_RCVD
X-Spam-Check-By: sourceware.org
Received: from cam-admin0.cambridge.arm.com (HELO cam-admin0.cambridge.arm.com) (217.140.96.50)    by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 08 Sep 2011 13:23:47 +0000
Received: from geoffrey.Emea.Arm.com (geoffrey.emea.arm.com [10.1.255.46])	by cam-admin0.cambridge.arm.com (8.12.6/8.12.6) with ESMTP id p88DNG11010880;	Thu, 8 Sep 2011 14:23:16 +0100 (BST)
Received: from cam-owa2.Emea.Arm.com ([10.1.105.18]) by geoffrey.Emea.Arm.com with Microsoft SMTPSVC(6.0.3790.3959);	 Thu, 8 Sep 2011 14:23:44 +0100
Received: from [10.1.67.65] ([10.1.255.212]) by cam-owa2.Emea.Arm.com with Microsoft SMTPSVC(6.0.3790.3959);	 Thu, 8 Sep 2011 14:23:43 +0100
Message-ID: <4E68C1DF.805@arm.com>
Date: Thu, 08 Sep 2011 13:24:00 -0000
From: Richard Earnshaw <rearnsha@arm.com>
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2
MIME-Version: 1.0
To: Diego Novillo <dnovillo@google.com>
CC: Richard Guenther <richard.guenther@gmail.com>,   "gcc@gcc.gnu.org" <gcc@gcc.gnu.org>
Subject: Re: RFC: Improving support for known testsuite failures
References: <20110907152813.GA28540@google.com> <CAFiYyc2kziNeCaeuXJaO0dOvvHho45GOP-sH7BqV47Q1MxJJiA@mail.gmail.com> <CAD_=9DQ7CaJc8HN9emsN7-7xiQBQFdwrbJ4mKevSWkHakBgrPw@mail.gmail.com> <CAFiYyc3wDoCi4MM1Vdan=Ztv+EPgA4NTyvAYcUfcDedooc-4SA@mail.gmail.com> <CAD_=9DR9tMJGwoErk-cB-byB84fv0z6DFpFir1p6k_iuQnF=Zg@mail.gmail.com>
In-Reply-To: <CAD_=9DR9tMJGwoErk-cB-byB84fv0z6DFpFir1p6k_iuQnF=Zg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm
Precedence: bulk
List-Id: <gcc.gcc.gnu.org>
List-Archive: <http://gcc.gnu.org/ml/gcc/>
List-Post: <mailto:gcc@gcc.gnu.org>
List-Help: <http://gcc.gnu.org/ml/>
Sender: gcc-owner@gcc.gnu.org
X-SW-Source: 2011-09/txt/msg00075.txt.bz2

On 08/09/11 12:33, Diego Novillo wrote:
> On Thu, Sep 8, 2011 at 07:16, Richard Guenther
> <richard.guenther@gmail.com> wrote:
> 
>> Well, you'd need to maintain a list of known XPASS/FAILs anyway.
> 
> Yes, of course.  That's the manifest of things you expect to be broken.
> 

And that's only going to work if all the test names are unique.  I
currently see quite a few tests that appear in my log as both PASS and
FAIL in a single run.  For example:

FAIL: gcc.dg/torture/stackalign/non-local-goto-2.c  -Os  execution test
PASS: gcc.dg/torture/stackalign/non-local-goto-2.c  -Os  execution test

FAIL: gcc.dg/torture/stackalign/setjmp-3.c  -Os  execution test
PASS: gcc.dg/torture/stackalign/setjmp-3.c  -Os  execution test

R.