From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18289 invoked by alias); 23 Sep 2014 14:59:46 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 18259 invoked by uid 89); 23 Sep 2014 14:59:43 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00,RP_MATCHES_RCVD,SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Received: from mx1.redhat.com (HELO mx1.redhat.com) (209.132.183.28) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Tue, 23 Sep 2014 14:59:42 +0000 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s8NExdqU013969 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 23 Sep 2014 10:59:39 -0400 Received: from [127.0.0.1] (ovpn01.gateway.prod.ext.ams2.redhat.com [10.39.146.11]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s8NExbOr001155; Tue, 23 Sep 2014 10:59:38 -0400 Message-ID: <54218AD9.5050801@redhat.com> Date: Tue, 23 Sep 2014 14:59:00 -0000 From: Pedro Alves User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.0 MIME-Version: 1.0 To: Doug Evans , Jan Kratochvil CC: "gdb-patches@sourceware.org" Subject: Re: automated testing comment [Re: time to workaround libc/13097 in fsf gdb?] References: <5411CFAE.7040805@redhat.com> <20140912115452.GA5626@host2.jankratochvil.net> <5412E3AC.80203@redhat.com> <20140912123320.GA8704@host2.jankratochvil.net> <5412EB1F.40309@redhat.com> <20140917201049.GA22880@host2.jankratochvil.net> <541C3FCF.4000400@redhat.com> <541C409E.6010408@redhat.com> <20140920213033.GA6255@host2.jankratochvil.net> <541F2311.1040404@redhat.com> <20140923105855.GA10164@host2.jankratochvil.net> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-SW-Source: 2014-09/txt/msg00695.txt.bz2 On 09/23/2014 03:48 PM, Doug Evans wrote: > On Tue, Sep 23, 2014 at 3:58 AM, Jan Kratochvil > wrote: >> And sure deploying automated testing with the current GDB testsuite as is >> would not work now automatically as the testsuite has fuzzy results. Although >> at least running new testcases (from the patch under review) would work which >> would be sufficient in this case (but not in other cases - regression cases). > > Sometimes tests are flaky and one isn't aware of it, and not > everything gets caught by patch review. > > IWBN if automated testing reran everything failing test N times (10?) > and reported those results as something like "fail" (always failed) or > "flaky" (inconsistent). Indeed. I think that would be doable as part of gdb's own testsuite machinery even, which would make it easily available to every one. Thanks, Pedro Alves