From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from elastic.org (elastic.org [96.126.110.187]) by sourceware.org (Postfix) with ESMTPS id 229653858D1E; Tue, 26 Apr 2022 09:40:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 229653858D1E Received: from vpn-home.elastic.org ([10.0.0.2] helo=elastic.org) by elastic.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1njHgI-0001NS-2E; Tue, 26 Apr 2022 09:40:26 +0000 Received: from fche by elastic.org with local (Exim 4.94.2) (envelope-from ) id 1njHgH-000FEL-Iy; Tue, 26 Apr 2022 05:40:25 -0400 Date: Tue, 26 Apr 2022 05:40:25 -0400 From: "Frank Ch. Eigler" To: Luis Machado Cc: Mark Wielaard , Overseers mailing list , binutils@sourceware.org, "gdb@sourceware.org" Subject: Re: Adding binutils to the GNU Toolchain buildbot on sourceware Message-ID: References: <5c1f217a-109c-2973-6c69-abf412133dee@arm.com> <524b04b7-a78c-7aae-4605-b40f61e6830c@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <524b04b7-a78c-7aae-4605-b40f61e6830c@arm.com> X-Spam-Status: No, score=-101.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, SPF_HELO_PASS, SPF_PASS, TXREP, USER_IN_WELCOMELIST, USER_IN_WHITELIST autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2022 09:40:29 -0000 Hi - > I agree with the quick build idea. That's why I suggested only making sure > gdb/gdbserver/sim builds properly. That's reasonably reliable. OK, so looking for build-breaking regressions. > Unfortunately gdb's testsuite is not too reliable. It's been improved over > the years, but still gives quite a bit of non-deterministic results based on > distro version/compiler version etc. So I'd leave those out in favor of just > making sure things build properly. This problem is why we're building out a gadget called bunsen, which is a tool to absorb histories of testsuites, and draw statistical conclusions. Still early days, but noisy testsuites are not a problem. > > If the is a make check-something that can be executed quickly, < 5 > > minutes runtime, and that should always be green please include > > it. But please exclude anything that takes too long, isn't known > > all-green or contains flaky tests. > > Some tests should always work, no matter what. We could hand-pick some tests > from the GDB testsuite that we consider critical. [...] Bunsen should help -identify- such tests. - FChE