From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 6DC53382B5F0; Thu, 9 Jun 2022 23:09:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6DC53382B5F0 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from reform (deer0x09.wildebeest.org [172.31.17.139]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id D286B302BBEB; Fri, 10 Jun 2022 01:09:19 +0200 (CEST) Received: by reform (Postfix, from userid 1000) id ABECB2E83DD1; Fri, 10 Jun 2022 01:09:19 +0200 (CEST) Date: Fri, 10 Jun 2022 01:09:19 +0200 From: Mark Wielaard To: Luis Machado Cc: buildbot@sourceware.org, gdb@sourceware.org Subject: Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp Message-ID: References: <20220608185138.675964-1-mark@klomp.org> <2281be8e-8920-7ba5-9e96-ffd713462b9c@arm.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="6PX80NTCvLq7OMwH" Content-Disposition: inline In-Reply-To: <2281be8e-8920-7ba5-9e96-ffd713462b9c@arm.com> X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jun 2022 23:09:25 -0000 --6PX80NTCvLq7OMwH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Luis, On Thu, Jun 09, 2022 at 10:37:58AM +0100, Luis Machado wrote: > > With the removal of the selftest.exp there are only two unittest.exp > > tests (one for gdb and one for gdbserver). This does seem very > > minimal. Could someone suggest stable, known good passing tests that > > should be added because when they do start failing it really is a > > regression that should be flagged? > > > > Note that the intention is to also allow a trybot that runs the same > > set of CI tests so it can be used as pre-commit check. > [...] > I always use gdb.base/break.exp as a good smoke test. If that one fails, then things > are really broken. > > I think gdb.base/break*.exp should make a good smoke test list. We just need to exclude > gdb.base/break-interp.exp, which is problematic on some targets. It never is just easy is it? :) You are right, I saw break-interp.exp fail... I tried to come up with a regexp but gave up given that it has to go throug python first and then we don't know whether the worker uses bash as /bin/sh so I just added them all (exclusing break-interp.exp) as a list. > If you also want to exercise gdbserver, then run with native-gdbserver and native-extended-gdbserver > boards. For example: > > make check-gdb TESTS="gdb.base/break*.exp" RUNTESTFLAGS="--target_board=native-gdbserver" -j$(nproc). OK, so I added two more steps, one with RUNTESTFLAGS="--target_board=native-gdbserver" and one with RUNTESTFLAGS="--target_board=native-extended-gdbserver" So now each CI run does make check-gdb three times. Frank, you might want to look at the bunsen upload code, I didn't know how to adapt it, so it only records the logs of the first make check-gdb step. Thanks, Mark --6PX80NTCvLq7OMwH Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-gdb-Add-break-exp-TESTS-and-gdb_check_native_gdbserv.patch" >From 9ef7c1e94e9a4f30acc5d8b320d4f7b44ff4cfef Mon Sep 17 00:00:00 2001 From: Mark Wielaard Date: Fri, 10 Jun 2022 00:51:59 +0200 Subject: [PATCH] gdb: Add break*exp TESTS and gdb_check_native_gdbserver[_extended]_step Add gdb.base/break*exp, but not break-interp.exp, as gdb_test_exp list. Add two more make gdb-check steps with RUNTESTFLAGS set to --target_board=native-gdbserver and --target_board=native-extended-gdbserver. --- builder/master.cfg | 48 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/builder/master.cfg b/builder/master.cfg index 647acd0..f6895b5 100644 --- a/builder/master.cfg +++ b/builder/master.cfg @@ -1447,19 +1447,59 @@ gdb_make_step = steps.Compile( name='make', haltOnFailure=True) # Only a small subset of tests that are fast and known to PASS. +gdb_test_exp = ("TESTS= " + "gdb.base/break-always.exp " + "gdb.base/break-caller-line.exp " + "gdb.base/break-entry.exp " + "gdb.base/break.exp " + "gdb.base/break-fun-addr.exp " + "gdb.base/break-idempotent.exp " + "gdb.base/break-include.exp " + "gdb.base/break-inline.exp " + "gdb.base/break-main-file-remove-fail.exp " + "gdb.base/break-on-linker-gcd-function.exp " + "gdb.base/breakpoint-in-ro-region.exp " + "gdb.base/breakpoint-shadow.exp " + "gdb.base/break-probes.exp " + "gdb.base/break-unload-file.exp " + "gdb.gdb/unittest.exp " + "gdb.server/unittest.exp ") + gdb_check_step = steps.Test( workdir='gdb-build', command=['make', util.Interpolate('-j%(prop:ncpus)s'), 'check-gdb', - ("TESTS= " - "gdb.gdb/unittest.exp " - "gdb.server/unittest.exp ")], + gdb_test_exp], name='make check-gdb', logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum", "gdb.log": "gdb/testsuite/gdb.log" }, haltOnFailure=False, flunkOnFailure=True) +gdb_check_native_gdbserver_step = steps.Test( + workdir='gdb-build', + command=['make', + util.Interpolate('-j%(prop:ncpus)s'), + 'check-gdb', + gdb_test_exp, + 'RUNTESTFLAGS="--target_board=native-gdbserver"'], + name='make check-gdb native-gdbserver', + logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum", + "gdb.log": "gdb/testsuite/gdb.log" }, + haltOnFailure=False, flunkOnFailure=True) + +gdb_check_native_extended_gdbserver_step = steps.Test( + workdir='gdb-build', + command=['make', + util.Interpolate('-j%(prop:ncpus)s'), + 'check-gdb', + gdb_test_exp, + 'RUNTESTFLAGS="--target_board=native-extended-gdbserver"'], + name='make check-gdb native-extended-gdbserver', + logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum", + "gdb.log": "gdb/testsuite/gdb.log" }, + haltOnFailure=False, flunkOnFailure=True) + gdb_factory = util.BuildFactory() gdb_factory.addStep(gdb_git_step) gdb_factory.addStep(gdb_rm_step) @@ -1468,6 +1508,8 @@ gdb_factory.addStep(gdb_make_step) gdb_factory.addStep(gdb_check_step) gdb_factory.addSteps(bunsen_logfile_upload_steps([ # only a few tests being run but still (["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")])) +gdb_factory.addStep(gdb_check_native_gdbserver_step) +gdb_factory.addStep(gdb_check_native_extended_gdbserver_step) # Used for armhf only because of # https://sourceware.org/bugzilla/show_bug.cgi?id=28561 -- 2.30.2 --6PX80NTCvLq7OMwH--