Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp

[Mark Wielaard <mark@klomp.org>]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

Hi Luis,

On Thu, Jun 09, 2022 at 10:37:58AM +0100, Luis Machado wrote:
> > With the removal of the selftest.exp there are only two unittest.exp
> > tests (one for gdb and one for gdbserver). This does seem very
> > minimal. Could someone suggest stable, known good passing tests that
> > should be added because when they do start failing it really is a
> > regression that should be flagged?
> > 
> > Note that the intention is to also allow a trybot that runs the same
> > set of CI tests so it can be used as pre-commit check.
> [...] 
> I always use gdb.base/break.exp as a good smoke test. If that one fails, then things
> are really broken.
> 
> I think gdb.base/break*.exp should make a good smoke test list. We just need to exclude
> gdb.base/break-interp.exp, which is problematic on some targets.

It never is just easy is it? :) You are right, I saw break-interp.exp
fail...  I tried to come up with a regexp but gave up given that it
has to go throug python first and then we don't know whether the
worker uses bash as /bin/sh so I just added them all (exclusing
break-interp.exp) as a list.

> If you also want to exercise gdbserver, then run with native-gdbserver and native-extended-gdbserver
> boards. For example:
> 
> make check-gdb TESTS="gdb.base/break*.exp" RUNTESTFLAGS="--target_board=native-gdbserver" -j$(nproc).

OK, so I added two more steps, one with
RUNTESTFLAGS="--target_board=native-gdbserver" and one with
RUNTESTFLAGS="--target_board=native-extended-gdbserver"

So now each CI run does make check-gdb three times.  Frank, you might
want to look at the bunsen upload code, I didn't know how to adapt it,
so it only records the logs of the first make check-gdb step.

Thanks,

Mark

[-- Attachment #2: 0001-gdb-Add-break-exp-TESTS-and-gdb_check_native_gdbserv.patch --]
[-- Type: text/x-diff, Size: 3899 bytes --]

From 9ef7c1e94e9a4f30acc5d8b320d4f7b44ff4cfef Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Fri, 10 Jun 2022 00:51:59 +0200
Subject: [PATCH] gdb: Add break*exp TESTS and
 gdb_check_native_gdbserver[_extended]_step

Add gdb.base/break*exp, but not break-interp.exp, as gdb_test_exp
list.  Add two more make gdb-check steps with RUNTESTFLAGS set to
--target_board=native-gdbserver and
--target_board=native-extended-gdbserver.
---
 builder/master.cfg | 48 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 647acd0..f6895b5 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -1447,19 +1447,59 @@ gdb_make_step = steps.Compile(
         name='make',
         haltOnFailure=True)
 # Only a small subset of tests that are fast and known to PASS.
+gdb_test_exp = ("TESTS= "
+                "gdb.base/break-always.exp "
+                "gdb.base/break-caller-line.exp "
+                "gdb.base/break-entry.exp "
+                "gdb.base/break.exp "
+                "gdb.base/break-fun-addr.exp "
+                "gdb.base/break-idempotent.exp "
+                "gdb.base/break-include.exp "
+                "gdb.base/break-inline.exp "
+                "gdb.base/break-main-file-remove-fail.exp "
+                "gdb.base/break-on-linker-gcd-function.exp "
+                "gdb.base/breakpoint-in-ro-region.exp "
+                "gdb.base/breakpoint-shadow.exp "
+                "gdb.base/break-probes.exp "
+                "gdb.base/break-unload-file.exp "
+                "gdb.gdb/unittest.exp "
+                "gdb.server/unittest.exp ")
+
 gdb_check_step = steps.Test(
         workdir='gdb-build',
         command=['make',
                  util.Interpolate('-j%(prop:ncpus)s'),
                  'check-gdb',
-                 ("TESTS= "
-                  "gdb.gdb/unittest.exp "
-                  "gdb.server/unittest.exp ")],
+                 gdb_test_exp],
         name='make check-gdb',
         logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum",
                    "gdb.log": "gdb/testsuite/gdb.log" },
         haltOnFailure=False, flunkOnFailure=True)
 
+gdb_check_native_gdbserver_step = steps.Test(
+        workdir='gdb-build',
+        command=['make',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'check-gdb',
+                 gdb_test_exp,
+                 'RUNTESTFLAGS="--target_board=native-gdbserver"'],
+        name='make check-gdb native-gdbserver',
+        logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum",
+                   "gdb.log": "gdb/testsuite/gdb.log" },
+        haltOnFailure=False, flunkOnFailure=True)
+
+gdb_check_native_extended_gdbserver_step = steps.Test(
+        workdir='gdb-build',
+        command=['make',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'check-gdb',
+                 gdb_test_exp,
+                 'RUNTESTFLAGS="--target_board=native-extended-gdbserver"'],
+        name='make check-gdb native-extended-gdbserver',
+        logfiles={ "gdb.sum": "gdb/testsuite/gdb.sum",
+                   "gdb.log": "gdb/testsuite/gdb.log" },
+        haltOnFailure=False, flunkOnFailure=True)
+
 gdb_factory = util.BuildFactory()
 gdb_factory.addStep(gdb_git_step)
 gdb_factory.addStep(gdb_rm_step)
@@ -1468,6 +1508,8 @@ gdb_factory.addStep(gdb_make_step)
 gdb_factory.addStep(gdb_check_step)
 gdb_factory.addSteps(bunsen_logfile_upload_steps([ # only a few tests being run but still
         (["../gdb-build/gdb/testsuite/gdb.sum", "../gdb-build/gdb/testsuite/gdb.log", "../gdb-build/gdb/config.log"],"gdb")]))
+gdb_factory.addStep(gdb_check_native_gdbserver_step)
+gdb_factory.addStep(gdb_check_native_extended_gdbserver_step)
 
 # Used for armhf only because of
 # https://sourceware.org/bugzilla/show_bug.cgi?id=28561
-- 
2.30.2