public inbox for buildbot@sourceware.org
 help / color / mirror / Atom feed
From: Luis Machado <luis.machado@arm.com>
To: Mark Wielaard <mark@klomp.org>, Carl Love <cel@us.ibm.com>,
	Will Schmidt <will_schmidt@vnet.ibm.com>
Cc: gdb@sourceware.org, buildbot@sourceware.org
Subject: Re: [builder] gdb_check_step: remove gdb.gdb/selftest.exp
Date: Fri, 10 Jun 2022 11:58:33 +0100	[thread overview]
Message-ID: <85fa6e2d-caf5-8afe-a7ec-40cc62ff347a@arm.com> (raw)
In-Reply-To: <8bfa7ec63758afc45bde1f10b0e4ab91e21e9d06.camel@klomp.org>

On 6/10/22 11:50, Mark Wielaard wrote:
> Hi,
> 
> On Fri, 2022-06-10 at 01:21 +0200, Mark Wielaard wrote:
>> On Fri, Jun 10, 2022 at 01:09:19AM +0200, Mark Wielaard wrote:
>>> On Thu, Jun 09, 2022 at 10:37:58AM +0100, Luis Machado wrote:
>>>> I always use gdb.base/break.exp as a good smoke test. If that one
>>>> fails, then things
>>>> are really broken.
>>>>
>>>> I think gdb.base/break*.exp should make a good smoke test list.
>>>> We just need to exclude
>>>> gdb.base/break-interp.exp, which is problematic on some targets.
>>>
>>> It never is just easy is it? :) You are right, I saw break-
>>> interp.exp
>>> fail...  I tried to come up with a regexp but gave up given that it
>>> has to go throug python first and then we don't know whether the
>>> worker uses bash as /bin/sh so I just added them all (exclusing
>>> break-interp.exp) as a list.
>>
>> Sigh, sorry, looks like gdb.base/break-unload-file.exp also sometimes
>> fails.
>> I have removed from the list. Hopefully the remaining list does
>> actually pass.
> 
> And it didn't :{
> 

Yeah. As expected, the GDB testsuite is a bit delicate when you start dealing with
multiple architectures and modes. But I think this is good progress already.

> Both debian-ppc64 and fedora-ppc64le failed (UNRESOLVED)
> gdb.base/break-idempotent.exp under both native-gdbserver and native-
> extended-gdbserver
> https://builder.sourceware.org/buildbot/#builders/76/builds/446
> https://builder.sourceware.org/buildbot/#builders/85/builds/294

Those might be genuine issues. I'm cc-ing Carl and Will so they can chime in.

> 
> So I have removed that one too from the list.
> The CI test list now looks like:
> 
> # Only a small subset of tests that are fast and known to PASS.
> gdb_test_exp = ("TESTS= "
>                  "gdb.base/break-always.exp "
>                  "gdb.base/break-caller-line.exp "
>                  "gdb.base/break-entry.exp "
>                  "gdb.base/break.exp "
>                  "gdb.base/break-fun-addr.exp "
>                  "gdb.base/break-include.exp "
>                  "gdb.base/break-inline.exp "
>                  "gdb.base/break-main-file-remove-fail.exp "
>                  "gdb.base/break-on-linker-gcd-function.exp "
>                  "gdb.base/breakpoint-in-ro-region.exp "
>                  "gdb.base/breakpoint-shadow.exp "
>                  "gdb.base/break-probes.exp "
>                  "gdb.gdb/unittest.exp "
>                  "gdb.server/unittest.exp ")
> 
> Which will be run three times with make gdb-check, once without a
> target_board, once with native-gdbserver and once with native-extended-
> gdbserver on centos-x86_64, fedora-x86_64, debian-armhf, debian-arm64,
> fedora-s390x, debian-ppc64, fedora-ppc64le, opensusetw-x86_64,
> opensuseleap-x86_64 (debian-armhf only does a build, no make gdb-check
> because of https://sourceware.org/bugzilla/show_bug.cgi?id=28561)
> I'll add a debian-i386 builder so there is more 32bit coverage.
> 
> All are green now (with the latest change to remove break-idempotent)
> https://builder.sourceware.org/buildbot/#/builders?tags=gdb

Looks nice! :-)

> 
> Question is if this is a good list, does it need more tests? And should
> it maybe be maintained in the binutils-gdb repo instead of in the
> builder repo?
> 
> For example we could have a make check-gdb-ci target which does what
> the buildbot would do (and then the buildbot could just call that).

Having a new check-gdb target that only does minimal smoke tests should
be easy to do. Once we determine a subset of critical tests, we can put something
together if folks think it is a good idea. I like it, as it make it easier to deal with
stability issues of GDB's testsuite across different targets.

> 
> Cheers,
> 
> Mark


  reply	other threads:[~2022-06-10 10:58 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-08 18:51 Mark Wielaard
2022-06-09  9:37 ` Luis Machado
2022-06-09 23:09   ` Mark Wielaard
2022-06-09 23:21     ` Mark Wielaard
2022-06-10 10:50       ` Mark Wielaard
2022-06-10 10:58         ` Luis Machado [this message]
2022-06-10 15:17           ` will schmidt
2022-06-10 15:54           ` Carl Love
2022-06-10 19:11             ` Mark Wielaard
2022-06-10 20:11               ` Carl Love
2022-06-10 22:23                 ` Frank Ch. Eigler
2022-06-29 15:58               ` Carl Love
2022-06-29 22:42                 ` Mark Wielaard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=85fa6e2d-caf5-8afe-a7ec-40cc62ff347a@arm.com \
    --to=luis.machado@arm.com \
    --cc=buildbot@sourceware.org \
    --cc=cel@us.ibm.com \
    --cc=gdb@sourceware.org \
    --cc=mark@klomp.org \
    --cc=will_schmidt@vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).