Re: Buildbot failure in Wildebeest Builder on whole buildset

[Mark Wielaard <mark@klomp.org>]

Hi,

On Wed, Feb 23, 2022 at 12:26:26PM +0100, Mark Wielaard wrote:
> So there were 3 issues:
> 
> - The debian-i386 build had pre-existing failures that weren't
>   reported before.
> - The recent commits introduced some tests which failed, and then
>   some code changes that fixed those tests so they succeeded.
>   IMHO the buildbot was correct to flag those because they are real
>   failures, even though they were fixed 2 commits later. This makes
>   e.g. bisecting issues a bit of a pain. So I would like to see us
>   commit in the "correct order".

So on irc we did briefly discusss that this "correct order" is
slightly opposite to what you do when using Test Driven Development
(TDD). And I agree that test driven development is great. First
writing the testcases, then the code to match the intended
behaviour. But if popssible please first commit the code, then the
testcases (or both together). Or if you wrote the tests first,
committed them locally, then wrote the code to make them PASS as a
separate commit, git rebase the commits to swap (or merge/squash) them
before pushing. git rebase --interactive is really cool BTW.

> - The buildbot test was incorrect when /bin/sh wasn't bash (as on
>   the debian builders, where it is dash).
>   test $? == 1 should have been $? -eq 1
>   Fixed.

So everything, except debian-i386, is green again:
https://builder.wildebeest.org/buildbot/#/builders?tags=gccrust

The buildbot will only sent email when another builder switches from
green to red. So it will no longer sent email about the already red
debian-i386 builder.

So currently we have a BuildSetStatusGenerator with mode=('problem',)
where problem is "Include a build which failed when the previous build
has passed."

https://docs.buildbot.net/current/manual/configuration/report_generators/buildset.html

But we could use a different setting, maybe 'failing' to always sent
mail when a build fails (but currently that means all builds, because
debian-i386 fails). Or 'change' to report on any builder that changed
status.

Cheers,

Mark