buildbot status

[Mark Wielaard <mark@klomp.org>]

Hi,

Since the splitting of the mega debuginfod tests into separate
testcases we have seen some instability of the buildbot. I believe that
is mostly solved now.

I do sometimes see one of the run-debuginfod-federation-
{link,metric,sqlite}.sh tests fail when they are testing propagating of
a client request between servers when the request is for some symlinked
debuginfo. But when I showed that to Frank he immediately said that the
shell logic for that test was bogus, so I am hoping that when he cleans
that up that also gets rid of the false positives. At least we should
now get better error reports when something fails (previously a runtest
that failed, but not a normal shell command, would simply cleanup
instead of spitting out the metrics and server logs).

There also seems to be a spurious fail of the native core backtrace
test on centos-x86_64. I looked at it, but cannot really figure it out.
When it fails we believe to be in __clone inside libc.so.6 but at a pc
that doesn't have cfi, so we cannot unwind. Maybe this is simply a bug
in either the core file writing under centos7 or glibc really is
missing some cfi for clone. But that doesn't explain why it is non-
deterministic.

Finally things took a really long time because most buildbot workers
were doing a full distcheck, which involves a normal build, a normal
make check, generating a dist tar ball, unpacking it again, valgrind,
the gcc undefined   sanitizer, a srcdir == builddir, a srcdir !=
builddir, an install check and another build check. On some builders
this took more than an hour. And for some reason the s390x builder was
so slow that it took 6 hours (!) for each commit.

So I have reconfigured the builders a bit:

- Only elfutils-fedora-x86_64 does a full distcheck
- elfutils-centos-x86_64, elfutils-debian-amd64, elfutils-debian-i386,
  elfutils-fedora-ppc64le and elfutils-fedora-s390x
  configure --enable-valgrind --enable-sanitize-undefined and do a
  make && make check
- elfutils-debian-arm64
  configure --enable-valgrind && make && make check
- elfutils-fedora-ppc64 and elfutils-debian-armhf
  configure without valgrind and sanitizer and simply
  make && make check

Hopefully that gives us buildbot results a bit quicker.

Cheers,

Mark