Re: glibc builder.sourceware.org integration

[Mark Wielaard <mark@klomp.org>]

[-- Attachment #1: Type: text/plain, Size: 2386 bytes --]

Hi all,

On Mon, Jun 27, 2022 at 12:23:55AM +0200, Mark Wielaard wrote:
> The compile-only builders, glibc-debian-i386, glibc-debian-ppc64,
> glibc-fedora-ppc64le, glibc-fedora-s390x, glibc-opensuseleap-x86_64
> and glibc-opensusetw-x86_64 are all green. The builds are pretty quick
> (using ccache) and take a couple of minutes at most. So they run on
> each separate commit to the master branch.

There is now also a glibc-rawhide-x86_64 builder.
https://builder.sourceware.org/buildbot/#/builders/glibc-rawhide-x86_64

> There are currently two builders which do a make && make check.
> glibc-fedora-x86_64 and glibc-fedrawhide-x86_64. The first is a
> container build of fedora-latest (f36). The second is a fedora rawhide
> VM.
> 
> The container builder has a couple of failures for which I just sent
> patches. With those patches the build should turn green. At which
> point we can also use it to detect test regressions.
> 
> Doing a full make && make check in the container takes ~15 to ~20
> minutes and so isn't don't on every individual build, but batches up
> changes till there have been none for 10 minutes. So it won't run
> behind. It should detect a commit that caused a regression within 30
> minutes. All test results (including all .out files) are uploaded into
> the bunsendb for analysis:
> https://builder.sourceware.org/testruns/?has_keyvalue_like_k=testrun.git_describe&has_keyvalue_like_v=%25glibc%25
> 
> Not all hardware is powerful enough to do a full make check on every
> commit. But it would be good to make them do some real tests and not
> just do a compile-only run. Is there a good, small, quick, sub-test
> that could be added?

For now I have added a make subdirs=elf -jncpus check step.  This
seems to have a couple of hundred tests which should be quick enough
to run on all builders. Test results are uploaded to the bunsendb for
comparison between distr/arches. Builders all seem to be green (except
for the full testrun builders of course), but the glibc-fedora-ppc64le builder has one FAIL:
FAIL: elf/tst-ldconfig-bad-aux-cache
https://builder.sourceware.org/buildbot/#/builders?tags=glibc
https://builder.sourceware.org/buildbot/#/builders/127/builds/35

Frank, I cannot find the corresponding bunsen test results for that
build under https://builder.sourceware.org/testruns/ did I make a
mistake in the bunsen upload step?

Cheers,

Mark

[-- Attachment #2: 0001-Add-make-subdirs-elf-jncpus-check-to-glibc_build_fac.patch --]
[-- Type: text/x-diff, Size: 2061 bytes --]

From 61c7d4849db2246f2792085e50e63e9ab3ea4c67 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Fri, 1 Jul 2022 00:07:34 +0200
Subject: [PATCH] Add make subdirs=elf -jncpus check to glibc_build_factory

A full make check takes too long to run on all builder. But a make
subdirs=elf does run at least some tests in a couple of minutes. Also
upload results to bunsendb.

Factory is used on glibc-debian-i386, glibc-fedora-s390x,
glibc-debian-ppc64, glibc-fedora-ppc64le, glibc-opensusetw-x86_64,
glibc-opensuseleap-x86_64 and glibc-rawhide-x86_64.
---
 builder/master.cfg | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 729ec37..e67df26 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -2224,6 +2224,15 @@ glibc_check_step = steps.Test(
         name='make check',
         haltOnFailure=False, flunkOnFailure=True)
 
+glibc_check_build_step = steps.Test(
+        workdir='glibc-build',
+        command=['make',
+                 'subdirs=elf',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'check'],
+        name='make check',
+        haltOnFailure=False, flunkOnFailure=True)
+
 glibc_bunsen_steps = bunsen_logfile_upload_cpio_steps(
         ["config.log", "*.out", "*.test-result", "*.sum"],
         workdir='../glibc-build')
@@ -2237,12 +2246,14 @@ glibc_factory.addStep(glibc_check_step)
 glibc_factory.addSteps(glibc_bunsen_steps)
 
 # A full check is too heavy-weight for some builders
-# Just do a build, no checks
+# Just do a build, only check one subdir (elf).
 glibc_build_factory = util.BuildFactory()
 glibc_build_factory.addStep(glibc_git_step)
 glibc_build_factory.addStep(glibc_rm_step)
 glibc_build_factory.addStep(glibc_configure_step)
 glibc_build_factory.addStep(glibc_make_build_step)
+glibc_build_factory.addStep(glibc_check_build_step)
+glibc_build_factory.addSteps(glibc_bunsen_steps)
 
 glibc_fedrawhide_x86_64_builder = util.BuilderConfig(
         name="glibc-fedrawhide-x86_64",
-- 
2.30.2