Re: buildbot users try branch builders

[Mark Wielaard <mark@klomp.org>]

Hi Szabolcs,

On Tue, 2022-06-21 at 11:30 +0100, Szabolcs Nagy wrote:
> The 06/20/2022 12:48, Mark Wielaard wrote:
> > The armhf and arm64 builders have left out for now since they are a
> > bit too slow. We'll hopefully get a 8 core Ampare machine from the
> > Oregon State University this week so we can also enable those arm
> > builders.
> 
> what is the requirement for the machine? i have an aarch64 machine
> for public gnu tools ci, and i can set up build slaves there if
> there are instructions how to do it.

The requirements are not that big, in theory that tinker-board and
odroid-board are fine as buildbot-workers. But only for smaller builds.
To actually run some of the larger builds it should have at least 8GB
and 4cpus. More is better of course. Especially for the larger projects
if you want to do full test runs and not just "quick" CI builds.

Disk again depends on how many builds/projects you want to support. It
is really beneficial to have a bigger ~5GB ccache for repeat builds.
binutils takes ~2G (plus another ~2G for the try builder), similar for
gdb and libabigail, gccrust is ~4.5G, valgrind ~600MB, elfutils ~300MB,
bzip2, dwz are just a couple of MB. If we want to add glibc and enable
try builders for more projects I estimate 30GB to 50GB. The gccrust
builder does just a --disable-bootstrap build, so a full gcc builder
would add another 10G at least (maybe 2 or 3 times if we want to add
try builders). If you like to do container builds then those are almost
full distros, so add at least another 4GB per container/distro type (we
currently support 4). So you can get away with ~15GB, but might need up
to ~300GB if you want to do more and larger builds across multiple
distros.

There is a little documentation on adding workers (they used to be
called slaves, but buildbot renamed them some time ago) in the README:
https://sourceware.org/git/?p=builder.git;a=blob;f=README;hb=HEAD#l111
But that is mostly from the side of the buildbot master.

The easiest setup is one with a dedicated distro install with a
buildbot-worker: 
https://docs.buildbot.net/current/manual/installation/worker.html

All you really need to do is make sure a full distro is installed with
all development packages. You can take a peek at one of the container
files to see which: 
https://sourceware.org/git/?p=builder.git;a=tree;f=builder/containers;hb=HEAD
Then create a user which can run the buildbot-worker.
Please contact the buildbot@sourceware.org mailinglist for connection
info.

Another way is making sure you have docker installed (podman sadly
seems incompatible at the moment) and a builder user that is in the
docker group which can be accessed through ssh to create full container
images. But that is a bit more work (and currently depends on some
specific user uid to make in/out container sharing easier). Please
again contact the buildbot@sourceware.org mailinglist for more info.

There are several people on the list who have setup buildbot-workers
who can help. And then we can coordinate which projects/builders you
want to support. https://sourceware.org/mailman/listinfo/buildbot

Cheers,

Mark