From cd13ea399e2b24d607fdcaa26b5405ee89b95468 Mon Sep 17 00:00:00 2001 From: Mark Wielaard Date: Wed, 25 May 2022 01:52:36 +0200 Subject: [PATCH] Add documentation on adding a dedicated or container worker --- README | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- SETUP | 10 ++++++++ 2 files changed, 84 insertions(+), 1 deletion(-) diff --git a/README b/README index 72779d1..2574424 100644 --- a/README +++ b/README @@ -72,6 +72,19 @@ htdocs: document root of https://builder.sourceware.org/ factory=project_factory) c['builders'].append(project__builder) + The above uses a dedicated distro-arch worker. If you want to use a + container image worker then you need to add a container-file + property to the builder and use one or more of the container image + workers (currently there are only 3 x86_64 container image workers + bb1, bb2 and bb3). + + properties={'container-file': + readContainerFile('fedora-latest')}, + workernames=["bb1", "bb2", "bb3"], + + Look under builder/containers/ContainerFile-* to the container image + names you can use. + There are a couple of common steps that might be useful to reuse in you factory. A factory using common steps look something like: @@ -95,7 +108,67 @@ htdocs: document root of https://builder.sourceware.org/ - How to add a new worker - ... + + There are two kinds of workers. Dedicated distr-arch workers + (worker.Worker) and container image workers + (worker.DockerLatentWorker). The first has a whole distro installed + and runs a dedicated buildbot-worker which connects to + builder.sourceware.org to know what to build. The second runs a + (probably minimal) install with docker (or podman, but there are + still some incompatabilities) to which builder.sourceware.org + connects when there is a build providing a container file inside + which the build should be run (the container file will actually + contain a buildbot-worker which connects back to + builder.sourceware.org). + + Both kind of workers need a little setup on the server. The + dedicated workers need a worker name and password, the container + image builders need a ssh setup (see SETUP). + + The dedicated workers need the have all the developer packages + installed (look at one of the ContainerFiles to see which + ones). Since most builds will be almost the same with just a few + file changes, it is a good idea to have ccache installed and make + sure the ccache wrapper binaries are in the PATH before running the + buildbot-worker. + + The container image builders should have a dedicated user (builder, + uid 1001) and a "shared" directory in which the container images can + put shared data (worker status, git checkouts, ccache). They should + also periodically clean up the images to make sure they get + regenerated with new packages. Run docker system prune -a -f once a + week. + +- How to add a new ContainerFile + + The container files are all under builder/containers/ and should be + named ContainerFile-image-name so a builder can set the container + file property with readContainerFile('image-name'). + + The ContainerFile has four parts. First the base image and package + setup. Second the builder user setup (needs to match the builder uid + 1001 from the host). Third the installation of the buildbot-worker + startup wrapper script. Fourth launching the startup wrapper script. + + The base image and packages should install all development packages + that are needed to build the projects. + + The builder user setup needs to be the same uid 1001 as the host + builder user and have /home/builder as $HOME so the host shared + directory can be mapped. + + The startup script installation is slightly tricky. It either needs + to create a new buildbot-worker setup or edit an existing worker + buildbot.tac file to set the new password (which changes on each + run). The current ContainerFiles use some bashisms to create the + script, so make sure the image actually uses bash as /bin/sh (see + the debian containers for an example how to make sure it really is + bash). When using the existing setup script you must change the + image_name=... and maybe the export PATH=/usr/lib/ccache:$PATH line + (if the ccache wrappers are installed somewhere else). + + Finally the script (that launches buildbot-worker) should be run as + the builder user. = Local hacking diff --git a/SETUP b/SETUP index 972e725..ed82db3 100644 --- a/SETUP +++ b/SETUP @@ -136,3 +136,13 @@ WantedBy=multi-user.target systemctl enable buildbot-worker systemctl start buildbot-worker + +For each dedicated distro-arch worker make sure there is a name and +password in ~/buildbot.config. + +For the DockerLatentWorkers make sure that there is a private/public +key pair under /sourceware/builder/.ssh (ssh-keygen) with the public +part going into the docker_host builder user .ssh/authorized_keys. So +the DockerLatentWorker can run docker system dial-stdio on the +docker_host. Since the buildbot will use the ~/.ssh config make sure +that the hosts are in .known_hosts (by logging in by hand once). \ No newline at end of file -- 2.30.2