From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id DEE94383D832 for ; Tue, 24 May 2022 23:59:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DEE94383D832 Received: from reform (deer0x07.wildebeest.org [172.31.17.137]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 4D45A302BBEB; Wed, 25 May 2022 01:59:38 +0200 (CEST) Received: by reform (Postfix, from userid 1000) id D16882E81FE6; Wed, 25 May 2022 01:59:37 +0200 (CEST) Date: Wed, 25 May 2022 01:59:37 +0200 From: Mark Wielaard To: Overseers mailing list Cc: Philip Herron , Julian Seward , Thomas Fitzsimmons , Sergio Durigan Junior , Dan =?iso-8859-1?Q?Hor=E1k?= , Martin =?utf-8?B?TGnFoWth?= Subject: Re: Moving buildbot master to sourceware Message-ID: References: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="d4aYE9LK0yI4Yxfw" Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-10.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: overseers@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Overseers mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2022 23:59:42 -0000 --d4aYE9LK0yI4Yxfw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, May 24, 2022 at 03:55:08AM +0200, Mark Wielaard via Overseers wrote: > There is no documentation yet. Sorry. Documentation on adding workers and container files attached (and committed). Let me know if I missed something. Cheers, Mark --d4aYE9LK0yI4Yxfw Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-Add-documentation-on-adding-a-dedicated-or-container.patch" >From cd13ea399e2b24d607fdcaa26b5405ee89b95468 Mon Sep 17 00:00:00 2001 From: Mark Wielaard Date: Wed, 25 May 2022 01:52:36 +0200 Subject: [PATCH] Add documentation on adding a dedicated or container worker --- README | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- SETUP | 10 ++++++++ 2 files changed, 84 insertions(+), 1 deletion(-) diff --git a/README b/README index 72779d1..2574424 100644 --- a/README +++ b/README @@ -72,6 +72,19 @@ htdocs: document root of https://builder.sourceware.org/ factory=project_factory) c['builders'].append(project__builder) + The above uses a dedicated distro-arch worker. If you want to use a + container image worker then you need to add a container-file + property to the builder and use one or more of the container image + workers (currently there are only 3 x86_64 container image workers + bb1, bb2 and bb3). + + properties={'container-file': + readContainerFile('fedora-latest')}, + workernames=["bb1", "bb2", "bb3"], + + Look under builder/containers/ContainerFile-* to the container image + names you can use. + There are a couple of common steps that might be useful to reuse in you factory. A factory using common steps look something like: @@ -95,7 +108,67 @@ htdocs: document root of https://builder.sourceware.org/ - How to add a new worker - ... + + There are two kinds of workers. Dedicated distr-arch workers + (worker.Worker) and container image workers + (worker.DockerLatentWorker). The first has a whole distro installed + and runs a dedicated buildbot-worker which connects to + builder.sourceware.org to know what to build. The second runs a + (probably minimal) install with docker (or podman, but there are + still some incompatabilities) to which builder.sourceware.org + connects when there is a build providing a container file inside + which the build should be run (the container file will actually + contain a buildbot-worker which connects back to + builder.sourceware.org). + + Both kind of workers need a little setup on the server. The + dedicated workers need a worker name and password, the container + image builders need a ssh setup (see SETUP). + + The dedicated workers need the have all the developer packages + installed (look at one of the ContainerFiles to see which + ones). Since most builds will be almost the same with just a few + file changes, it is a good idea to have ccache installed and make + sure the ccache wrapper binaries are in the PATH before running the + buildbot-worker. + + The container image builders should have a dedicated user (builder, + uid 1001) and a "shared" directory in which the container images can + put shared data (worker status, git checkouts, ccache). They should + also periodically clean up the images to make sure they get + regenerated with new packages. Run docker system prune -a -f once a + week. + +- How to add a new ContainerFile + + The container files are all under builder/containers/ and should be + named ContainerFile-image-name so a builder can set the container + file property with readContainerFile('image-name'). + + The ContainerFile has four parts. First the base image and package + setup. Second the builder user setup (needs to match the builder uid + 1001 from the host). Third the installation of the buildbot-worker + startup wrapper script. Fourth launching the startup wrapper script. + + The base image and packages should install all development packages + that are needed to build the projects. + + The builder user setup needs to be the same uid 1001 as the host + builder user and have /home/builder as $HOME so the host shared + directory can be mapped. + + The startup script installation is slightly tricky. It either needs + to create a new buildbot-worker setup or edit an existing worker + buildbot.tac file to set the new password (which changes on each + run). The current ContainerFiles use some bashisms to create the + script, so make sure the image actually uses bash as /bin/sh (see + the debian containers for an example how to make sure it really is + bash). When using the existing setup script you must change the + image_name=... and maybe the export PATH=/usr/lib/ccache:$PATH line + (if the ccache wrappers are installed somewhere else). + + Finally the script (that launches buildbot-worker) should be run as + the builder user. = Local hacking diff --git a/SETUP b/SETUP index 972e725..ed82db3 100644 --- a/SETUP +++ b/SETUP @@ -136,3 +136,13 @@ WantedBy=multi-user.target systemctl enable buildbot-worker systemctl start buildbot-worker + +For each dedicated distro-arch worker make sure there is a name and +password in ~/buildbot.config. + +For the DockerLatentWorkers make sure that there is a private/public +key pair under /sourceware/builder/.ssh (ssh-keygen) with the public +part going into the docker_host builder user .ssh/authorized_keys. So +the DockerLatentWorker can run docker system dial-stdio on the +docker_host. Since the buildbot will use the ~/.ssh config make sure +that the hosts are in .known_hosts (by logging in by hand once). \ No newline at end of file -- 2.30.2 --d4aYE9LK0yI4Yxfw--