public inbox for
 help / color / mirror / Atom feed
* [COMMITTED] Add README_workers
@ 2022-07-05 10:37 Mark Wielaard
  0 siblings, 0 replies; only message in thread
From: Mark Wielaard @ 2022-07-05 10:37 UTC (permalink / raw)
  To: buildbot; +Cc: Mark Wielaard

All the information to add a new buildbot worker.
 README_workers | 182 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
 create mode 100644 README_workers

diff --git a/README_workers b/README_workers
new file mode 100644
index 0000000..76ed9c3
--- /dev/null
+++ b/README_workers
@@ -0,0 +1,182 @@
+Adding a new buildbot-worker
+A worker is a (vitual) machine that does the actual builds. supports two kinds of workers.  A worker in a
+full distro setup with all build dependencies already insalled.  And a
+container latent worker which can create containers from a buildbot
+provided file to run the build inside.
+The container latent worker is a bit more work to install and
+requires ssh access to the host running the worker.
+The full distro worker is easier to setup, doesn't need to be
+reachable as long as it can make a connection to, but requires all build dependency packages to
+be install up front.
+Adding a (virtual) machine (distro) builder
+Setup a normal distro install and make sure all build dependencies are
+installed. A good list can be in the container files for various
+distros that uses (examples for debian, fedora
+and opensuse):
+And it is a good idea to install ccache and put it in the worker user
+PATH because most builds will compile the exact same sources. (See below)
+Install buildbot-worker either as distro package or through pip:
+$ pip install buildbot-worker
+To create the worker you'll need a name and a password.  Please
+contact to get a password.  Also say what the
+"normal" number of cpus/cores that builds can be used and what the max
+number of cpus/cores is that a "heavy" build can use (these values can
+be the same). Also say what the maximum number of concurrent builds
+can be.
+- ncpus: 4
+- maxcpus: 16
+- max_builds: 3
+We'll add something like the following to the master.cfg file:
+name_worker = worker.Worker("name",
+			     getpw("name"),
+                             max_builds=3,
+                             properties={'ncpus': 4, 'maxcpus': 16},
+                             notify_on_missing='');
+The password will be provided off-list and won't be checked into git.
+It will have to be added to the /sourceware/builder/buildbot.conf file
+by someone with builder permissions. And an extra entry will be added
+to the buildbot.config.sample file.
+Then you can create the worker, do this under a user where the builds
+will be running:
+- buildbot-worker create-worker workerdir name passwd
+Create a startup script like:
+= =
+# ccache should be in front, sometimes in /usr/lib64
+export PATH=/usr/lib/ccache:$PATH
+# the worker dir, same as first argument to create-worker
+cd ~/workerdir
+# Add info for buildbot users
+echo "Worker admin contact <>" > info/admin
+# Distro name, glibc version, binutils version, gcc version
+echo "Distro name" > info/host
+iconv --version | head -1 >> info/host
+objcopy --version | head -1 >> info/host
+gcc --version | head -1 >> info/host
+# This will demonize after startup
+buildbot-worker start
+And put it in a cronjob so it starts on system reboot:
+@reboot rm -f ~/workerdir/; ~/
+Adding a container (latent) worker
+This is a bit more work, but has as advantage that you don't have to
+keep track of updates or any missing packages. It also take a bit more
+disk space since it creates (full distro) container images.
+The setup requires you to provide a builder user with uid 1001 and a
+/home/builder/shared directory that keeps ccache, git checkouts and
+builds with the right permissions. And allows builder to connect to
+that user through ssh to setup a docker socket connection. Note that
+podman doesn't work here since it has a couple of small, but
+significant incompatibilities. e.g.
+Easiest setup is using something like coreos with a butane/ignotion
+file. The advantage of coreos is that it autoupdates, already comes
+with docker/moby, uses selinux for isolation and that you can simply
+reinstall (the images and shared directory will just be recreated by
+the buildbot). This sample butane.yaml file is for a libvirt setup,
+you will likely need to change the device:, /etc/hostname contents and
+core ssh_authorized_keys.
+variant: fcos
+version: 1.4.0
+  disks:
+  - device: /dev/vda
+    wipe_table: false
+    partitions:
+    # 10GB root (already exists, resize)
+    - number: 4
+      label: root
+      size_mib: 10240
+      resize: true
+    # 8GB swap
+    - number: 5
+      label: swap
+      size_mib: 8192
+    # Everything else for /var (which includes everything writable)
+    - size_mib: 0
+      label: var
+  filesystems:
+    - device: /dev/disk/by-partlabel/swap
+      format: swap
+      wipe_filesystem: true
+      with_mount_unit: true
+    - path: /var
+      device: /dev/disk/by-partlabel/var
+      # We can select the filesystem we'd like.
+      format: ext4
+      wipe_filesystem: true
+      # Ask Butane to generate a mount unit for us so that this filesystem
+      # gets mounted in the real root.
+      with_mount_unit: true
+  files:
+    - path: /etc/hostname
+      mode: 0644
+      contents:
+        inline: sourceware-builder1
+  units:
+    # Create and set selinux context for shared builder dir
+    - name: builder-shared-dir.service
+      enabled: true
+      contents: |
+        [Unit]
+        Description=Builder shared dir
+        Before=zincati.service
+        ConditionPathExists=!/var/lib/%N.stamp
+        [Service]
+        Type=oneshot
+        RemainAfterExit=yes
+        ExecStart=/bin/mkdir -p /home/builder/shared
+        ExecStart=/bin/chown builder.builder /home/builder/shared
+        ExecStart=/bin/chcon -t container_file_t /home/builder/shared
+        ExecStart=/bin/touch /var/lib/%N.stamp
+        [Install]
+  users:
+    - name: core
+      ssh_authorized_keys:
+        - YOUR_PUB_KEY
+    - name: builder
+      uid: 1001
+      groups: [docker]
+      ssh_authorized_keys:
+        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnB//uE66TX+nmLuox3ETfZ7dvjUl2mlwN4Q+107EWCuP05eKxcahyzwenmzODzzC8/hUVeUtuCJJyMW8+CpqCluHd2bEUT9WIzP1C/T22jBUbhY7zXLufSYfKmZe9zHNUizEbqbGAfR01D3jv3wis8JNOq/yopGLKJmAPd/Ye8XN5lsfAUTREIF+xygjxSb6SZPU9UvKlJmnwZoCN2Fd1u4IhSWBcD3dyZyirD80ZLGsSdOUmzX2Z2NaSI5tG6MxjYtQZbzmsG/Z0HY1Y/KiSolX989+7dMKMy2V911SkS6Cx9aVuSY+ig9Rq6GpfjGkBnHrNCCeVFe8PlYJl6DXZ

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2022-07-05 10:37 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-05 10:37 [COMMITTED] Add README_workers Mark Wielaard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).