From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 8DBDB3857BAD for ; Mon, 13 Jun 2022 09:51:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 8DBDB3857BAD Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from tarox.wildebeest.org (83-87-18-245.cable.dynamic.v4.ziggo.nl [83.87.18.245]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id B0282302BBEB; Mon, 13 Jun 2022 11:51:38 +0200 (CEST) Received: by tarox.wildebeest.org (Postfix, from userid 1000) id 32D4E4000EF5; Mon, 13 Jun 2022 11:51:38 +0200 (CEST) From: Mark Wielaard To: buildbot@sourceware.org Cc: Mark Wielaard Subject: [COMMITTED] Add README_containers - Debugging issues with CI container files Date: Mon, 13 Jun 2022 11:51:25 +0200 Message-Id: <20220613095125.27140-1-mark@klomp.org> X-Mailer: git-send-email 2.18.4 X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: buildbot@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "The https://builder.sourceware.org/ buildbot" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jun 2022 09:51:45 -0000 A quick reference on how to debug issues inside the container files. --- README_containers | 112 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 README_containers diff --git a/README_containers b/README_containers new file mode 100644 index 0000000..66ac558 --- /dev/null +++ b/README_containers @@ -0,0 +1,112 @@ +Debugging issues with CI container files +---------------------------------------- + +TL;DR; + +- Setup podman (or docker) +- git clone https://sourceware.org/git/builder.git +- cd builder +- podman build -t builder-debian-stable \ + -f builder/containers/Containerfile-debian-stable \ + builder/containers +- podman run --name debian-stable -ti --entrypoint /bin/bash \ + builder-debian-stable + +Full intructions below. + +Some CI builders on https://builder.sourceware.org/ will use container +files to run builds and tests. When the logs don't make clear how/why +a test fails and local debugging doesn't help it is useful to be able +to recreate the container environment locally. + +We will assume an x86_64 GNU/Linux install with docker or podman and +user namespaces support. The none-x86_64 hosts on +https://builder.sourceware.org/ don't use container files and you can +contact buildbot@sourceware.org to get access if debugging locally +isn't possible. + +The container files +------------------- + +The container files can be found in builder.git together with the +buildbot configuration: + +git clone https://sourceware.org/git/builder.git +cd builder +ls builder/containers + +Containerfile-debian-stable Containerfile-opensuse-leap +Containerfile-fedora-latest Containerfile-opensuse-tumbleweed + +Setting up docker or podman +--------------------------- + +- Make sure that you have docker installed and that the user is in the + docker group + +sudo apt install docker.io or sudo dnf install moby-engine +sudo systemctl enable docker.service +sudo systemctl start docker.service +sudo usermod -a -G docker $(whoami) + +- Or make sure to have podman and user namespaces setup + +sudo apt install podman slirp4netns fuse-overlayfs +or +sudo dnf install podman slirp4netns fuse-overlayfs + +Make sure cat /proc/sys/user/max_user_namespaces is >= 15000: +if not, sudo sysctl user.max_user_namespaces=15000 +or set it in /etc/sysctl.conf + +Make sure that /etc/subuid and /etc/subgid are setup for your user, +both should contain something like: youruser:100000:65536 +if not, +sudo usermod --add-subuids 100000-165535 \ + --add-subgids 100000-165535 $(whoami) + +Building the image +------------------ + +Note that the container files are rebuild once a week for the +builder.sourceware.org workers. So although unlikely it might be that +a freshly build container contains slightly newer versions of the +packages than the one that ran on the builder.sourceware.org worker. + +The containers contain all build dependencies for all projects +including for generating the documentation. So they are similar in +size to a full distribution. Make sure you have several GB available +in /var/lib/docker for docker or in $HOME/.local/share/containers and +/var/tmp for podman + +git clone https://sourceware.org/git/builder.git +cd builder +podman build -t builder-debian-stable \ + -f builder/containers/Containerfile-debian-stable \ + builder/containers + +Or use docker instead of podman, they take the same arguments +(although the last PATH is optional in some versions). + +Running the image +----------------- + +The image is setup for running the buildbot-worker as user builder. To +use it interactively we'll have to override the entrypoint with +/bin/bash. + +podman run --name debian-stable -ti --entrypoint /bin/bash \ + builder-debian-stable + +To reuse the same container run (with interactive attach): + +podman start -ai debian-stable + +The container has all the packages needed for building the projects, +including git. But if you find some package is missing start the +container and enter it as root: + +podman start debian-stable +podman exec -it --user root debian-stable apt install emacs + +The same works with the docker command. -- 2.18.4