[PATCH] containers: add openSUSE containers (Leap, Tumbleweed)

[PATCH] containers: add openSUSE containers (Leap, Tumbleweed)

[Martin Liška]

---
 .../containers/Containerfile-opensuse-leap    | 49 +++++++++++++++++++
 .../Containerfile-opensuse-tumbleweed         | 49 +++++++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 builder/containers/Containerfile-opensuse-leap
 create mode 100644 builder/containers/Containerfile-opensuse-tumbleweed

diff --git a/builder/containers/Containerfile-opensuse-leap b/builder/containers/Containerfile-opensuse-leap
new file mode 100644
index 0000000..dcdaf82
--- /dev/null
+++ b/builder/containers/Containerfile-opensuse-leap
@@ -0,0 +1,49 @@
+# Setup the (minimal) image, upgrade and install all devel packages
+FROM registry.opensuse.org/opensuse/leap
+RUN zypper -n dup
+RUN zypper -n install \
+        gcc gcc-c++ libtool ccache valgrind \
+        isl-devel mpc-devel mpfr-devel gmp-devel \
+        patch util-linux diffutils iproute2 bsdtar cpio procps \
+        coreutils make git autoconf dejagnu automake gettext-devel bison flex \
+        bzip2 libbz2-devel xz xz-devel gzip zlib-devel zstd libzstd-devel \
+        findutils file tar curl libarchive-devel libcurl-devel \
+        libmicrohttpd-devel sqlite-devel libelf-devel libdw-devel \
+        xmlto poppler-tools libxslt docbook-dtds docbook-xsl-stylesheets \
+        libxml2-devel dos2unix dpkg python3-devel python3-mock \
+        python3-rpm python3-pyxdg python3-pip openmpi-devel gdb help2man wget
+RUN zypper clean
+
+# Install buildbot-worker
+RUN pip install buildbot-worker
+
+# Setup user with same id as host user id.
+RUN useradd -u 1001 -U -d /home/builder -m builder
+
+# Create buildbot-worker script which sets up shared worker and ccache
+# data with the host. Each run gets a new password, so update that too.
+RUN cd /home/builder; \
+    echo $'image_name=opensuse-tumbleweed\n\
+        worker_dir=shared/$image_name/worker\n\
+        tac_file=$worker_dir/buildbot.tac\n\
+        if [ ! -f $tac_file ]; then\n\
+        mkdir -p $worker_dir\n\
+        buildbot-worker create-worker $worker_dir \
+          $BUILDMASTER:$BUILDMASTER_PORT $WORKERNAME $WORKERPASS\n\
+        echo overseers@sourceware.org > $worker_dir/info/admin\n\
+        echo $image_name > $worker_dir/info/host\n\
+      else\n\
+        sed -ie \"s/^passwd = .*/passwd = \'$WORKERPASS\'/\" $tac_file\n\
+      fi\n\
+      unset WORKERPASS\n\
+      export PATH=/usr/lib64/ccache:$PATH\n\
+      mkdir -p shared/$image_name/ccache\n\
+      export CCACHE_DIR=/home/builder/shared/$image_name/ccache\n\
+      buildbot-worker start --nodaemon $worker_dir\n' \
+	> bb-start.sh && \
+    chmod 755 bb-start.sh
+
+# And now run the script
+USER builder
+WORKDIR /home/builder
+ENTRYPOINT /home/builder/bb-start.sh
diff --git a/builder/containers/Containerfile-opensuse-tumbleweed b/builder/containers/Containerfile-opensuse-tumbleweed
new file mode 100644
index 0000000..001b956
--- /dev/null
+++ b/builder/containers/Containerfile-opensuse-tumbleweed
@@ -0,0 +1,49 @@
+# Setup the (minimal) image, upgrade and install all devel packages
+FROM registry.opensuse.org/opensuse/tumbleweed:latest
+RUN zypper -n dup
+RUN zypper -n install \
+        gcc gcc-c++ libtool ccache valgrind \
+        isl-devel mpc-devel mpfr-devel gmp-devel \
+        patch util-linux diffutils iproute2 bsdtar cpio procps \
+        coreutils make git autoconf dejagnu automake gettext-devel bison flex \
+        bzip2 libbz2-devel xz xz-devel gzip zlib-devel zstd libzstd-devel \
+        findutils file tar curl libarchive-devel libcurl-devel \
+        libmicrohttpd-devel sqlite-devel libelf-devel libdw-devel \
+        xmlto poppler-tools libxslt docbook-dtds docbook-xsl-stylesheets \
+        libxml2-devel dos2unix dpkg python3-devel python3-mock \
+        python3-rpm python3-pyxdg python3-pip openmpi-devel gdb help2man wget
+RUN zypper clean
+
+# Install buildbot-worker
+RUN pip install buildbot-worker
+
+# Setup user with same id as host user id.
+RUN useradd -u 1001 -U -d /home/builder -m builder
+
+# Create buildbot-worker script which sets up shared worker and ccache
+# data with the host. Each run gets a new password, so update that too.
+RUN cd /home/builder; \
+    echo $'image_name=opensuse-tumbleweed\n\
+        worker_dir=shared/$image_name/worker\n\
+        tac_file=$worker_dir/buildbot.tac\n\
+        if [ ! -f $tac_file ]; then\n\
+        mkdir -p $worker_dir\n\
+        buildbot-worker create-worker $worker_dir \
+          $BUILDMASTER:$BUILDMASTER_PORT $WORKERNAME $WORKERPASS\n\
+        echo overseers@sourceware.org > $worker_dir/info/admin\n\
+        echo $image_name > $worker_dir/info/host\n\
+      else\n\
+        sed -ie \"s/^passwd = .*/passwd = \'$WORKERPASS\'/\" $tac_file\n\
+      fi\n\
+      unset WORKERPASS\n\
+      export PATH=/usr/lib64/ccache:$PATH\n\
+      mkdir -p shared/$image_name/ccache\n\
+      export CCACHE_DIR=/home/builder/shared/$image_name/ccache\n\
+      buildbot-worker start --nodaemon $worker_dir\n' \
+	> bb-start.sh && \
+    chmod 755 bb-start.sh
+
+# And now run the script
+USER builder
+WORKDIR /home/builder
+ENTRYPOINT /home/builder/bb-start.sh
-- 
2.36.1

Re: [PATCH] containers: add openSUSE containers (Leap, Tumbleweed)

[Mark Wielaard]

Hi Martin,

On Wed, 2022-05-25 at 14:57 +0200, Martin Liška via Overseers wrote:
> ---
>  .../containers/Containerfile-opensuse-leap    | 49
> +++++++++++++++++++
>  .../Containerfile-opensuse-tumbleweed         | 49
> +++++++++++++++++++
>  2 files changed, 98 insertions(+)
>  create mode 100644 builder/containers/Containerfile-opensuse-leap
>  create mode 100644 builder/containers/Containerfile-opensuse-
> tumbleweed

Thanks, pushed.

I quickly tried out the tumbleweed container to build elfutils,
valgrind and libabigail. Looked fine.

Now all we need to decide is which builders to setup to do builds.

Cheers,

Mark

[PATCH] Add openSUSE builders.

[Martin Liška]

Hi.

Add builders for bzip2 and elfutils.

Let's start with these for testing purpose. I'm going to add others soon..

Cheers,
Martin

---
 builder/master.cfg | 44 ++++++++++++++++++++++++++++++++++++++++++--
 htdocs/index.html  |  8 ++++++++
 2 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 4fabce7..26f57c2 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -293,7 +293,9 @@ elfutils_scheduler = schedulers.SingleBranchScheduler(
                       "elfutils-debian-armhf",
                       "elfutils-fedora-s390x",
                       "elfutils-debian-ppc64",
-                      "elfutils-fedora-ppc64le"])
+                      "elfutils-fedora-ppc64le",
+                      "elfutils-opensusetw-x86_64",
+                      "elfutils-opensuseleap-x86_64"])
 c['schedulers'].append(elfutils_scheduler)
 
 libabigail_scheduler = schedulers.SingleBranchScheduler(
@@ -341,7 +343,9 @@ bzip2_scheduler = schedulers.SingleBranchScheduler(
                       "bzip2-debian-armhf",
                       "bzip2-fedora-s390x",
                       "bzip2-debian-ppc64",
-                      "bzip2-fedora-ppc64le"])
+                      "bzip2-fedora-ppc64le",
+                      "bzip2-opensusetw-x86_64",
+                      "bzip2-opensuseleap-x86_64"])
 c['schedulers'].append(bzip2_scheduler)
 
 valgrind_scheduler = schedulers.SingleBranchScheduler(
@@ -675,6 +679,24 @@ bzip2_debian_amd64_builder = util.BuilderConfig(
         factory=bzip2_factory_no_valgrind)
 c['builders'].append(bzip2_debian_amd64_builder)
 
+bzip2_opensusetw_x86_64_builder = util.BuilderConfig(
+        name="bzip2-opensusetw-x86_64",
+	properties={'container-file':
+		    readContainerFile('opensuse-tumbleweed')},
+	workernames=vm_workers,
+        tags=["bzip2", "opensusetw", "x86_64"],
+        factory=bzip2_factory)
+c['builders'].append(bzip2_opensusetw_x86_64_builder)
+
+bzip2_opensuseleap_x86_64_builder = util.BuilderConfig(
+        name="bzip2-opensuseleap-x86_64",
+	properties={'container-file':
+		    readContainerFile('opensuse-leap')},
+	workernames=vm_workers,
+        tags=["bzip2", "opensuseleap", "x86_64"],
+        factory=bzip2_factory)
+c['builders'].append(bzip2_opensuseleap_x86_64_builder)
+
 bzip2_debian_armhf_builder = util.BuilderConfig(
         name="bzip2-debian-armhf",
         workernames=["debian-armhf"],
@@ -1033,6 +1055,24 @@ elfutils_debian_amd64_builder = util.BuilderConfig(
         factory=elfutils_factory_mvu)
 c['builders'].append(elfutils_debian_amd64_builder)
 
+elfutils_opensusetw_x86_64_builder = util.BuilderConfig(
+        name="elfutils-opensusetw-x86_64",
+	properties={'container-file':
+		    readContainerFile('opensusetw-latest')},
+	workernames=vm_workers,
+        tags=["elfutils", "opensusetw", "x86_64"],
+        factory=elfutils_factory_distcheck)
+c['builders'].append(elfutils_opensusetw_x86_64_builder)
+
+elfutils_opensuseleap_x86_64_builder = util.BuilderConfig(
+        name="elfutils-opensuseleap-x86_64",
+	properties={'container-file':
+		    readContainerFile('opensuseleap-latest')},
+	workernames=vm_workers,
+        tags=["elfutils", "opensuseleap", "x86_64"],
+        factory=elfutils_factory_distcheck)
+c['builders'].append(elfutils_opensuseleap_x86_64_builder)
+
 elfutils_debian_armhf_builder = util.BuilderConfig(
         name="elfutils-debian-armhf",
         workernames=["debian-armhf"],
diff --git a/htdocs/index.html b/htdocs/index.html
index 9d9d723..6a87be6 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -68,6 +68,10 @@
 	      <img src="/buildbot/badges/bzip2-debian-ppc64.svg"></a></td>
 	  <td><a href="/buildbot/#builders/bzip2-fedora-ppc64le">fedora-ppc64le<br>
 	      <img src="/buildbot/badges/bzip2-fedora-ppc64le.svg"></a></td>
+	  <td><a href="/buildbot/#builders/bzip2-opensusetw-x86_64">opensusetw-x86_64<br>
+	      <img src="/buildbot/badges/bzip2-opensusetw-x86_64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/bzip2-opensuseleap-x86_64">opensuseleap-x86_64<br>
+	      <img src="/buildbot/badges/bzip2-opensuseleap-x86_64.svg"></a></td>
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=debugedit">debugedit</a></td>
@@ -131,6 +135,10 @@
 	      <img src="/buildbot/badges/elfutils-debian-ppc64.svg"></a></td>
 	  <td><a href="/buildbot/#builders/elfutils-fedora-ppc64le">fedora-ppc64le<br>
 	      <img src="/buildbot/badges/elfutils-fedora-ppc64le.svg"></a></td>
+	  <td><a href="/buildbot/#builders/elfutils-opensusetw-x86_64">opensusetw-x86_64<br>
+	      <img src="/buildbot/badges/elfutils-opensusetw-x86_64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/elfutils-opensuseleap-x86_64">opensuseleap-x86_64<br>
+	      <img src="/buildbot/badges/elfutils-opensuseleap-x86_64.svg"></a></td>
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=gccrust">gccrust</a></td>
-- 
2.36.1

Re: [PATCH] Add openSUSE builders.

[Mark Wielaard]

Hi Martin,

On Wed, May 25, 2022 at 10:01:30PM +0200, Martin Liška wrote:
> Add builders for bzip2 and elfutils.
> 
> Let's start with these for testing purpose. I'm going to add others soon..

Note you mispelled the container names in the elfutils builders (but
they are correct in the bzip2 builders).

> +elfutils_opensusetw_x86_64_builder = util.BuilderConfig(
> +        name="elfutils-opensusetw-x86_64",
> +	properties={'container-file':
> +		    readContainerFile('opensusetw-latest')},

opensuse-tumbleweed

> +	workernames=vm_workers,
> +        tags=["elfutils", "opensusetw", "x86_64"],
> +        factory=elfutils_factory_distcheck)
> +c['builders'].append(elfutils_opensusetw_x86_64_builder)
> +
> +elfutils_opensuseleap_x86_64_builder = util.BuilderConfig(
> +        name="elfutils-opensuseleap-x86_64",
> +	properties={'container-file':
> +		    readContainerFile('opensuseleap-latest')},

opensuse-leap

> +	workernames=vm_workers,
> +        tags=["elfutils", "opensuseleap", "x86_64"],
> +        factory=elfutils_factory_distcheck)
> +c['builders'].append(elfutils_opensuseleap_x86_64_builder)

Everything else looks correct. Note that I only noticed because I ran
buildbot checkconfig builder locally.

I added you to the builder group so you can push yourself after the
fix (and please do run checkconfig locally). And so you can help if
anything blows up :)

Cheers,

Mark

Re: [PATCH] Add openSUSE builders.

[Martin Liška]

On 5/25/22 23:48, Mark Wielaard wrote:
> Hi Martin,
> 
> On Wed, May 25, 2022 at 10:01:30PM +0200, Martin Liška wrote:
>> Add builders for bzip2 and elfutils.
>>
>> Let's start with these for testing purpose. I'm going to add others soon..
> 
> Note you mispelled the container names in the elfutils builders (but
> they are correct in the bzip2 builders).

Sure, thanks for heads up.

> 
>> +elfutils_opensusetw_x86_64_builder = util.BuilderConfig(
>> +        name="elfutils-opensusetw-x86_64",
>> +	properties={'container-file':
>> +		    readContainerFile('opensusetw-latest')},
> 
> opensuse-tumbleweed
> 
>> +	workernames=vm_workers,
>> +        tags=["elfutils", "opensusetw", "x86_64"],
>> +        factory=elfutils_factory_distcheck)
>> +c['builders'].append(elfutils_opensusetw_x86_64_builder)
>> +
>> +elfutils_opensuseleap_x86_64_builder = util.BuilderConfig(
>> +        name="elfutils-opensuseleap-x86_64",
>> +	properties={'container-file':
>> +		    readContainerFile('opensuseleap-latest')},
> 
> opensuse-leap
> 
>> +	workernames=vm_workers,
>> +        tags=["elfutils", "opensuseleap", "x86_64"],
>> +        factory=elfutils_factory_distcheck)
>> +c['builders'].append(elfutils_opensuseleap_x86_64_builder)
> 
> Everything else looks correct. Note that I only noticed because I ran
> buildbot checkconfig builder locally.

Fixed and pushed.

> 
> I added you to the builder group so you can push yourself after the
> fix (and please do run checkconfig locally). And so you can help if
> anything blows up :)

Cool, thanks for the trust.

Cheers,
Martin

> 
> Cheers,
> 
> Mark