public inbox for buildbot@sourceware.org
 help / color / mirror / Atom feed
From: Mark Wielaard <mark@klomp.org>
To: Christophe Lyon <christophe.lyon@arm.com>
Cc: buildbot@sourceware.org
Subject: Re: Arm GCC buildbot workers
Date: Thu, 11 Aug 2022 22:39:56 +0200	[thread overview]
Message-ID: <20220811203956.GE5520@gnu.wildebeest.org> (raw)
In-Reply-To: <84fe482f-0b5e-face-9a7a-bb04ac162686@arm.com>

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

Hi Christophe,

On Thu, Aug 11, 2022 at 09:54:35AM +0200, Christophe Lyon wrote:
> I'd like to start 2 more workers, for GCC: one for armhf, one for arm64.
> They will be running ubuntu-22.04, 20 CPUs each.
> 
> So:
> ncpus:20
> maxcpus:20
> max_builds:1
> 
> I'd like them to run "gcc_full", which should take ~2h according to my
> experiments.

Cool. I did add a "gcc-full" build for x86_64 last week to use one of
the workers that has 12 or 16 cores. It takes ~50 to ~140 minutes
depending on which worker it picks and the other load on the machine.

When a build takes this long you cannot really test each commit
separately. 12 patches and you have a whole day of building. So you
either only build periodically, or you collapse the pending build
requests. The existing gcc-full builders collapse the builds. So I'll
suggest these new builders do too, but let me know if you rather have
them try a build only periodically (say every 8 hours?)

> Looking at https://builder.sourceware.org/buildbot/#/workers, I suspect
> there may be at least a naming problem since the other workers have
> "generic names" (I mean we already have similar workers, dedicated to gdb).
> 
> BTW, I'm wondering whether it was a mistake to have our other workers
> named armXX-ubuntu-YY, since the other ones are named distro-arch?

Naming isn't 100% consistent, and it doesn't really matter much for
worker names. It does matter a little for the builder names, sadly
those also aren't fully consistent. But as long as we are somewhat
consitent in the tag names hopefully people can still sort interesting
builds together.

Renaming the workers would mean changing them on both sides, so lets
stick with the original names for now. Then we can name these the
other way around:

ubuntu22_04-arm64
ubuntu22_04-armhf

See the attached patch.

Cheers,

Mark

[-- Attachment #2: 0001-Add-ubuntu22_04-arm64-and-ubuntu22_04-armhf-workers-.patch --]
[-- Type: text/plain, Size: 4810 bytes --]

From 5b16ab1d7a8e0fbc3ec2218a6087ad292dba41dd Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Thu, 11 Aug 2022 22:36:42 +0200
Subject: [PATCH] Add ubuntu22_04-arm64 and ubuntu22_04-armhf workers for gcc

Two more workers from the Works on Arm initiative to do full
gcc builds (bootstrap plus make check).
---
 buildbot.config.sample |  2 ++
 builder/master.cfg     | 39 ++++++++++++++++++++++++++++++++++++++-
 htdocs/index.html.post |  4 ++--
 3 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/buildbot.config.sample b/buildbot.config.sample
index b4c3f11..7986c6a 100644
--- a/buildbot.config.sample
+++ b/buildbot.config.sample
@@ -20,6 +20,8 @@ arm64-ubuntu20_04=frob
 arm64-ubuntu22_04=frob
 armhf-ubuntu20_04=frob
 armhf-ubuntu22_04=frob
+ubuntu22_04-arm64=frob
+ubuntu22_04-armhf=frob
 fedora-arm64=frob
 
 # Users
diff --git a/builder/master.cfg b/builder/master.cfg
index 55f8d3b..f945b14 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -241,6 +241,8 @@ ibm_power10_worker = worker.Worker("ibm_power10",
                                                       'cel@us.ibm.com']);
 c['workers'].append(ibm_power10_worker)
 
+# Various arm64/armhf workers provided by the Works on Arm initiative
+# 4 used for gdb
 arm64_ubuntu20_04_worker = worker.Worker("arm64-ubuntu20_04",
                                          getpw("arm64-ubuntu20_04"),
                                          max_builds=1,
@@ -269,6 +271,23 @@ armhf_ubuntu22_04_worker = worker.Worker("armhf-ubuntu22_04",
                                          notify_on_missing=['christophe.lyon@arm.com']);
 c['workers'].append(armhf_ubuntu22_04_worker)
 
+# 2 used for gcc
+ubuntu20_04_armhf_worker = worker.Worker("ubuntu22_04-armhf",
+                                         getpw("ubuntu22_04-armhf"),
+                                         properties={'ncpus': 20,
+                                                     'maxcpus': 20},
+                                         max_builds=1,
+                                         notify_on_missing=['christophe.lyon@arm.com']);
+c['workers'].append(ubuntu20_04_armhf_worker)
+
+ubuntu20_04_arm64_worker = worker.Worker("ubuntu22_04-arm64",
+                                         getpw("ubuntu22_04-arm64"),
+                                         properties={'ncpus': 20,
+                                                     'maxcpus': 20},
+                                         max_builds=1,
+                                         notify_on_missing=['christophe.lyon@arm.com']);
+c['workers'].append(ubuntu20_04_arm64_worker)
+
 # 8 core arm64 server hosted by osuosl
 fedora_arm64_worker = worker.Worker("fedora-arm64",
                                          getpw("fedora-arm64"),
@@ -788,7 +807,9 @@ gcc_scheduler = schedulers.SingleBranchScheduler(
                                         branch="master"),
         reason="gcc project master branch update",
         builderNames=["gcc-fedrawhide-x86_64",
-                      "gcc-full-debian-amd64"])
+                      "gcc-full-debian-amd64",
+                      "gcc-full-ubuntu-armhf",
+                      "gcc-full-ubuntu-arm64"])
 c['schedulers'].append(gcc_scheduler)
 
 systemtap_scheduler = schedulers.SingleBranchScheduler(
@@ -3002,6 +3023,22 @@ gcc_full_debian_amd64_builder = util.BuilderConfig(
         factory=gcc_full_build_factory)
 c['builders'].append(gcc_full_debian_amd64_builder)
 
+gcc_full_ubuntu_armhf_builder = util.BuilderConfig(
+        name="gcc-full-ubuntu-armhf",
+        collapseRequests=True,
+        workernames=["ubuntu22_04-armhf"],
+        tags=["gcc-full", "ubuntu", "armhf"],
+        factory=gcc_full_build_factory)
+c['builders'].append(gcc_full_ubuntu_armhf_builder)
+
+gcc_full_ubuntu_arm64_builder = util.BuilderConfig(
+        name="gcc-full-ubuntu-arm64",
+        collapseRequests=True,
+        workernames=["ubuntu22_04-arm64"],
+        tags=["gcc-full", "ubuntu", "arm64"],
+        factory=gcc_full_build_factory)
+c['builders'].append(gcc_full_ubuntu_arm64_builder)
+
 
 # glibc build steps, factory, builders
 
diff --git a/htdocs/index.html.post b/htdocs/index.html.post
index 56cf675..c9767c6 100644
--- a/htdocs/index.html.post
+++ b/htdocs/index.html.post
@@ -12,8 +12,8 @@
 	<li>Frank Eigler for the fedrawhide-x86_64 worker</li>
 	<li>IBM for the ibm_power8, ibm_power9 and ibm_power10 workers</li>
 	<li>The Works on Arm initiative for the arm64-ubuntu20_04,
-            arm64-ubuntu22_04, armhf-ubuntu20_04 and
-            armhf-ubuntu22_04 workers</li>
+            arm64-ubuntu22_04, armhf-ubuntu20_04, armhf-ubuntu22_04
+            ubuntu22_04-arm64 and ubuntu22_04-armhf workers</li>
 	<li>OSUOSL for the fedora-arm64 worker</li>
       </ul>
     </p>
-- 
2.31.1


  reply	other threads:[~2022-08-11 20:39 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-11  7:54 Christophe Lyon
2022-08-11 20:39 ` Mark Wielaard [this message]
2022-08-12  7:23   ` Christophe Lyon
2022-08-12 16:44     ` Mark Wielaard
2022-08-16 11:54       ` Christophe Lyon
2022-08-17 11:51         ` Christophe Lyon
2022-08-17 13:00         ` Mark Wielaard
2022-08-17 13:45           ` Christophe Lyon
2022-08-17 20:42             ` Mark Wielaard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220811203956.GE5520@gnu.wildebeest.org \
    --to=mark@klomp.org \
    --cc=buildbot@sourceware.org \
    --cc=christophe.lyon@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).