ibm_power8 and ibm_power10 workers

ibm_power8 and ibm_power10 workers

[Mark Wielaard]

Hi Will,

I see the ibm_power8 and ibm_power10 workers are not connected to the
buildbot. I assumed it was because of the recent upgrade, but it looks
like they haven't connected for more than a week. But the ibm_power9
worker is happily doing builds.

Cheers,

Mark

Re: ibm_power8 and ibm_power10 workers

[will schmidt]

On Sun, 2022-10-09 at 16:41 +0200, Mark Wielaard wrote:
> Hi Will,
> 
> I see the ibm_power8 and ibm_power10 workers are not connected to the
> buildbot. I assumed it was because of the recent upgrade, but it
> looks
> like they haven't connected for more than a week. But the ibm_power9
> worker is happily doing builds.
> 
> Cheers,
> 
> Mark

Hi Mark, etc,
 Thanks for the poke.    Something non-functional with the @reboot
entry in the crontab on the power10.   I've started it manually, and
will investigate a bit to try to fix it for the long term.

Our power8 is down, being rebuilt, etc.  it should be back after a
time.

Thanks
-Will

Re: ibm_power8 and ibm_power10 workers

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

Hi Will,

On Mon, Oct 10, 2022 at 12:01:26PM -0500, will schmidt wrote:
>  Thanks for the poke.    Something non-functional with the @reboot
> entry in the crontab on the power10.   I've started it manually, and
> will investigate a bit to try to fix it for the long term.

The power10 builders seem fine again. Thanks.

> Our power8 is down, being rebuilt, etc.  it should be back after a
> time.

For now I disabled the gdb and valgrind power8 builders. Otherwise
they build queue just keeps growing. Please let us know when the
power8 worker is back and we'll reenable them again.

Cheers,

Mark

[-- Attachment #2: 0001-Disable-gdb-valgrind-ibm-power8-builders-for-now.patch --]
[-- Type: text/x-diff, Size: 2958 bytes --]

From d9e9f402c08cf9e2a51af15f0a7f5f4a011a9594 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sun, 16 Oct 2022 18:53:11 +0200
Subject: [PATCH] Disable {gdb,valgrind}-ibm-power8 builders for now

power8 is down, being rebuilt, etc.  it should be back after a time.
---
 builder/master.cfg | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 61bf4f1..a2d0ab1 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -584,7 +584,7 @@ valgrind_scheduler = schedulers.SingleBranchScheduler(
                       "valgrind-debian-testing-x86_64",
                       "valgrind-debian-armhf",
                       "valgrind-debian-i386",
-                      "valgrind-ibm-power8",
+#                      "valgrind-ibm-power8",
                       "valgrind-ibm-power9",
                       "valgrind-ibm-power10",
                       "valgrind-fedora-ppc64le",
@@ -762,7 +762,7 @@ gdb_scheduler = schedulers.SingleBranchScheduler(
                       "gdb-armhf-ubuntu20_04",
                       "gdb-armhf-ubuntu22_04",
                       "gdb-debian-i386",
-                      "gdb-ibm-power8",
+#                      "gdb-ibm-power8",
                       "gdb-ibm-power9",
                       "gdb-ibm-power10",
                       "gdb-fedora-s390x",
@@ -2708,12 +2708,12 @@ gdb_try_debian_testing_x86_64_builder = util.BuilderConfig(
         factory=gdb_factory)
 c['builders'].append(gdb_try_debian_testing_x86_64_builder)
 
-gdb_ibm_power8_builder = util.BuilderConfig(
-	name="gdb-ibm-power8",
-        workernames=["ibm_power8"],
-        tags=["gdb", "ibm", "power8"],
-        factory=gdb_factory)
-c['builders'].append(gdb_ibm_power8_builder)
+# gdb_ibm_power8_builder = util.BuilderConfig(
+# 	name="gdb-ibm-power8",
+#         workernames=["ibm_power8"],
+#         tags=["gdb", "ibm", "power8"],
+#         factory=gdb_factory)
+# c['builders'].append(gdb_ibm_power8_builder)
 
 gdb_ibm_power9_builder = util.BuilderConfig(
 	name="gdb-ibm-power9",
@@ -3629,12 +3629,12 @@ valgrind_fedora_ppc64le_builder = util.BuilderConfig(
         factory=valgrind_make_check_aux_factory)
 c['builders'].append(valgrind_fedora_ppc64le_builder)
 
-valgrind_ibm_power8_builder = util.BuilderConfig(
-        name="valgrind-ibm-power8",
-        workernames=["ibm_power8"],
-        tags=["valgrind", "ibm", "power8"],
-        factory=valgrind_make_check_aux_factory)
-c['builders'].append(valgrind_ibm_power8_builder)
+# valgrind_ibm_power8_builder = util.BuilderConfig(
+#         name="valgrind-ibm-power8",
+#         workernames=["ibm_power8"],
+#         tags=["valgrind", "ibm", "power8"],
+#         factory=valgrind_make_check_aux_factory)
+# c['builders'].append(valgrind_ibm_power8_builder)
 
 valgrind_ibm_power9_builder = util.BuilderConfig(
         name="valgrind-ibm-power9",
-- 
2.30.2

Re: ibm_power8 and ibm_power10 workers

[will schmidt]

On Sun, 2022-10-16 at 19:05 +0200, Mark Wielaard wrote:
> Hi Will,
> 
> On Mon, Oct 10, 2022 at 12:01:26PM -0500, will schmidt wrote:
> >  Thanks for the poke.    Something non-functional with the @reboot
> > entry in the crontab on the power10.   I've started it manually, and
> > will investigate a bit to try to fix it for the long term.
> 
> The power10 builders seem fine again. Thanks.
> 
> > Our power8 is down, being rebuilt, etc.  it should be back after a
> > time.
> 
> For now I disabled the gdb and valgrind power8 builders. Otherwise
> they build queue just keeps growing. Please let us know when the
> power8 worker is back and we'll reenable them again.

I've restarted the power8 buildbot on a new host, a moment ago.  It appears to
be connected OK, and should be good to go.

Thanks
-Will


> 
> Cheers,
> 
> Mark

Re: ibm_power8 and ibm_power10 workers

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Hi Will,

On Tue, 2022-11-15 at 10:35 -0600, will schmidt wrote:
> On Sun, 2022-10-16 at 19:05 +0200, Mark Wielaard wrote:
> > > Our power8 is down, being rebuilt, etc.  it should be back after
> > > a time.
> > 
> > For now I disabled the gdb and valgrind power8 builders. Otherwise
> > they build queue just keeps growing. Please let us know when the
> > power8 worker is back and we'll reenable them again.
> 
> I've restarted the power8 buildbot on a new host, a moment ago.  It
> appears to be connected OK, and should be good to go.

Great, I enabled the gdb and valgrind builders again, as attached.

Cheers,

Mark

[-- Attachment #2: 0001-Revert-Disable-gdb-valgrind-ibm-power8-builders-for-.patch --]
[-- Type: text/x-patch, Size: 2962 bytes --]

From a2769d408fde564f05e0c986bb3900152428db71 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Tue, 15 Nov 2022 18:02:40 +0100
Subject: [PATCH] Revert "Disable {gdb,valgrind}-ibm-power8 builders for now"

This reverts commit d9e9f402c08cf9e2a51af15f0a7f5f4a011a9594.
---
 builder/master.cfg | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 072b7d0..2bc46ef 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -585,7 +585,7 @@ valgrind_scheduler = schedulers.SingleBranchScheduler(
                       "valgrind-debian-testing-x86_64",
                       "valgrind-debian-armhf",
                       "valgrind-debian-i386",
-#                      "valgrind-ibm-power8",
+                      "valgrind-ibm-power8",
                       "valgrind-ibm-power9",
                       "valgrind-ibm-power10",
                       "valgrind-fedora-ppc64le",
@@ -763,7 +763,7 @@ gdb_scheduler = schedulers.SingleBranchScheduler(
 #                      "gdb-armhf-ubuntu20_04",
 #                      "gdb-armhf-ubuntu22_04",
                       "gdb-debian-i386",
-#                      "gdb-ibm-power8",
+                      "gdb-ibm-power8",
                       "gdb-ibm-power9",
                       "gdb-ibm-power10",
                       "gdb-fedora-s390x",
@@ -2710,12 +2710,12 @@ gdb_try_debian_testing_x86_64_builder = util.BuilderConfig(
         factory=gdb_factory)
 c['builders'].append(gdb_try_debian_testing_x86_64_builder)
 
-# gdb_ibm_power8_builder = util.BuilderConfig(
-# 	name="gdb-ibm-power8",
-#         workernames=["ibm_power8"],
-#         tags=["gdb", "ibm", "power8"],
-#         factory=gdb_factory)
-# c['builders'].append(gdb_ibm_power8_builder)
+gdb_ibm_power8_builder = util.BuilderConfig(
+	name="gdb-ibm-power8",
+        workernames=["ibm_power8"],
+        tags=["gdb", "ibm", "power8"],
+        factory=gdb_factory)
+c['builders'].append(gdb_ibm_power8_builder)
 
 gdb_ibm_power9_builder = util.BuilderConfig(
 	name="gdb-ibm-power9",
@@ -3631,12 +3631,12 @@ valgrind_fedora_ppc64le_builder = util.BuilderConfig(
         factory=valgrind_make_check_aux_factory)
 c['builders'].append(valgrind_fedora_ppc64le_builder)
 
-# valgrind_ibm_power8_builder = util.BuilderConfig(
-#         name="valgrind-ibm-power8",
-#         workernames=["ibm_power8"],
-#         tags=["valgrind", "ibm", "power8"],
-#         factory=valgrind_make_check_aux_factory)
-# c['builders'].append(valgrind_ibm_power8_builder)
+valgrind_ibm_power8_builder = util.BuilderConfig(
+        name="valgrind-ibm-power8",
+        workernames=["ibm_power8"],
+        tags=["valgrind", "ibm", "power8"],
+        factory=valgrind_make_check_aux_factory)
+c['builders'].append(valgrind_ibm_power8_builder)
 
 valgrind_ibm_power9_builder = util.BuilderConfig(
         name="valgrind-ibm-power9",
-- 
2.18.4

Re: ibm_power8 and ibm_power10 workers

[Mark Wielaard]

Hi Will,

On Tue, Nov 15, 2022 at 06:05:03PM +0100, Mark Wielaard wrote:
> On Tue, 2022-11-15 at 10:35 -0600, will schmidt wrote:
> > I've restarted the power8 buildbot on a new host, a moment ago.  It
> > appears to be connected OK, and should be good to go.
> 
> Great, I enabled the gdb and valgrind builders again, as attached.

And it happily did various builds for a couple of days, but then got
some update failures and disconnected:
https://builder.sourceware.org/buildbot/#/workers/18

Could you take a look?

Thanks,

Mark

Re: ibm_power8 and ibm_power10 workers

[will schmidt]

On Thu, 2022-11-17 at 11:09 +0100, Mark Wielaard wrote:
> Hi Will,
> 
> On Tue, Nov 15, 2022 at 06:05:03PM +0100, Mark Wielaard wrote:
> > On Tue, 2022-11-15 at 10:35 -0600, will schmidt wrote:
> > > I've restarted the power8 buildbot on a new host, a moment ago.  It
> > > appears to be connected OK, and should be good to go.
> > 
> > Great, I enabled the gdb and valgrind builders again, as attached.
> 
> And it happily did various builds for a couple of days, but then got
> some update failures and disconnected:
> https://builder.sourceware.org/buildbot/#/workers/18
> 
> Could you take a look?

I typo'd the start-bb-worker in the cron @reboot stanza. fixed and manually
restarted, should be good again now. 

Thanks for the poke. :-)

-Will


> 
> Thanks,
> 
> Mark
>