public inbox for buildbot@sourceware.org
 help / color / mirror / Atom feed
* Arm GCC buildbot workers
@ 2022-08-11  7:54 Christophe Lyon
  2022-08-11 20:39 ` Mark Wielaard
  0 siblings, 1 reply; 9+ messages in thread
From: Christophe Lyon @ 2022-08-11  7:54 UTC (permalink / raw)
  To: buildbot

Hi Mark,

I'd like to start 2 more workers, for GCC: one for armhf, one for arm64.
They will be running ubuntu-22.04, 20 CPUs each.

So:
ncpus:20
maxcpus:20
max_builds:1

I'd like them to run "gcc_full", which should take ~2h according to my
experiments.

Looking at https://builder.sourceware.org/buildbot/#/workers, I suspect
there may be at least a naming problem since the other workers have
"generic names" (I mean we already have similar workers, dedicated to gdb).

BTW, I'm wondering whether it was a mistake to have our other workers
named armXX-ubuntu-YY, since the other ones are named distro-arch?

Thanks,

Christophe
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-11  7:54 Arm GCC buildbot workers Christophe Lyon
@ 2022-08-11 20:39 ` Mark Wielaard
  2022-08-12  7:23   ` Christophe Lyon
  0 siblings, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2022-08-11 20:39 UTC (permalink / raw)
  To: Christophe Lyon; +Cc: buildbot

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

Hi Christophe,

On Thu, Aug 11, 2022 at 09:54:35AM +0200, Christophe Lyon wrote:
> I'd like to start 2 more workers, for GCC: one for armhf, one for arm64.
> They will be running ubuntu-22.04, 20 CPUs each.
> 
> So:
> ncpus:20
> maxcpus:20
> max_builds:1
> 
> I'd like them to run "gcc_full", which should take ~2h according to my
> experiments.

Cool. I did add a "gcc-full" build for x86_64 last week to use one of
the workers that has 12 or 16 cores. It takes ~50 to ~140 minutes
depending on which worker it picks and the other load on the machine.

When a build takes this long you cannot really test each commit
separately. 12 patches and you have a whole day of building. So you
either only build periodically, or you collapse the pending build
requests. The existing gcc-full builders collapse the builds. So I'll
suggest these new builders do too, but let me know if you rather have
them try a build only periodically (say every 8 hours?)

> Looking at https://builder.sourceware.org/buildbot/#/workers, I suspect
> there may be at least a naming problem since the other workers have
> "generic names" (I mean we already have similar workers, dedicated to gdb).
> 
> BTW, I'm wondering whether it was a mistake to have our other workers
> named armXX-ubuntu-YY, since the other ones are named distro-arch?

Naming isn't 100% consistent, and it doesn't really matter much for
worker names. It does matter a little for the builder names, sadly
those also aren't fully consistent. But as long as we are somewhat
consitent in the tag names hopefully people can still sort interesting
builds together.

Renaming the workers would mean changing them on both sides, so lets
stick with the original names for now. Then we can name these the
other way around:

ubuntu22_04-arm64
ubuntu22_04-armhf

See the attached patch.

Cheers,

Mark

[-- Attachment #2: 0001-Add-ubuntu22_04-arm64-and-ubuntu22_04-armhf-workers-.patch --]
[-- Type: text/plain, Size: 4810 bytes --]

From 5b16ab1d7a8e0fbc3ec2218a6087ad292dba41dd Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Thu, 11 Aug 2022 22:36:42 +0200
Subject: [PATCH] Add ubuntu22_04-arm64 and ubuntu22_04-armhf workers for gcc

Two more workers from the Works on Arm initiative to do full
gcc builds (bootstrap plus make check).
---
 buildbot.config.sample |  2 ++
 builder/master.cfg     | 39 ++++++++++++++++++++++++++++++++++++++-
 htdocs/index.html.post |  4 ++--
 3 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/buildbot.config.sample b/buildbot.config.sample
index b4c3f11..7986c6a 100644
--- a/buildbot.config.sample
+++ b/buildbot.config.sample
@@ -20,6 +20,8 @@ arm64-ubuntu20_04=frob
 arm64-ubuntu22_04=frob
 armhf-ubuntu20_04=frob
 armhf-ubuntu22_04=frob
+ubuntu22_04-arm64=frob
+ubuntu22_04-armhf=frob
 fedora-arm64=frob
 
 # Users
diff --git a/builder/master.cfg b/builder/master.cfg
index 55f8d3b..f945b14 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -241,6 +241,8 @@ ibm_power10_worker = worker.Worker("ibm_power10",
                                                       'cel@us.ibm.com']);
 c['workers'].append(ibm_power10_worker)
 
+# Various arm64/armhf workers provided by the Works on Arm initiative
+# 4 used for gdb
 arm64_ubuntu20_04_worker = worker.Worker("arm64-ubuntu20_04",
                                          getpw("arm64-ubuntu20_04"),
                                          max_builds=1,
@@ -269,6 +271,23 @@ armhf_ubuntu22_04_worker = worker.Worker("armhf-ubuntu22_04",
                                          notify_on_missing=['christophe.lyon@arm.com']);
 c['workers'].append(armhf_ubuntu22_04_worker)
 
+# 2 used for gcc
+ubuntu20_04_armhf_worker = worker.Worker("ubuntu22_04-armhf",
+                                         getpw("ubuntu22_04-armhf"),
+                                         properties={'ncpus': 20,
+                                                     'maxcpus': 20},
+                                         max_builds=1,
+                                         notify_on_missing=['christophe.lyon@arm.com']);
+c['workers'].append(ubuntu20_04_armhf_worker)
+
+ubuntu20_04_arm64_worker = worker.Worker("ubuntu22_04-arm64",
+                                         getpw("ubuntu22_04-arm64"),
+                                         properties={'ncpus': 20,
+                                                     'maxcpus': 20},
+                                         max_builds=1,
+                                         notify_on_missing=['christophe.lyon@arm.com']);
+c['workers'].append(ubuntu20_04_arm64_worker)
+
 # 8 core arm64 server hosted by osuosl
 fedora_arm64_worker = worker.Worker("fedora-arm64",
                                          getpw("fedora-arm64"),
@@ -788,7 +807,9 @@ gcc_scheduler = schedulers.SingleBranchScheduler(
                                         branch="master"),
         reason="gcc project master branch update",
         builderNames=["gcc-fedrawhide-x86_64",
-                      "gcc-full-debian-amd64"])
+                      "gcc-full-debian-amd64",
+                      "gcc-full-ubuntu-armhf",
+                      "gcc-full-ubuntu-arm64"])
 c['schedulers'].append(gcc_scheduler)
 
 systemtap_scheduler = schedulers.SingleBranchScheduler(
@@ -3002,6 +3023,22 @@ gcc_full_debian_amd64_builder = util.BuilderConfig(
         factory=gcc_full_build_factory)
 c['builders'].append(gcc_full_debian_amd64_builder)
 
+gcc_full_ubuntu_armhf_builder = util.BuilderConfig(
+        name="gcc-full-ubuntu-armhf",
+        collapseRequests=True,
+        workernames=["ubuntu22_04-armhf"],
+        tags=["gcc-full", "ubuntu", "armhf"],
+        factory=gcc_full_build_factory)
+c['builders'].append(gcc_full_ubuntu_armhf_builder)
+
+gcc_full_ubuntu_arm64_builder = util.BuilderConfig(
+        name="gcc-full-ubuntu-arm64",
+        collapseRequests=True,
+        workernames=["ubuntu22_04-arm64"],
+        tags=["gcc-full", "ubuntu", "arm64"],
+        factory=gcc_full_build_factory)
+c['builders'].append(gcc_full_ubuntu_arm64_builder)
+
 
 # glibc build steps, factory, builders
 
diff --git a/htdocs/index.html.post b/htdocs/index.html.post
index 56cf675..c9767c6 100644
--- a/htdocs/index.html.post
+++ b/htdocs/index.html.post
@@ -12,8 +12,8 @@
 	<li>Frank Eigler for the fedrawhide-x86_64 worker</li>
 	<li>IBM for the ibm_power8, ibm_power9 and ibm_power10 workers</li>
 	<li>The Works on Arm initiative for the arm64-ubuntu20_04,
-            arm64-ubuntu22_04, armhf-ubuntu20_04 and
-            armhf-ubuntu22_04 workers</li>
+            arm64-ubuntu22_04, armhf-ubuntu20_04, armhf-ubuntu22_04
+            ubuntu22_04-arm64 and ubuntu22_04-armhf workers</li>
 	<li>OSUOSL for the fedora-arm64 worker</li>
       </ul>
     </p>
-- 
2.31.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-11 20:39 ` Mark Wielaard
@ 2022-08-12  7:23   ` Christophe Lyon
  2022-08-12 16:44     ` Mark Wielaard
  0 siblings, 1 reply; 9+ messages in thread
From: Christophe Lyon @ 2022-08-12  7:23 UTC (permalink / raw)
  To: Mark Wielaard; +Cc: buildbot

Hi Mark,


On 8/11/22 22:39, Mark Wielaard wrote:
> Hi Christophe,
>
> On Thu, Aug 11, 2022 at 09:54:35AM +0200, Christophe Lyon wrote:
>> I'd like to start 2 more workers, for GCC: one for armhf, one for arm64.
>> They will be running ubuntu-22.04, 20 CPUs each.
>>
>> So:
>> ncpus:20
>> maxcpus:20
>> max_builds:1
>>
>> I'd like them to run "gcc_full", which should take ~2h according to my
>> experiments.
>
> Cool. I did add a "gcc-full" build for x86_64 last week to use one of
> the workers that has 12 or 16 cores. It takes ~50 to ~140 minutes
> depending on which worker it picks and the other load on the machine.
>
> When a build takes this long you cannot really test each commit
> separately. 12 patches and you have a whole day of building. So you
> either only build periodically, or you collapse the pending build
> requests. The existing gcc-full builders collapse the builds. So I'll
> suggest these new builders do too, but let me know if you rather have
> them try a build only periodically (say every 8 hours?)
>
>> Looking at https://builder.sourceware.org/buildbot/#/workers, I suspect
>> there may be at least a naming problem since the other workers have
>> "generic names" (I mean we already have similar workers, dedicated to gdb).
>>
>> BTW, I'm wondering whether it was a mistake to have our other workers
>> named armXX-ubuntu-YY, since the other ones are named distro-arch?
>
> Naming isn't 100% consistent, and it doesn't really matter much for
> worker names. It does matter a little for the builder names, sadly
> those also aren't fully consistent. But as long as we are somewhat
> consitent in the tag names hopefully people can still sort interesting
> builds together.
>
> Renaming the workers would mean changing them on both sides, so lets
> stick with the original names for now. Then we can name these the
> other way around:
>
> ubuntu22_04-arm64
> ubuntu22_04-armhf
>
> See the attached patch.

Thanks, LGTM, the 2 new workers are now running.

Thanks!

Christophe

>
> Cheers,
>
> Mark
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-12  7:23   ` Christophe Lyon
@ 2022-08-12 16:44     ` Mark Wielaard
  2022-08-16 11:54       ` Christophe Lyon
  0 siblings, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2022-08-12 16:44 UTC (permalink / raw)
  To: Christophe Lyon; +Cc: buildbot

Hi Christophe,

On Fri, Aug 12, 2022 at 09:23:36AM +0200, Christophe Lyon wrote:
> >ubuntu22_04-arm64
> >ubuntu22_04-armhf
> >
> >See the attached patch.
> 
> Thanks, LGTM, the 2 new workers are now running.

The ubuntu22_04-arm64 worker seems to happily do full gcc bootstrap
and check builds:
https://builder.sourceware.org/buildbot/#/workers/34

It does find a couple of failing testcases, which you might want to
look at, but most seem expected. All results are recorded in bunsen to
be compared.

The ubuntu22_04-armhf worker however fails to do a full gcc build:
https://builder.sourceware.org/buildbot/#/workers/33

In file included from ../../../gcc/libgcc/../gcc/tsystem.h:87,
                 from ../../../gcc/libgcc/libgcc2.c:27:
/usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No such file or directory
   27 | #include <bits/libc-header-start.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

It looks like that happens while building libcc for
armv8l-unknown-linux-gnueabihf. Maybe missing devel header files? Or
does the build needs to be configured to exclude this target?

Cheers,

Mark

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-12 16:44     ` Mark Wielaard
@ 2022-08-16 11:54       ` Christophe Lyon
  2022-08-17 11:51         ` Christophe Lyon
  2022-08-17 13:00         ` Mark Wielaard
  0 siblings, 2 replies; 9+ messages in thread
From: Christophe Lyon @ 2022-08-16 11:54 UTC (permalink / raw)
  To: buildbot

Hi Mark,

On 8/12/22 18:44, Mark Wielaard wrote:
> Hi Christophe,
>
> On Fri, Aug 12, 2022 at 09:23:36AM +0200, Christophe Lyon wrote:
>>> ubuntu22_04-arm64
>>> ubuntu22_04-armhf
>>>
>>> See the attached patch.
>>
>> Thanks, LGTM, the 2 new workers are now running.
>
> The ubuntu22_04-arm64 worker seems to happily do full gcc bootstrap
> and check builds:
> https://builder.sourceware.org/buildbot/#/workers/34
>
> It does find a couple of failing testcases, which you might want to
> look at, but most seem expected. All results are recorded in bunsen to
> be compared.

Great, thanks! I guess the errors are the same as observed on
gcc-full-debian-arm64?

Anyway I don't think any target has a clean 'make check' in GCC :-(

This will increase the need for a regression-detection mechanism in the
buildbot, otherwise we'll only have red bullets which people will ignore....


> The ubuntu22_04-armhf worker however fails to do a full gcc build:
> https://builder.sourceware.org/buildbot/#/workers/33
>
> In file included from ../../../gcc/libgcc/../gcc/tsystem.h:87,
>                   from ../../../gcc/libgcc/libgcc2.c:27:
> /usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No such file or directory
>     27 | #include <bits/libc-header-start.h>
>        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
>
> It looks like that happens while building libcc for
> armv8l-unknown-linux-gnueabihf. Maybe missing devel header files? Or
> does the build needs to be configured to exclude this target?

So... I dug a bit and... you need to add --with-float=hard on GCC's
configure line. For some reason GCC's configure is not able to infer
that from the trailing 'hf' in the target name. Can you add it for this
target only or is the configure line shared by all GCC workers?

Thanks

Christophe

>
> Cheers,
>
> Mark
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-16 11:54       ` Christophe Lyon
@ 2022-08-17 11:51         ` Christophe Lyon
  2022-08-17 13:00         ` Mark Wielaard
  1 sibling, 0 replies; 9+ messages in thread
From: Christophe Lyon @ 2022-08-17 11:51 UTC (permalink / raw)
  To: buildbot



On 8/16/22 13:54, Christophe Lyon wrote:
> Hi Mark,
>
> On 8/12/22 18:44, Mark Wielaard wrote:
>> Hi Christophe,
>>
>> On Fri, Aug 12, 2022 at 09:23:36AM +0200, Christophe Lyon wrote:
>>>> ubuntu22_04-arm64
>>>> ubuntu22_04-armhf
>>>>
>>>> See the attached patch.
>>>
>>> Thanks, LGTM, the 2 new workers are now running.
>>
>> The ubuntu22_04-arm64 worker seems to happily do full gcc bootstrap
>> and check builds:
>> https://builder.sourceware.org/buildbot/#/workers/34
>>
>> It does find a couple of failing testcases, which you might want to
>> look at, but most seem expected. All results are recorded in bunsen to
>> be compared.
>
> Great, thanks! I guess the errors are the same as observed on
> gcc-full-debian-arm64?
>
> Anyway I don't think any target has a clean 'make check' in GCC :-(
>
> This will increase the need for a regression-detection mechanism in the
> buildbot, otherwise we'll only have red bullets which people will
> ignore....
>
>
>> The ubuntu22_04-armhf worker however fails to do a full gcc build:
>> https://builder.sourceware.org/buildbot/#/workers/33
>>
>> In file included from ../../../gcc/libgcc/../gcc/tsystem.h:87,
>>                   from ../../../gcc/libgcc/libgcc2.c:27:
>> /usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No
>> such file or directory
>>     27 | #include <bits/libc-header-start.h>
>>        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
>> compilation terminated.
>>
>> It looks like that happens while building libcc for
>> armv8l-unknown-linux-gnueabihf. Maybe missing devel header files? Or
>> does the build needs to be configured to exclude this target?
>
> So... I dug a bit and... you need to add --with-float=hard on GCC's
> configure line. For some reason GCC's configure is not able to infer
> that from the trailing 'hf' in the target name. Can you add it for this
> target only or is the configure line shared by all GCC workers?
>

Finally this shouldn't be needed since I managed to commit a fix in GCC
for that problem. (I didn't think it could be fixed so smoothly ;-) )
So, once the buildbot takes my commit into account, it should build.

https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=06bca6950b154d7c85762acf1593d6792f8a2617

Thanks


> Thanks
>
> Christophe
>
>>
>> Cheers,
>>
>> Mark
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-16 11:54       ` Christophe Lyon
  2022-08-17 11:51         ` Christophe Lyon
@ 2022-08-17 13:00         ` Mark Wielaard
  2022-08-17 13:45           ` Christophe Lyon
  1 sibling, 1 reply; 9+ messages in thread
From: Mark Wielaard @ 2022-08-17 13:00 UTC (permalink / raw)
  To: Christophe Lyon; +Cc: buildbot

[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]

Hi Christophe,

On Tue, Aug 16, 2022 at 01:54:00PM +0200, Christophe Lyon wrote:
> On 8/12/22 18:44, Mark Wielaard wrote:
> > The ubuntu22_04-arm64 worker seems to happily do full gcc bootstrap
> > and check builds:
> > https://builder.sourceware.org/buildbot/#/workers/34
> > 
> > It does find a couple of failing testcases, which you might want to
> > look at, but most seem expected. All results are recorded in bunsen to
> > be compared.
> 
> Great, thanks! I guess the errors are the same as observed on
> gcc-full-debian-arm64?

I assume so, but don't have internet access at the moment to check.
You can use the bunsen results database to compare different builds:
https://builder.sourceware.org/testruns/

> Anyway I don't think any target has a clean 'make check' in GCC :-(
> 
> This will increase the need for a regression-detection mechanism in the
> buildbot, otherwise we'll only have red bullets which people will ignore....

Yes. I am hoping we can use the bunsen results database to check for
regressions. We have the current build number of the builder, so we
should be able to pull the bunsen results of the previous build. Then
we should be able to compare the dejagnu summary.

> > The ubuntu22_04-armhf worker however fails to do a full gcc build:
> > https://builder.sourceware.org/buildbot/#/workers/33
> > 
> > In file included from ../../../gcc/libgcc/../gcc/tsystem.h:87,
> >                   from ../../../gcc/libgcc/libgcc2.c:27:
> > /usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No such file or directory
> >     27 | #include <bits/libc-header-start.h>
> >        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
> > compilation terminated.
> > 
> > It looks like that happens while building libcc for
> > armv8l-unknown-linux-gnueabihf. Maybe missing devel header files? Or
> > does the build needs to be configured to exclude this target?
> 
> So... I dug a bit and... you need to add --with-float=hard on GCC's
> configure line. For some reason GCC's configure is not able to infer
> that from the trailing 'hf' in the target name. Can you add it for this
> target only or is the configure line shared by all GCC workers?

Currently the configure line is shared by all gcc-full builders. But
that can be changed. As in the attached patch (apologies for my python
skills). I'll push it once I have internet access again.

Cheers,

Mark

[-- Attachment #2: 0001-Add-gcc_full_build_factory_gen-function.patch --]
[-- Type: text/x-diff, Size: 4719 bytes --]

From dc366c58f2f63cc5aa4955f60ba2c337203c05d0 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Wed, 17 Aug 2022 14:58:40 +0200
Subject: [PATCH] Add gcc_full_build_factory_gen function

Call it with extra_configure_arg="--with-float=hard" for the
gcc_full_ubuntu_armhf_builder.
---
 builder/master.cfg | 62 +++++++++++++++++++++++++---------------------
 1 file changed, 34 insertions(+), 28 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index f945b14..dc51fd0 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -2987,31 +2987,36 @@ gcc_debian_amd64_builder = util.BuilderConfig(
         factory=gcc_build_factory)
 c['builders'].append(gcc_debian_amd64_builder)
 
-gcc_full_build_factory = util.BuildFactory()
-gcc_full_build_factory.addStep(gcc_build_git_step)
-gcc_full_build_factory.addStep(gcc_rm_build_step)
-gcc_full_build_configure_step = steps.Configure(
-        workdir='gcc-build',
-        command=['../gcc/configure',],
-        name='configure',
-        haltOnFailure=True)
-gcc_full_build_factory.addStep(gcc_full_build_configure_step)
-gcc_full_build_factory.addStep(steps.Compile(
-        workdir='gcc-build',
-        command=['make', util.Interpolate('-j%(prop:maxcpus)s')],
-        name='make',
-        haltOnFailure=True))
-# We want parallelism to get through this as quickly as possible.
-# Even if that means bunsen gets some parallel/duplicate log files
-gcc_full_build_factory.addStep(steps.Test(
-        workdir='gcc-build',
-        command=['make', 'check', util.Interpolate('-j%(prop:maxcpus)s')],
-        name='make check',
-        haltOnFailure=False, flunkOnFailure=True))
-gcc_full_build_factory.addSteps(bunsen_logfile_upload_cpio_steps(
-        ["*.log", "*.sum"],
-        workdir='gcc-build'))
-gcc_full_build_factory.addStep(gcc_rm_build_step)
+def gcc_full_build_factory_gen(extra_configure_arg=None):
+        gcc_full_build_factory = util.BuildFactory()
+        gcc_full_build_factory.addStep(gcc_build_git_step)
+        gcc_full_build_factory.addStep(gcc_rm_build_step)
+        configure_command = ['../gcc/configure']
+        if extra_configure_arg:
+                configure_command.append(extra_configure_arg)
+        gcc_full_build_configure_step = steps.Configure(
+                workdir='gcc-build',
+                command=configure_command,
+                name='configure',
+                haltOnFailure=True)
+        gcc_full_build_factory.addStep(gcc_full_build_configure_step)
+        gcc_full_build_factory.addStep(steps.Compile(
+                workdir='gcc-build',
+                command=['make', util.Interpolate('-j%(prop:maxcpus)s')],
+                name='make',
+                haltOnFailure=True))
+        # We want parallelism to get through this as quickly as possible.
+        # Even if that means bunsen gets some parallel/duplicate log files
+        gcc_full_build_factory.addStep(steps.Test(
+                workdir='gcc-build',
+                command=['make', 'check', util.Interpolate('-j%(prop:maxcpus)s')],
+                name='make check',
+                haltOnFailure=False, flunkOnFailure=True))
+        gcc_full_build_factory.addSteps(bunsen_logfile_upload_cpio_steps(
+                ["*.log", "*.sum"],
+                workdir='gcc-build'))
+        gcc_full_build_factory.addStep(gcc_rm_build_step)
+        return gcc_full_build_factory
 
 gcc_full_debian_amd64_builder = util.BuilderConfig(
         name="gcc-full-debian-amd64",
@@ -3020,7 +3025,7 @@ gcc_full_debian_amd64_builder = util.BuilderConfig(
                     readContainerFile('debian-stable')},
         workernames=big_vm_workers,
         tags=["gcc-full", "debian", "x86_64"],
-        factory=gcc_full_build_factory)
+        factory=gcc_full_build_factory_gen())
 c['builders'].append(gcc_full_debian_amd64_builder)
 
 gcc_full_ubuntu_armhf_builder = util.BuilderConfig(
@@ -3028,7 +3033,8 @@ gcc_full_ubuntu_armhf_builder = util.BuilderConfig(
         collapseRequests=True,
         workernames=["ubuntu22_04-armhf"],
         tags=["gcc-full", "ubuntu", "armhf"],
-        factory=gcc_full_build_factory)
+        factory=gcc_full_build_factory_gen(
+                extra_configure_arg="--with-float=hard"))
 c['builders'].append(gcc_full_ubuntu_armhf_builder)
 
 gcc_full_ubuntu_arm64_builder = util.BuilderConfig(
@@ -3036,7 +3042,7 @@ gcc_full_ubuntu_arm64_builder = util.BuilderConfig(
         collapseRequests=True,
         workernames=["ubuntu22_04-arm64"],
         tags=["gcc-full", "ubuntu", "arm64"],
-        factory=gcc_full_build_factory)
+        factory=gcc_full_build_factory_gen())
 c['builders'].append(gcc_full_ubuntu_arm64_builder)
 
 
-- 
2.30.2


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-17 13:00         ` Mark Wielaard
@ 2022-08-17 13:45           ` Christophe Lyon
  2022-08-17 20:42             ` Mark Wielaard
  0 siblings, 1 reply; 9+ messages in thread
From: Christophe Lyon @ 2022-08-17 13:45 UTC (permalink / raw)
  To: buildbot



On 8/17/22 15:00, Mark Wielaard wrote:
> Hi Christophe,
>
> On Tue, Aug 16, 2022 at 01:54:00PM +0200, Christophe Lyon wrote:
>> On 8/12/22 18:44, Mark Wielaard wrote:
>>> The ubuntu22_04-arm64 worker seems to happily do full gcc bootstrap
>>> and check builds:
>>> https://builder.sourceware.org/buildbot/#/workers/34
>>>
>>> It does find a couple of failing testcases, which you might want to
>>> look at, but most seem expected. All results are recorded in bunsen to
>>> be compared.
>>
>> Great, thanks! I guess the errors are the same as observed on
>> gcc-full-debian-arm64?
>
> I assume so, but don't have internet access at the moment to check.
> You can use the bunsen results database to compare different builds:
> https://builder.sourceware.org/testruns/
>
>> Anyway I don't think any target has a clean 'make check' in GCC :-(
>>
>> This will increase the need for a regression-detection mechanism in the
>> buildbot, otherwise we'll only have red bullets which people will ignore....
>
> Yes. I am hoping we can use the bunsen results database to check for
> regressions. We have the current build number of the builder, so we
> should be able to pull the bunsen results of the previous build. Then
> we should be able to compare the dejagnu summary.
>
>>> The ubuntu22_04-armhf worker however fails to do a full gcc build:
>>> https://builder.sourceware.org/buildbot/#/workers/33
>>>
>>> In file included from ../../../gcc/libgcc/../gcc/tsystem.h:87,
>>>                    from ../../../gcc/libgcc/libgcc2.c:27:
>>> /usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No such file or directory
>>>      27 | #include <bits/libc-header-start.h>
>>>         |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
>>> compilation terminated.
>>>
>>> It looks like that happens while building libcc for
>>> armv8l-unknown-linux-gnueabihf. Maybe missing devel header files? Or
>>> does the build needs to be configured to exclude this target?
>>
>> So... I dug a bit and... you need to add --with-float=hard on GCC's
>> configure line. For some reason GCC's configure is not able to infer
>> that from the trailing 'hf' in the target name. Can you add it for this
>> target only or is the configure line shared by all GCC workers?
>
> Currently the configure line is shared by all gcc-full builders. But
> that can be changed. As in the attached patch (apologies for my python
> skills). I'll push it once I have internet access again.
>

In case you have missed my follow-up message, the buildbot patch is not
needed anymore: I pushed a fix in GCC instead :-)

Thanks,

Christophe

> Cheers,
>
> Mark
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Arm GCC buildbot workers
  2022-08-17 13:45           ` Christophe Lyon
@ 2022-08-17 20:42             ` Mark Wielaard
  0 siblings, 0 replies; 9+ messages in thread
From: Mark Wielaard @ 2022-08-17 20:42 UTC (permalink / raw)
  To: Christophe Lyon; +Cc: buildbot

[-- Attachment #1: Type: text/plain, Size: 867 bytes --]

Hi Christophe,

On Wed, Aug 17, 2022 at 03:45:07PM +0200, Christophe Lyon wrote:
> > Currently the configure line is shared by all gcc-full builders. But
> > that can be changed. As in the attached patch (apologies for my python
> > skills). I'll push it once I have internet access again.
> 
> In case you have missed my follow-up message, the buildbot patch is not
> needed anymore: I pushed a fix in GCC instead :-)

Yes, sorry. I am on vacation in the mountains with spotty interweb
access, so sending/receiving emails in batches. Our emails crossed.

I like your gcc patch very much. It is much better if configure does
the right thing by default.

I removed the extra configure argument for the
gcc_full_ubuntu_armhf_builder. But kept the generator function just in
case we do want to have different configure arguments per builder in
the future.

Cheers,

Mark

[-- Attachment #2: 0001-Remove-with-float-hard-for-gcc_full_ubuntu_armhf_bui.patch --]
[-- Type: text/x-diff, Size: 964 bytes --]

From f4711d650a5f706d599a3122e6e54ca4f2cc2286 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Wed, 17 Aug 2022 22:39:28 +0200
Subject: [PATCH] Remove --with-float=hard for gcc_full_ubuntu_armhf_builder

gcc configure should now do the right thing automatically.
---
 builder/master.cfg | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index dc51fd0..318adad 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -3033,8 +3033,7 @@ gcc_full_ubuntu_armhf_builder = util.BuilderConfig(
         collapseRequests=True,
         workernames=["ubuntu22_04-armhf"],
         tags=["gcc-full", "ubuntu", "armhf"],
-        factory=gcc_full_build_factory_gen(
-                extra_configure_arg="--with-float=hard"))
+        factory=gcc_full_build_factory_gen())
 c['builders'].append(gcc_full_ubuntu_armhf_builder)
 
 gcc_full_ubuntu_arm64_builder = util.BuilderConfig(
-- 
2.30.2


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-08-17 20:42 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-11  7:54 Arm GCC buildbot workers Christophe Lyon
2022-08-11 20:39 ` Mark Wielaard
2022-08-12  7:23   ` Christophe Lyon
2022-08-12 16:44     ` Mark Wielaard
2022-08-16 11:54       ` Christophe Lyon
2022-08-17 11:51         ` Christophe Lyon
2022-08-17 13:00         ` Mark Wielaard
2022-08-17 13:45           ` Christophe Lyon
2022-08-17 20:42             ` Mark Wielaard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).