Adding binutils to the GNU Toolchain buildbot on sourceware

Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]

Hi,

I was talking to Nick what the best way was to add binutils to the
buildbot at https://builder.sourceware.org/. We came up with the
following which tests a minimal setup on a somewhat older but stable
distro:

- Debian 10, x86_64 which has Linux 4.19.235, GLIBC 2.28, gcc 8.3.0
  GNU and Binutils 2.31.1

- Only triggers a build when any files under bfd/ binutils/ gas/ ld/
  gold/ elfcpp/ include/ libiberty/ opcodes/ or the top-level
  configure or Makefile.in changes.

- configure --enable-gold --enable-shared --enable-target=all

- make all-gas all-ld all-binutils all-gold

- make check-ld check-gas check-binutils

- log files ld/ld.{log,sum} gas/testsuite/gas.{log,sum}
  binutils/binutils.{log,sum}

- Sents email to this list and any committers when the build changes
  from sucess to failure (or back).

All of this can of course be changed/extended.
There are other distros/arches available:
https://builder.sourceware.org/buildbot/#/workers
And we hope to add container based workers to support even more
distros.

We only build a smaller set of tools to make sure you get a
notification within 10 minutes if anything was broken (or fixed). We
do build gold, but don't test it because the testsuite isn't
zero-fail.

If you want Frank also has scripts to add the log files to bunsen.

The intention was that this was a zero-fail testsuite subset
(gas,ld,binutils) but we did find two ld test failures:
FAIL: Run p_align-1b with PIE
FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE
These tests seem to rely on a specific glibc version/bug fix. Nick is
working on a fix.

Please let me know if any if this should be changed. The intention is
that the buildbot will help with maintaining binutils, keep it
building and zero-fail. If it isn't doing that, but get annoying let
change the configuration.

Cheers,

Mark

[-- Attachment #2: 0001-Add-binutils-as-project-and-documentation-on-adding-.patch --]
[-- Type: text/x-diff, Size: 10863 bytes --]

From 7e67984ba110dad6bafea0b22a56584494f11c56 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Mon, 25 Apr 2022 11:02:30 +0200
Subject: [PATCH] Add binutils as project and documentation on adding a new
 project

---
 README             | 90 +++++++++++++++++++++++++++++++++++++++++-----
 TODO               |  7 ++--
 builder/master.cfg | 90 ++++++++++++++++++++++++++++++++++++++++++++++
 htdocs/index.html  |  5 +++
 4 files changed, 180 insertions(+), 12 deletions(-)

diff --git a/README b/README
index 3dda795..3afc8f2 100644
--- a/README
+++ b/README
@@ -8,13 +8,87 @@ builder/master.cfg: buildbot configuration
 htdocs: document root of https://builder.sourceware.org/
 
 - How to add a new project
-  ... Add a change_source
-  ... Add a scheduler
-  ... Add a builder, see below
-  ... Add a mail notifier
-
-- How to add a new builder
-  ...
+  - Define a repourl to be used in the GitPoller and any Build Steps
+    project_repourl = 'git://sourceware.org/git/project.git'
+
+  - Define a change source, e.g. a GitPoller and add it
+    project_gitpoller = changes.GitPoller(repourl=project_repourl,
+                                          branches=['main'],
+                                          pollInterval=227,
+                                          project='project')
+    c['change_source'].append(project_gitpoller)
+
+    You can make it listen to multiple branches, use a pollInterval
+    not used by any other poller. Note that one repourl can only be
+    used once, if projects share a git repo the schedulers for the
+    project can select which files in the repo belong to which project.
+    See for example the gccrs or binutils-gdb schedulers.
+
+  - Add a scheduler
+    project_scheduler = schedulers.SingleBranchScheduler(
+        name="project",
+	change_filter=util.ChangeFilter(project="project",
+                                        branch="master"),
+        builderNames=["project-distro-arch", ...])
+    c['schedulers'].append(project_scheduler)
+
+    You might want to add a isImportantFile filter if not
+    all file changes must trigger a build (see gccrust or binutils
+    for examples).
+
+  - Add a builder, factory and build steps see below...
+
+  - Add a mail notifier
+    In general it is a good idea to only sent email for all
+    the builders from a scheduler combined based on the scheduler
+    project tag and only for changes (from good to bad or bad to good):
+
+   generator_project = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['project'])
+   mn_project = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=True, # The patch committers.
+        extraRecipients=['project-list@sourceware.org'],
+        generators=[generator_project])
+   c['services'].append(mn_project)
+
+- How to add a new builder/fectory/steps
+  Asssuming there is already a project defined as above with a
+  repourl, change-source, scheduler and (mail) notifiers
+
+  The scheduler lists one or more builder names, each builder
+  defines a factory that generates the build steps, one or more
+  workers to execute the steps and defines tags which can be used
+  by reporters.
+
+  project_distro_arch_builder = util.BuilderConfig(
+       name="binutils-debian-amd64",
+       workernames=["distro-arch"],
+       tags=["project", "distro", "arch"],
+       factory=project_factory)
+  c['builders'].append(project__builder)
+
+  There are a couple of common steps that might be useful to
+  reuse in you factory. A factory using common steps look something
+  like:
+
+  project_factory = util.BuildFactory()
+  project_factory.addStep(steps.Git(
+        repourl=project_repourl,
+        mode='full', method='fresh',
+        name="git checkout",
+        haltOnFailure=True))
+  project_factory.addStep(autoreconf_step)
+  project_factory.addStep(configure_step)
+  project_factory.addStep(make_step)
+  project_factory.addStep(make_check_test_suite_step)
+
+  But often you need to tweak the steps a little.
+
+  Finally add the new builder to htdocs/index.html under the right project
+
+  <td><a href="/buildbot/#builders/project-distro-arch">distro-arch<br>
+      <img src="/buildbot/badges/project-distro-arch.svg"></a></td>
 
 - How to add a new worker
   ...
@@ -118,4 +192,4 @@ Commit the changes. NOTE for actually upgrading the virtual-env on
 sourceware you will need a shell account and run
 /sourceware/builder/bin/buildbot-upgrade.sh as user builder.  This
 will shutdown the builder, do the above updates based on the
-buildbot.requirements file, upgrade-master and restart the builder.
\ No newline at end of file
+buildbot.requirements file, upgrade-master and restart the builder.
diff --git a/TODO b/TODO
index 495ac99..fe82827 100644
--- a/TODO
+++ b/TODO
@@ -1,11 +1,10 @@
-- More documentation on adding builders and workers.
-  Can use the debian-ppc64/fedora-ppc64 and binutils/gdb builders projects
-  as examples.
+- More documentation on adding  workers.
+  Can use the debian-ppc64/fedora-ppc64 as examples.
 
 - We can deprecate the fedora-ppc64 worker if the debian-ppc64 works out.
   fedora-ppc64 is slow, no working ccache and runs unsupported Fedora 22.
 
-- Add a binutils builder (start small).
+- Generate the "dashboard" in htdocs/index.html automatically.
 
 - Integrate parts of the old gdb buildbot
   Old GDB buildbot documentation and config files
diff --git a/builder/master.cfg b/builder/master.cfg
index 5e8b544..e66e60a 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -201,6 +201,13 @@ gccrust_gitpoller = changes.GitPoller(repourl=gccrust_repourl,
                                       project='gccrust')
 c['change_source'].append(gccrust_gitpoller)
 
+binutils_gdb_repourl='git://sourceware.org/git/binutils-gdb.git'
+binutils_gdb_gitpoller = changes.GitPoller(repourl=binutils_gdb_repourl,
+                                           branches=['main'],
+                                           pollInterval=227,
+                                           project='binutils-gdb')
+c['change_source'].append(binutils_gdb_gitpoller)
+
 ####### SCHEDULERS
 
 # Configure the Schedulers, which decide how to react to incoming changes.
@@ -338,6 +345,29 @@ gccrust_scheduler = schedulers.SingleBranchScheduler(
                       "gccrust-debian-ppc64"])
 c['schedulers'].append(gccrust_scheduler)
 
+# Only trigger scheduler for changes to binutils (or deps)
+binutils_files = ["bfd/",
+                  "binutils/", "gas/", "ld/",
+                  "gold/", "elfcpp/",
+                  "include/", "libiberty/", "opcodes/",
+                  "configure", "Makefile.in"]
+
+def binutilsImportant(change):
+  for file in change.files:
+    for pattern in binutils_files:
+      match = re.match(pattern, file)
+      if match:
+        return True
+  return False
+
+binutils_scheduler = schedulers.SingleBranchScheduler(
+        name="binutils",
+        change_filter=util.ChangeFilter(project="binutils",
+                                        branch="master"),
+        fileIsImportant=binutilsImportant,
+        builderNames=["binutils-debian-amd64"])
+c['schedulers'].append(binutils_scheduler)
+
 ####### BUILDERS
 
 # The 'builders' list defines the Builders, which tell Buildbot how to perform a build:
@@ -1021,6 +1051,56 @@ gccrust_debian_ppc64_builder = util.BuilderConfig(
         factory=gccrust_factory)
 c['builders'].append(gccrust_debian_ppc64_builder)
 
+# binutils build steps, factory and builders
+
+binutils_factory = util.BuildFactory()
+binutils_factory.addStep(steps.Git(
+        workdir='binutils-gdb',
+        repourl=binutils_gdb_repourl,
+        mode='full', method='fresh',
+        name="git checkout",
+        haltOnFailure=True))
+binutils_factory.addStep(steps.ShellCommand(
+        command=["rm", "-rf",
+                 util.Interpolate ("%(prop:builddir)s/binutils-build")],
+        name="rm -rf binutils-build",
+        haltOnFailure=True))
+binutils_factory.addStep(steps.Configure(
+        workdir='binutils-build',
+        command=['../binutils-gdb/configure',
+                 '--enable-gold',
+                 '--enable-shared',
+                 '--enable-target=all'],
+        name='configure',
+        haltOnFailure=True))
+binutils_factory.addStep(steps.Compile(
+        workdir='binutils-build',
+        command=['make',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'all-gas', 'all-ld', 'all-binutils', 'all-gold'],
+        name='make',
+        haltOnFailure=True))
+binutils_factory.addStep(steps.Compile(
+        workdir='binutils-build',
+        command=['make',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'check-ld', 'check-gas', 'check-binutils'],
+        name='make check',
+        logfiles={ "ld.sum": "ld/ld.sum",
+                   "ld.log": "ld/ld.log",
+                   "gas.sum": "gas/testsuite/gas.sum",
+                   "gas.log": "gas/testsuite/gas.log",
+                   "binutils.sum": "binutils/binutils.sum",
+                   "binutils.log": "binutils/binutils.log" },
+        haltOnFailure=True))
+
+binutils_debian_amd64_builder = util.BuilderConfig(
+	name="binutils-debian-amd64",
+        workernames=["debian-amd64"],
+        tags=["binutils", "debian", "x86_64"],
+        factory=binutils_factory)
+c['builders'].append(binutils_debian_amd64_builder)
+
 # libabigail build steps, factory and builders
 
 libabigail_package_name_step = steps.SetProperty(
@@ -1309,6 +1389,16 @@ mn_gccrust = reporters.MailNotifier(
         generators=[generator_gccrust])
 c['services'].append(mn_gccrust)
 
+# Report for the whole binutils tagged builder set
+generator_binutils = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['binutils'])
+mn_binutils = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=True,
+        extraRecipients=['binutils@sourceware.org'],
+        generators=[generator_binutils])
+c['services'].append(mn_binutils)
+
 # Report for the whole libabigail tagged builder set
 generator_libabigail = reporters.BuildSetStatusGenerator(
         mode=('change',), tags=['libabigail'])
diff --git a/htdocs/index.html b/htdocs/index.html
index 9a2d64e..9d12fb9 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -30,6 +30,11 @@
     <h2>Projects</h2>
     <p>
       <table>
+	<tr>
+	  <td><a href="/buildbot/#/builders?tags=binutils">binutils</a></td>
+	  <td><a href="/buildbot/#builders/binutils-debian-amd64">debian-amd64<br>
+              <img src="/buildbot/badges/binutils-debian-amd64.svg"></a></td>
+	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=bzip2">bzip2</a></td>
 	  <td><a href="/buildbot/#builders/bzip2-centos-x86_64">centos-x86_64<br>
-- 
2.30.2

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Luis Machado]

On 4/25/22 10:04, Mark Wielaard wrote:
> Hi,
> 
> I was talking to Nick what the best way was to add binutils to the
> buildbot at https://builder.sourceware.org/. We came up with the
> following which tests a minimal setup on a somewhat older but stable
> distro:
> 
> - Debian 10, x86_64 which has Linux 4.19.235, GLIBC 2.28, gcc 8.3.0
>    GNU and Binutils 2.31.1
> 
> - Only triggers a build when any files under bfd/ binutils/ gas/ ld/
>    gold/ elfcpp/ include/ libiberty/ opcodes/ or the top-level
>    configure or Makefile.in changes.
> 
> - configure --enable-gold --enable-shared --enable-target=all
> 
> - make all-gas all-ld all-binutils all-gold
> 
> - make check-ld check-gas check-binutils
> 
> - log files ld/ld.{log,sum} gas/testsuite/gas.{log,sum}
>    binutils/binutils.{log,sum}
> 
> - Sents email to this list and any committers when the build changes
>    from sucess to failure (or back).
> 
> All of this can of course be changed/extended.
> There are other distros/arches available:
> https://builder.sourceware.org/buildbot/#/workers
> And we hope to add container based workers to support even more
> distros.
> 
> We only build a smaller set of tools to make sure you get a
> notification within 10 minutes if anything was broken (or fixed). We
> do build gold, but don't test it because the testsuite isn't
> zero-fail.
> 
> If you want Frank also has scripts to add the log files to bunsen.
> 
> The intention was that this was a zero-fail testsuite subset
> (gas,ld,binutils) but we did find two ld test failures:
> FAIL: Run p_align-1b with PIE
> FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE
> These tests seem to rely on a specific glibc version/bug fix. Nick is
> working on a fix.
> 
> Please let me know if any if this should be changed. The intention is
> that the buildbot will help with maintaining binutils, keep it
> building and zero-fail. If it isn't doing that, but get annoying let
> change the configuration.
> 
> Cheers,
> 
> Mark

As a general comment, I think we should have a single buildbot entry for 
the whole of binutils-gdb.

Given changes to bfd and opcodes can affect gdb, why not build gdb 
alongside the other tools? You don't need to run the gdb testsuite, 
which should make things much more deterministic.

I don't expect the build times including gdb/gdbserver/sim will be much 
longer.

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> As a general comment, I think we should have a single buildbot entry for the
> whole of binutils-gdb.

Perhaps!

> Given changes to bfd and opcodes can affect gdb, why not build gdb alongside
> the other tools? You don't need to run the gdb testsuite, which should make
> things much more deterministic.

But this is not an argument for building gdb as a part of testing binutils.
This is an argument for building & testing gdb if binutils changes.
That is: an additional buildbot job.

- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Luis Machado]

On 4/25/22 11:43, Frank Ch. Eigler wrote:
> Hi -
> 
>> As a general comment, I think we should have a single buildbot entry for the
>> whole of binutils-gdb.
> 
> Perhaps!
> 
>> Given changes to bfd and opcodes can affect gdb, why not build gdb alongside
>> the other tools? You don't need to run the gdb testsuite, which should make
>> things much more deterministic.
> 
> But this is not an argument for building gdb as a part of testing binutils.
> This is an argument for building & testing gdb if binutils changes.
> That is: an additional buildbot job

An additional buildbot job for gdb/gdbserver would be fine. It's just 
that a single build for everything is simpler, in my opinion.

Is the idea to revive the old buildbot we had for GDB, but for binutils?

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> An additional buildbot job for gdb/gdbserver would be fine. It's just that a
> single build for everything is simpler, in my opinion.

Yes, that would be interesting.

> Is the idea to revive the old buildbot we had for GDB, but for binutils?

We're welcoming the gdb project on the new buildbot server.  Just need
to set up some workers from the community.

- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Luis,

On Mon, Apr 25, 2022 at 01:16:44PM +0100, Luis Machado wrote:
> On 4/25/22 11:43, Frank Ch. Eigler wrote:
> > > As a general comment, I think we should have a single buildbot entry for the
> > > whole of binutils-gdb.
> > 
> > Perhaps!
> > 
> > > Given changes to bfd and opcodes can affect gdb, why not build gdb alongside
> > > the other tools? You don't need to run the gdb testsuite, which should make
> > > things much more deterministic.
> > 
> > But this is not an argument for building gdb as a part of testing binutils.
> > This is an argument for building & testing gdb if binutils changes.
> > That is: an additional buildbot job
> 
> An additional buildbot job for gdb/gdbserver would be fine. It's just that a
> single build for everything is simpler, in my opinion.
> 
> Is the idea to revive the old buildbot we had for GDB, but for binutils?

We could provide some of the builds that the old buildbot did for
GDB. But the new builder is not GDB specific and I hope we can learn
from the old gdb buildbot.

The problem with the old gdb buildbot is that it did too much and had
flaky test results. This caused people to not care, think the reports
were annoying, broken builds sometimes only got reported after hours or
even days.

I think we should try to keep builds/checks under 10 minutes, that the
checks should be for things that the maintainers think should always
be green. So that you get a report about something important breaking
within 10 minutes while you still know what you did. Another reason to
keep build/check times short is so you can test multiple commits per
hour.

If there are extra resources then we can also add builders that take
longer and/or run testsuites on arches/distros that are known
broken. But those would then probably not sent out reports but you
would use them to see the current status of some setup that might not
be 100% green. And that would probably mean adding more
hardware/workers.

I assume that knowing that gdb and gdbserver still build OK, without
running any tests might be important to the gdb maintainers. And that
just a build of gdb and gdbserver will take < 10 minutes on most
setups.

If so lets just pick one or more of the workers which seem like stable
distros/arches that should always build gdb:
https://builder.sourceware.org/buildbot/#/workers
(don't pick sourceware, which is special, debian-arm64, debian-armhf,
debian-i386 or fedora-ppc64 which are too slow)

Provide a file list (directories) of files in the binutils-gdb.git
repo that should trigger a build.

A configure and make line that does a quick build for just
gdb/gdbserver which should always build.

If the is a make check-something that can be executed quickly, < 5
minutes runtime, and that should always be green please include
it. But please exclude anything that takes too long, isn't known
all-green or contains flaky tests.

And the mailinglist to which to report any failing commits.
gdb-patches I assume?

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> I think we should try to keep builds/checks under 10 minutes, that the
> checks should be for things that the maintainers think should always
> be green. So that you get a report about something important breaking
> within 10 minutes while you still know what you did. Another reason to
> keep build/check times short is so you can test multiple commits per
> hour.

Can we suppress outgoing email notifications about failures?

> [...]
> If so lets just pick one or more of the workers which seem like stable
> distros/arches that should always build gdb:
> https://builder.sourceware.org/buildbot/#/workers
> [...]

I, um, kind of jumped the gun and set up a blanket native binutils-gdb
testbot here ("fedrawhide-x86_64"), just to give it a go.  Other than
bothersome outgoing notifications, it should run the full testsuites
and collect all that data into bunsen.


- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Frank,

On Mon, Apr 25, 2022 at 02:27:46PM -0400, Frank Ch. Eigler wrote:
> > I think we should try to keep builds/checks under 10 minutes, that the
> > checks should be for things that the maintainers think should always
> > be green. So that you get a report about something important breaking
> > within 10 minutes while you still know what you did. Another reason to
> > keep build/check times short is so you can test multiple commits per
> > hour.
> 
> Can we suppress outgoing email notifications about failures?

Yes, buildbot is fairly flexible. At the moment this is how the
reporter for binutils is defined [*]:

# Report for the whole binutils tagged builder set
generator_binutils = reporters.BuildSetStatusGenerator(
         mode=('change',), tags=['binutils'])
mn_binutils = reporters.MailNotifier(
        fromaddr="builder@sourceware.org",
        sendToInterestedUsers=True,
        extraRecipients=['binutils@sourceware.org'],
        generators=[generator_binutils])
c['services'].append(mn_binutils)

So the generator is for the whole buildset (all builders for a
particular source change) and sents one email for all builders with
the 'binutils' tag which have "changed" their build status. So it only
sents email when there are builders in the (binutils) buildset which
changed from succeeded to failed (or back). It doesn't sent emails for
every failed builder (if it was already failing before).

So if you want to have builders that don't trigger sending a message
then don't tag them with "binutils". You can also have a different
mode, only sent email for builds triggered by a particular named
scheduler or branches.

https://docs.buildbot.net/current/manual/configuration/report_generators/buildset.html

> > [...]
> > If so lets just pick one or more of the workers which seem like stable
> > distros/arches that should always build gdb:
> > https://builder.sourceware.org/buildbot/#/workers
> > [...]
> 
> I, um, kind of jumped the gun and set up a blanket native binutils-gdb
> testbot here ("fedrawhide-x86_64"), just to give it a go.  Other than
> bothersome outgoing notifications, it should run the full testsuites
> and collect all that data into bunsen.

Well, actions trump words. Especially since you added your own worker!
Thanks. Lets see how things work out.

Note that since you are interested gdb too you might want to add a
separate scheduler. One that triggers on the same fileset changes, but
also includes files under gdb, gdbserver, gdbsupport, gnulib,
readline... If this is what you and Luis want, then go for it.

But this is a "big builds" that generate lots of "data". And I am a
little afraid that is precisely why old buildbot CI efforts
failed. Dead by information overload.

I think the buildbot is more useful if we can (also) define "small"
(quick) builds that only report on immediately actionable results.
Build or test failures that need to be fixed asap.

Cheers,

Mark

[*] The builder configs are in a public git repo:
    https://sourceware.org/git/builder.git

    There is a README with some explation how thing should work and a
    SETUP file describing the setup. There is even a TODO file with
    ideas to make the builder even better. Patches welcome!

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Alan Modra]

On Mon, Apr 25, 2022 at 08:20:50PM +0200, Mark Wielaard wrote:
> I think we should try to keep builds/checks under 10 minutes, that the

Related to that, there are some x86_64 gas and ld tests that I find to
be annoying resource hogs when running multiple binutils target builds
in parallel.  gas/testsuite/gas/i386/rept and
ld/testsuite/ld-x86-64/pr17618 both require over 1G of memory.

HJ, would you please look at reducing the memory footprint of those
test?

-- 
Alan Modra
Australia Development Lab, IBM

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Jan Beulich]

On 26.04.2022 05:33, Alan Modra via Binutils wrote:
> On Mon, Apr 25, 2022 at 08:20:50PM +0200, Mark Wielaard wrote:
>> I think we should try to keep builds/checks under 10 minutes, that the
> 
> Related to that, there are some x86_64 gas and ld tests that I find to
> be annoying resource hogs when running multiple binutils target builds
> in parallel.  gas/testsuite/gas/i386/rept and
> ld/testsuite/ld-x86-64/pr17618 both require over 1G of memory.
> 
> HJ, would you please look at reducing the memory footprint of those
> test?

For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
the test as unreliable. I also put under question the purpose that it was
originally added for.

Jan

[1] https://sourceware.org/pipermail/binutils/2022-April/120339.html

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Nick Clifton]

Hi Jan,

> For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
> the test as unreliable. I also put under question the purpose that it was
> originally added for.
> 
> [1] https://sourceware.org/pipermail/binutils/2022-April/120339.html

Hmm, I seem to have missed this one.  In fact the entire patch series.  Sorry.
The series looks good to me, so please go ahead and apply it in its entirety

As for disabling the rept because it is so memory expensive: I think that we
used to.have an environment variable called something like RUN_EXPENSIVE_TESTS
that had to be set before certain tests were run.  Checking the sources though
I cannot find it, so maybe I am imagining things.  I still might be a good idea
though.  If we use it consistently in the binutils testsuites for the big tests
then users will probably appreciate the facility.

Cheers
   Nick

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Jan Beulich]

On 26.04.2022 14:27, Nick Clifton wrote:
>> For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
>> the test as unreliable. I also put under question the purpose that it was
>> originally added for.
>>
>> [1] https://sourceware.org/pipermail/binutils/2022-April/120339.html
> 
> Hmm, I seem to have missed this one.  In fact the entire patch series.  Sorry.
> The series looks good to me, so please go ahead and apply it in its entirety

No problem. With my new powers I had committed it already.

> As for disabling the rept because it is so memory expensive: I think that we
> used to.have an environment variable called something like RUN_EXPENSIVE_TESTS
> that had to be set before certain tests were run.  Checking the sources though
> I cannot find it, so maybe I am imagining things.  I still might be a good idea
> though.  If we use it consistently in the binutils testsuites for the big tests
> then users will probably appreciate the facility.

But a test which is run by almost nobody is more likely to break. Also I
think "expensive" has multiple dimensions (memory and time at least), and
depending on the system one may want to run (or suppress) one but not the
other kind. For the test in question, it is both memory and cycles hungry,
so there the distinction may not matter.

But I'd like to raise the question again: Is what the test was added for
actually a useful thing to test, at the risk of the test failing simply
because there's too little memory available? Iirc the problem was non-
graceful error handling. But the test does not check that the error in
question now is handled gracefully; it expects that there be no error.

Jan

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[H.J. Lu]

On Tue, Apr 26, 2022 at 6:49 AM Jan Beulich <jbeulich@suse.com> wrote:
>
> On 26.04.2022 14:27, Nick Clifton wrote:
> >> For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
> >> the test as unreliable. I also put under question the purpose that it was
> >> originally added for.
> >>
> >> [1] https://sourceware.org/pipermail/binutils/2022-April/120339.html
> >
> > Hmm, I seem to have missed this one.  In fact the entire patch series.  Sorry.
> > The series looks good to me, so please go ahead and apply it in its entirety
>
> No problem. With my new powers I had committed it already.
>
> > As for disabling the rept because it is so memory expensive: I think that we
> > used to.have an environment variable called something like RUN_EXPENSIVE_TESTS
> > that had to be set before certain tests were run.  Checking the sources though
> > I cannot find it, so maybe I am imagining things.  I still might be a good idea
> > though.  If we use it consistently in the binutils testsuites for the big tests
> > then users will probably appreciate the facility.
>
> But a test which is run by almost nobody is more likely to break. Also I
> think "expensive" has multiple dimensions (memory and time at least), and
> depending on the system one may want to run (or suppress) one but not the
> other kind. For the test in question, it is both memory and cycles hungry,
> so there the distinction may not matter.
>
> But I'd like to raise the question again: Is what the test was added for
> actually a useful thing to test, at the risk of the test failing simply
> because there's too little memory available? Iirc the problem was non-
> graceful error handling. But the test does not check that the error in
> question now is handled gracefully; it expects that there be no error.
>
>

gas/testsuite/gas/i386/rept.s failed to assemble before the fix:

commit 39a45edc073226e748566b497c216830cec119c4
Author: Alan Modra <amodra@gmail.com>
Date:   Thu Jun 7 12:47:23 2012 +0000

            PR gas/14201

We can limit it to 64-bit only if it helps.

-- 
H.J.

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Jan Beulich]

On 26.04.2022 17:47, H.J. Lu wrote:
> On Tue, Apr 26, 2022 at 6:49 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>> On 26.04.2022 14:27, Nick Clifton wrote:
>>>> For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
>>>> the test as unreliable. I also put under question the purpose that it was
>>>> originally added for.
>>>>
>>>> [1] https://sourceware.org/pipermail/binutils/2022-April/120339.html
>>>
>>> Hmm, I seem to have missed this one.  In fact the entire patch series.  Sorry.
>>> The series looks good to me, so please go ahead and apply it in its entirety
>>
>> No problem. With my new powers I had committed it already.
>>
>>> As for disabling the rept because it is so memory expensive: I think that we
>>> used to.have an environment variable called something like RUN_EXPENSIVE_TESTS
>>> that had to be set before certain tests were run.  Checking the sources though
>>> I cannot find it, so maybe I am imagining things.  I still might be a good idea
>>> though.  If we use it consistently in the binutils testsuites for the big tests
>>> then users will probably appreciate the facility.
>>
>> But a test which is run by almost nobody is more likely to break. Also I
>> think "expensive" has multiple dimensions (memory and time at least), and
>> depending on the system one may want to run (or suppress) one but not the
>> other kind. For the test in question, it is both memory and cycles hungry,
>> so there the distinction may not matter.
>>
>> But I'd like to raise the question again: Is what the test was added for
>> actually a useful thing to test, at the risk of the test failing simply
>> because there's too little memory available? Iirc the problem was non-
>> graceful error handling. But the test does not check that the error in
>> question now is handled gracefully; it expects that there be no error.
>>
>>
> 
> gas/testsuite/gas/i386/rept.s failed to assemble before the fix:
> 
> commit 39a45edc073226e748566b497c216830cec119c4
> Author: Alan Modra <amodra@gmail.com>
> Date:   Thu Jun 7 12:47:23 2012 +0000
> 
>             PR gas/14201

The problem wasn't that the file failed to assemble, but that it triggered
an assertion. With a suitably large repeat count, such a construct is
guaranteed to fail to build. And it's probably not just coincidence that
in the test you did use a repeat count only little more than half of what
the bug report had. Hence what the test should have checked for (and what
reporter as well as Alan also said) is that a proper error would result
instead of an assertion.

> We can limit it to 64-bit only if it helps.

I don't think that would make it any better.

Jan

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Nick Clifton]

Hi Jan,

> But I'd like to raise the question again: Is what the test was added for
> actually a useful thing to test, at the risk of the test failing simply
> because there's too little memory available? Iirc the problem was non-
> graceful error handling. But the test does not check that the error in
> question now is handled gracefully; it expects that there be no error.

Sorry - you are right - the test is basically flawed.  At this point I am
tempted to say that the simplest thing would be to just remove it.  It is
not really adding anything to the testsuite.  If you want to look into
crafting a more useful replacement please do, but for now I would suggest
that we simply delete it.

Cheers
  Nick

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Jan Beulich]

On 28.04.2022 14:10, Nick Clifton wrote:
>> But I'd like to raise the question again: Is what the test was added for
>> actually a useful thing to test, at the risk of the test failing simply
>> because there's too little memory available? Iirc the problem was non-
>> graceful error handling. But the test does not check that the error in
>> question now is handled gracefully; it expects that there be no error.
> 
> Sorry - you are right - the test is basically flawed.  At this point I am
> tempted to say that the simplest thing would be to just remove it.  It is
> not really adding anything to the testsuite.  If you want to look into
> crafting a more useful replacement please do, but for now I would suggest
> that we simply delete it.

H.J. did already send a patch to disable (but not delete) it.

Jan

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[H.J. Lu]

On Mon, Apr 25, 2022 at 8:33 PM Alan Modra <amodra@gmail.com> wrote:
>
> On Mon, Apr 25, 2022 at 08:20:50PM +0200, Mark Wielaard wrote:
> > I think we should try to keep builds/checks under 10 minutes, that the
>
> Related to that, there are some x86_64 gas and ld tests that I find to
> be annoying resource hogs when running multiple binutils target builds
> in parallel.  gas/testsuite/gas/i386/rept and
> ld/testsuite/ld-x86-64/pr17618 both require over 1G of memory.
>
> HJ, would you please look at reducing the memory footprint of those
> test?

These tests check if large memory footprints are handed properly.
What do you recommend?

-- 
H.J.

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Alan Modra]

On Tue, Apr 26, 2022 at 08:54:49AM -0700, H.J. Lu wrote:
> On Mon, Apr 25, 2022 at 8:33 PM Alan Modra <amodra@gmail.com> wrote:
> >
> > On Mon, Apr 25, 2022 at 08:20:50PM +0200, Mark Wielaard wrote:
> > > I think we should try to keep builds/checks under 10 minutes, that the
> >
> > Related to that, there are some x86_64 gas and ld tests that I find to
> > be annoying resource hogs when running multiple binutils target builds
> > in parallel.  gas/testsuite/gas/i386/rept and
> > ld/testsuite/ld-x86-64/pr17618 both require over 1G of memory.
> >
> > HJ, would you please look at reducing the memory footprint of those
> > test?
> 
> These tests check if large memory footprints are handed properly.
> What do you recommend?

Delete the gas test.  For the ld plt overflow test, don't use .space
in the .s file to create a huge .text section, instead use a linker
script to place sections so as to cause the address overflow.  Or just
delete the ld test too.  What future failure does it prevent?  Someone
deleting the plt overflow checks?

-- 
Alan Modra
Australia Development Lab, IBM

[PATCH] x86: Disable 2 tests with large memory requirement

[H.J. Lu]

[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]

On Tue, Apr 26, 2022 at 4:33 PM Alan Modra <amodra@gmail.com> wrote:
>
> On Tue, Apr 26, 2022 at 08:54:49AM -0700, H.J. Lu wrote:
> > On Mon, Apr 25, 2022 at 8:33 PM Alan Modra <amodra@gmail.com> wrote:
> > >
> > > On Mon, Apr 25, 2022 at 08:20:50PM +0200, Mark Wielaard wrote:
> > > > I think we should try to keep builds/checks under 10 minutes, that the
> > >
> > > Related to that, there are some x86_64 gas and ld tests that I find to
> > > be annoying resource hogs when running multiple binutils target builds
> > > in parallel.  gas/testsuite/gas/i386/rept and
> > > ld/testsuite/ld-x86-64/pr17618 both require over 1G of memory.
> > >
> > > HJ, would you please look at reducing the memory footprint of those
> > > test?
> >
> > These tests check if large memory footprints are handed properly.
> > What do you recommend?
>
> Delete the gas test.  For the ld plt overflow test, don't use .space
> in the .s file to create a huge .text section, instead use a linker
> script to place sections so as to cause the address overflow.  Or just
> delete the ld test too.  What future failure does it prevent?  Someone
> deleting the plt overflow checks?
>
>

I am disabling these tests.



--
H.J.

[-- Attachment #2: 0001-x86-Disable-2-tests-with-large-memory-requirement.patch --]
[-- Type: text/x-patch, Size: 1470 bytes --]

From 69f1317da0d9781c13dbf91a607f4bc80cfcf638 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Wed, 27 Apr 2022 11:26:44 -0700
Subject: [PATCH] x86: Disable 2 tests with large memory requirement

gas/

	* testsuite/gas/i386/i386.exp: Disable rept.

ld/

	* testsuite/ld-x86-64/x86-64.exp: Disable pr17618.
---
 gas/testsuite/gas/i386/i386.exp   | 3 ++-
 ld/testsuite/ld-x86-64/x86-64.exp | 4 ++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/gas/testsuite/gas/i386/i386.exp b/gas/testsuite/gas/i386/i386.exp
index 2c84ccc62b4..1e10277da31 100644
--- a/gas/testsuite/gas/i386/i386.exp
+++ b/gas/testsuite/gas/i386/i386.exp
@@ -726,7 +726,8 @@ run_list_test "string-bad" ""
 run_list_test "reg-bad" ""
 run_list_test "space1" "-al"
 run_list_test "xmmword" ""
-run_dump_test rept
+# Disabled due to large memory requirement.
+# run_dump_test rept
 run_dump_test pr19498
 run_list_test "nop-bad-1" ""
 run_list_test "unspec" ""
diff --git a/ld/testsuite/ld-x86-64/x86-64.exp b/ld/testsuite/ld-x86-64/x86-64.exp
index f337b485901..1a7d1eddc97 100644
--- a/ld/testsuite/ld-x86-64/x86-64.exp
+++ b/ld/testsuite/ld-x86-64/x86-64.exp
@@ -2314,8 +2314,8 @@ run_ld_link_tests [list \
     ] \
 ]
 
-# Linux only tests
-if { "$LD_CLASS" == "64bit" } then {
+# Linux only tests.  Disabled due to large memory requirement.
+if { "x$LD_CLASS" == "64bit" } then {
     # This test needs 64-bit linker.
     run_dump_test "pr17618"
 }
-- 
2.35.1

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Luis Machado]

Hi Mark,

I see Frank has put together a worker, but I just wanted to reply to 
some points.

On 4/25/22 19:20, Mark Wielaard wrote:
> Hi Luis,
> 
> On Mon, Apr 25, 2022 at 01:16:44PM +0100, Luis Machado wrote:
>> On 4/25/22 11:43, Frank Ch. Eigler wrote:
>>>> As a general comment, I think we should have a single buildbot entry for the
>>>> whole of binutils-gdb.
>>>
>>> Perhaps!
>>>
>>>> Given changes to bfd and opcodes can affect gdb, why not build gdb alongside
>>>> the other tools? You don't need to run the gdb testsuite, which should make
>>>> things much more deterministic.
>>>
>>> But this is not an argument for building gdb as a part of testing binutils.
>>> This is an argument for building & testing gdb if binutils changes.
>>> That is: an additional buildbot job
>>
>> An additional buildbot job for gdb/gdbserver would be fine. It's just that a
>> single build for everything is simpler, in my opinion.
>>
>> Is the idea to revive the old buildbot we had for GDB, but for binutils?
> 
> We could provide some of the builds that the old buildbot did for
> GDB. But the new builder is not GDB specific and I hope we can learn
> from the old gdb buildbot.
> 
> The problem with the old gdb buildbot is that it did too much and had
> flaky test results. This caused people to not care, think the reports
> were annoying, broken builds sometimes only got reported after hours or
> even days.
> 
> I think we should try to keep builds/checks under 10 minutes, that the
> checks should be for things that the maintainers think should always
> be green. So that you get a report about something important breaking
> within 10 minutes while you still know what you did. Another reason to
> keep build/check times short is so you can test multiple commits per
> hour.

I agree with the quick build idea. That's why I suggested only making 
sure gdb/gdbserver/sim builds properly. That's reasonably reliable.

Unfortunately gdb's testsuite is not too reliable. It's been improved 
over the years, but still gives quite a bit of non-deterministic results 
based on distro version/compiler version etc. So I'd leave those out in 
favor of just making sure things build properly.

If this effort can accomodate the gdb/gdbserver/sim builds, that would 
be great. I don't mean to broaden the scope of the initial proposed 
effort, but I think it is beneficial to make sure everything is building OK.

> 
> If there are extra resources then we can also add builders that take
> longer and/or run testsuites on arches/distros that are known
> broken. But those would then probably not sent out reports but you
> would use them to see the current status of some setup that might not
> be 100% green. And that would probably mean adding more
> hardware/workers.

That makes sense, but gdb-wise I think that can wait.

> 
> I assume that knowing that gdb and gdbserver still build OK, without
> running any tests might be important to the gdb maintainers. And that
> just a build of gdb and gdbserver will take < 10 minutes on most
> setups.
> 
> If so lets just pick one or more of the workers which seem like stable
> distros/arches that should always build gdb:
> https://builder.sourceware.org/buildbot/#/workers
> (don't pick sourceware, which is special, debian-arm64, debian-armhf,
> debian-i386 or fedora-ppc64 which are too slow)
> 
> Provide a file list (directories) of files in the binutils-gdb.git
> repo that should trigger a build.
> 
> A configure and make line that does a quick build for just
> gdb/gdbserver which should always build.

I think the following will do:

./configure --enable-targets=all; make all-gdb all-gdbserver

or

./configure --enable-targets=all --disable-sim; make all-gdb all-gdbserver

gdbserver, unlike gdb, will only build natively, hence why it would 
benefit from being built everywhere.

> 
> If the is a make check-something that can be executed quickly, < 5
> minutes runtime, and that should always be green please include
> it. But please exclude anything that takes too long, isn't known
> all-green or contains flaky tests.

Some tests should always work, no matter what. We could hand-pick some 
tests from the GDB testsuite that we consider critical. I don't think we 
should focus on that at this point though.

> 
> And the mailinglist to which to report any failing commits.
> gdb-patches I assume?

gdb-testers should be fine.

> 
> Cheers,
> 
> Mark
> 

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> I agree with the quick build idea. That's why I suggested only making sure
> gdb/gdbserver/sim builds properly. That's reasonably reliable.

OK, so looking for build-breaking regressions.

> Unfortunately gdb's testsuite is not too reliable. It's been improved over
> the years, but still gives quite a bit of non-deterministic results based on
> distro version/compiler version etc. So I'd leave those out in favor of just
> making sure things build properly.

This problem is why we're building out a gadget called bunsen, which
is a tool to absorb histories of testsuites, and draw statistical
conclusions.  Still early days, but noisy testsuites are not a
problem.

> > If the is a make check-something that can be executed quickly, < 5
> > minutes runtime, and that should always be green please include
> > it. But please exclude anything that takes too long, isn't known
> > all-green or contains flaky tests.
> 
> Some tests should always work, no matter what. We could hand-pick some tests
> from the GDB testsuite that we consider critical. [...]

Bunsen should help -identify- such tests.

- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1419 bytes --]

Hi Frank,

On Tue, Apr 26, 2022 at 05:40:25AM -0400, Frank Ch. Eigler via Overseers wrote:
> > Unfortunately gdb's testsuite is not too reliable. It's been improved over
> > the years, but still gives quite a bit of non-deterministic results based on
> > distro version/compiler version etc. So I'd leave those out in favor of just
> > making sure things build properly.
> 
> This problem is why we're building out a gadget called bunsen, which
> is a tool to absorb histories of testsuites, and draw statistical
> conclusions.  Still early days, but noisy testsuites are not a
> problem.

Right. And because Keith is very involved with this having builders
that feed bunsen lots of gdb testsuite results is important. Sorry if
I made it sound like "large checks" aren't important for the
builder. They are!

But I do think we should try to split the "heavy test everything
builders" from the "quick sanity check builders" a bit.

e.g. maybe we can use the fedrawhide-x86_64 worker only for the full
gdb-binutils builder and use another worker for the "quick" binutils
only builder. That way a "quick" builder doesn't get behind a couple
of "heavy" builds (which means you might get your quick result only
after a couple of hours).

How about using the other fedora-x86_64 worker (as attached)?

I tried a couple of other workers, but some were too slow, or didn't
generate a clean make check-ld results.

Cheers,

Mark

[-- Attachment #2: binutils-fedora-x86_64.patch --]
[-- Type: text/x-diff, Size: 2409 bytes --]

diff --git a/builder/master.cfg b/builder/master.cfg
index 340904f..b706401 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -123,7 +123,7 @@ debian_ppc64_worker = worker.Worker("debian-ppc64",
                                                        'fitzsim@fitzsim.org']);
 c['workers'].append(debian_ppc64_worker)
 
-# Power VM run by Tom on a Talos workstation
+# Frank's Fedora Rawhide builder
 fedrawhide_x86_64_worker = worker.Worker("fedrawhide-x86_64",
                                     getpw("fedrawhide-x86_64"),
                                     max_builds=1,
@@ -374,7 +374,8 @@ binutils_scheduler = schedulers.SingleBranchScheduler(
         change_filter=util.ChangeFilter(project="binutils-gdb",
                                         branch="master"),
         fileIsImportant=binutilsImportant,
-        builderNames=["binutils-debian-amd64", "binutils-fedrawhide-x86_64"])
+        builderNames=["binutils-debian-amd64",
+                      "binutils-fedora-x86_64"])
 c['schedulers'].append(binutils_scheduler)
 
 # Only trigger scheduler for changes to gdb (or deps)
@@ -1146,12 +1147,12 @@ binutils_debian_amd64_builder = util.BuilderConfig(
         factory=binutils_factory)
 c['builders'].append(binutils_debian_amd64_builder)
 
-binutils_fedrawhide_x86_64_builder = util.BuilderConfig(
-        name="binutils-fedrawhide-x86_64",
-        workernames=["fedrawhide-x86_64"],
+binutils_fedora_x86_64_builder = util.BuilderConfig(
+        name="binutils-fedora-x86_64",
+        workernames=["fedora-x86_64"],
         tags=["binutils", "fedora", "x86_64"],
         factory=binutils_factory)
-c['builders'].append(binutils_fedrawhide_x86_64_builder)
+c['builders'].append(binutils_fedora_x86_64_builder)
 
 # gdb build steps, factory and builders
 
diff --git a/htdocs/index.html b/htdocs/index.html
index b08a17b..d732d5d 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -34,6 +34,8 @@
 	  <td><a href="/buildbot/#/builders?tags=binutils">binutils</a></td>
 	  <td><a href="/buildbot/#builders/binutils-debian-amd64">debian-amd64<br>
               <img src="/buildbot/badges/binutils-debian-amd64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/binutils-fedora-x86_64">fedora-x86_64<br>
+              <img src="/buildbot/badges/binutils-fedora-x86_64.svg"></a></td>
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=bzip2">bzip2</a></td>

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

Hi Luis,

On Tue, Apr 26, 2022 at 08:01:33AM +0100, Luis Machado wrote:
> > If so lets just pick one or more of the workers which seem like stable
> > distros/arches that should always build gdb:
> > https://builder.sourceware.org/buildbot/#/workers
> > (don't pick sourceware, which is special, debian-arm64, debian-armhf,
> > debian-i386 or fedora-ppc64 which are too slow)
> > 
> > Provide a file list (directories) of files in the binutils-gdb.git
> > repo that should trigger a build.
> > 
> > A configure and make line that does a quick build for just
> > gdb/gdbserver which should always build.
> 
> I think the following will do:
> 
> ./configure --enable-targets=all; make all-gdb all-gdbserver
> 
> or
> 
> ./configure --enable-targets=all --disable-sim; make all-gdb all-gdbserver
> 
> gdbserver, unlike gdb, will only build natively, hence why it would benefit
> from being built everywhere.

I added a gdb-centos-x86_64 builder. That is a fairly old distro (it
is centos7) but the worker isn't used for that many other builds and
it can do a quick compile in under 5 minutes. Once it works out I'll
add some other distro/arches.

Please review the gdb important files list. I might have missed some
directoy.

Thanks,

Mark

[-- Attachment #2: 0001-Add-gdb-centos-x86_64-builder.patch --]
[-- Type: text/x-diff, Size: 4476 bytes --]

From fa23744e04367b2d6f2081fd90bc58ad6d7ff6a6 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Wed, 27 Apr 2022 00:29:38 +0200
Subject: [PATCH] Add gdb-centos-x86_64 builder

---
 builder/master.cfg | 68 ++++++++++++++++++++++++++++++++++++++++++++++
 htdocs/index.html  |  5 ++++
 2 files changed, 73 insertions(+)

diff --git a/builder/master.cfg b/builder/master.cfg
index 45066b8..340904f 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -377,7 +377,30 @@ binutils_scheduler = schedulers.SingleBranchScheduler(
         builderNames=["binutils-debian-amd64", "binutils-fedrawhide-x86_64"])
 c['schedulers'].append(binutils_scheduler)
 
+# Only trigger scheduler for changes to gdb (or deps)
+gdb_files = ["bfd/",
+             "gdb/", "gdbserver/", "gdbsupport/",
+             "gnulib/", "sim/",
+             "include/", "libiberty/", "opcodes/",
+             "configure", "Makefile.in"]
+
+def gdbImportant(change):
+  for file in change.files:
+    for pattern in gdb_files:
+      match = re.match(pattern, file)
+      if match:
+        return True
+  return False
+
+gdb_scheduler = schedulers.SingleBranchScheduler(
+        name="gdb",
+        change_filter=util.ChangeFilter(project="binutils-gdb",
+                                        branch="master"),
+        fileIsImportant=gdbImportant,
+        builderNames=["gdb-centos-x86_64"])
+c['schedulers'].append(gdb_scheduler)
 
+# A scheduler for everything binutils-gdb without filters
 binutils_gdb_scheduler = schedulers.SingleBranchScheduler(
         name="binutils-gdb",
         change_filter=util.ChangeFilter(project="binutils-gdb",
@@ -1130,6 +1153,41 @@ binutils_fedrawhide_x86_64_builder = util.BuilderConfig(
         factory=binutils_factory)
 c['builders'].append(binutils_fedrawhide_x86_64_builder)
 
+# gdb build steps, factory and builders
+
+gdb_factory = util.BuildFactory()
+gdb_factory.addStep(steps.Git(
+        workdir='binutils-gdb',
+        repourl=binutils_gdb_repourl,
+        mode='full', method='fresh',
+        name="git checkout",
+        haltOnFailure=True))
+gdb_factory.addStep(steps.ShellCommand(
+        command=["rm", "-rf",
+                 util.Interpolate ("%(prop:builddir)s/gdb-build")],
+        name="rm -rf gdb-build",
+        haltOnFailure=True))
+gdb_factory.addStep(steps.Configure(
+        workdir='gdb-build',
+        command=['../binutils-gdb/configure',
+                 '--enable-target=all'],
+        name='configure',
+        haltOnFailure=True))
+gdb_factory.addStep(steps.Compile(
+        workdir='gdb-build',
+        command=['make',
+                 util.Interpolate('-j%(prop:ncpus)s'),
+                 'all-gdb', 'all-gdbserver'],
+        name='make',
+        haltOnFailure=True))
+
+gdb_centos_x86_64_builder = util.BuilderConfig(
+	name="gdb-centos-x86_64",
+        workernames=["centos-x86_64"],
+        tags=["gdb", "centos", "x86_64"],
+        factory=gdb_factory)
+c['builders'].append(gdb_centos_x86_64_builder)
+
 
 # binutils-gdb build steps, factory and builders
 # just a native build
@@ -1468,6 +1526,16 @@ mn_binutils = reporters.MailNotifier(
         generators=[generator_binutils])
 c['services'].append(mn_binutils)
 
+# Report for the whole gdb tagged builder set
+generator_gdb = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['gdb'])
+mn_gdb = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=True,
+        extraRecipients=['gdb-testers@sourceware.org'],
+        generators=[generator_gdb])
+c['services'].append(mn_gdb)
+
 # Report for the whole libabigail tagged builder set
 generator_libabigail = reporters.BuildSetStatusGenerator(
         mode=('change',), tags=['libabigail'])
diff --git a/htdocs/index.html b/htdocs/index.html
index 9d12fb9..b08a17b 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -144,6 +144,11 @@
 	  <td><a href="/buildbot/#builders/gccrust-fedora-ppc64le">fedora-ppc64le<br>
 	      <img src="/buildbot/badges/gccrust-fedora-ppc64le.svg"></a></td>
 	</tr>
+	<tr>
+	  <td><a href="/buildbot/#/builders?tags=gdb">gdb</a></td>
+	  <td><a href="/buildbot/#builders/gdb-centos-x86_64">centos-x86_64<br>
+              <img src="/buildbot/badges/gdb-centos-x86_64.svg"></a></td>
+	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=libabigail">libabigail</a></td>
 	  <td><a href="/buildbot/#builders/libabigail-centos-x86_64">centos-x86_64<br>
-- 
2.30.2

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Luis Machado]

On 4/26/22 23:34, Mark Wielaard wrote:
> Hi Luis,
> 
> On Tue, Apr 26, 2022 at 08:01:33AM +0100, Luis Machado wrote:
>>> If so lets just pick one or more of the workers which seem like stable
>>> distros/arches that should always build gdb:
>>> https://builder.sourceware.org/buildbot/#/workers
>>> (don't pick sourceware, which is special, debian-arm64, debian-armhf,
>>> debian-i386 or fedora-ppc64 which are too slow)
>>>
>>> Provide a file list (directories) of files in the binutils-gdb.git
>>> repo that should trigger a build.
>>>
>>> A configure and make line that does a quick build for just
>>> gdb/gdbserver which should always build.
>>
>> I think the following will do:
>>
>> ./configure --enable-targets=all; make all-gdb all-gdbserver
>>
>> or
>>
>> ./configure --enable-targets=all --disable-sim; make all-gdb all-gdbserver
>>
>> gdbserver, unlike gdb, will only build natively, hence why it would benefit
>> from being built everywhere.
> 
> I added a gdb-centos-x86_64 builder. That is a fairly old distro (it
> is centos7) but the worker isn't used for that many other builds and
> it can do a quick compile in under 5 minutes. Once it works out I'll
> add some other distro/arches.
> 
> Please review the gdb important files list. I might have missed some
> directoy.
> 
> Thanks,
> 
> Mark
It would make sense to build-test gdb if any of the following changes:

bfd/
config/
gdb/
gdbserver/
gdbsupport/
include/
intl/
libbacktrace/
libctf/
libdecnumber/
libiberty/
opcodes/
readline/
sim/
zlib/

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> [...]
> It would make sense to build-test gdb if any of the following changes:
> 
> bfd/
> config/
> gdb/
> gdbserver/
> gdbsupport/
> include/
> intl/
> libbacktrace/
> libctf/
> libdecnumber/
> libiberty/
> opcodes/
> readline/
> sim/
> zlib/

(That's everything in binutils-gdb.git except binutils, cpu, gas,
gnulib, gold, gprof, gprofng, ld, texinfo.)


- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Luis Machado]

On 4/28/22 14:50, Frank Ch. Eigler wrote:
> Hi -
> 
>> [...]
>> It would make sense to build-test gdb if any of the following changes:
>>
>> bfd/
>> config/
>> gdb/
>> gdbserver/
>> gdbsupport/
>> include/
>> intl/
>> libbacktrace/
>> libctf/
>> libdecnumber/
>> libiberty/
>> opcodes/
>> readline/
>> sim/
>> zlib/
> 
> (That's everything in binutils-gdb.git except binutils, cpu, gas,
> gnulib, gold, gprof, gprofng, ld, texinfo.)
> 
> 
> - FChE
> 

Right. And now that you mentioned, I think gnulib as well. But most of 
these rarely change. do you think that is a bit too much?

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Frank Ch. Eigler]

Hi -

> Right. And now that you mentioned, I think gnulib as well. But most of these
> rarely change. do you think that is a bit too much?

I guess that's not for me to say.  (The fedora-rawhide x86-64 builder
I just started operating runs all the testsuites on all changes to
binutils-gdb, taking about 90 minutes per build.  I have no problem
with that.)

- FChE

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Frank,

On Thu, Apr 28, 2022 at 10:22:06AM -0400, Frank Ch. Eigler via Binutils wrote:
> (The fedora-rawhide x86-64 builder
> I just started operating runs all the testsuites on all changes to
> binutils-gdb, taking about 90 minutes per build.  I have no problem
> with that.)

That is of course fine if the worker maintainer (you) is OK with
that. But do note that since your worker also runs other builders,
those buildsets might have to wait for those longer builds to
finish. e.g. at the moment there is a binutils-fedrawhide-x86_64 build
waiting for 6 hours now:
https://builder.sourceware.org/buildbot/#/pendingbuildrequests

And it might be in the queue for a couple more. It will eventually get
triggered, but it means the changeset for that buildset only gets a
report on failure (change) after a couple of hours. Which is why I
would like you to consider swapping that builder to another fedora
worker that is less overloaded. That way committers to binutils will
get a report earlier if something breaks. See my proposed patch at:
https://sourceware.org/pipermail/binutils/2022-April/120561.html

BTW. If you haven't done so yet (and you have the disk space) try
installing ccache on your worker. It drastically reduces build times
since patches normally only change a few source files and the ccache
will save you from recompiling everything.

Also you can consider setting collapseRequests=True on the
binutils_gdb_fedrawhide_x86_64_builder to collapse changes into one
build to keep the number of pending builds shorter:
https://docs.buildbot.net/latest/manual/configuration/builders.html#collapsing-build-requests

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Luis,

On Thu, Apr 28, 2022 at 02:53:33PM +0100, Luis Machado wrote:
> On 4/28/22 14:50, Frank Ch. Eigler wrote:
> >(That's everything in binutils-gdb.git except binutils, cpu, gas,
> >gnulib, gold, gprof, gprofng, ld, texinfo.)
> 
> Right. And now that you mentioned, I think gnulib as well. But most
> of these rarely change. do you think that is a bit too much?

Added gnulib. I think it is important to do a quick build for any
change in files that might cause build issues with gdb/gdbserver.

BTW the newly added gdb-fedora-s390x builder showed a compile issue:
https://builder.sourceware.org/buildbot/#builders/gdb-fedora-s390x But
my c++ warning message reading foo is not strong enough to understand
what it is saying (or why it is not an issue for the other builders)

It didn't sent email yet because it isn't a new failure.

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

Hi Luis (added Tom ad Dan to CC for debian-ppc64/fedora-ppc64le),

On Thu, Apr 28, 2022 at 01:23:44PM +0100, Luis Machado wrote:
> It would make sense to build-test gdb if any of the following changes:

Thanks, updated the gdb_files list.  Also added fedora-x86_64,
debian-armhf, debian-arm64 and fedora-s390 builders.  See attached.

Tom, would it be OK to add a debian-ppc64 builder for gdb? As you can
see in the master.cfg it only does a build of gdb and gdbserver for
now, no tests are run. So it just would make sure that things
build. We'll might add tests later, if we can figure out a sane subset
that should always be green (and won't take more than a couple of
minutes to run). BTW. I am also working on adding more builders for
binutils (gas, ld, binutils) but the current testsuite doesn't seem
clean on anything than latest fedora-x86_64.

Dan, the fedora-ppc64le worker currently doesn't have enough diskspace
to add binutils and gdb builders. There is only 4GB left. I was
wondering if we should deprecate the fedora-ppc64 builder now that we
have Tom's debian-ppc64 which is much faster and has a more modern GNU
Toolchain installed. The fedora-ppc64 setup is no longer supported and
I believe we are the only users. So if we deprecate it maybe we can
repurpose some of the resources (disk space) to the fedora-ppc64le VM,
so we can add builders for binutils and gdb.

Thanks,

Mark

[-- Attachment #2: 0001-Update-gdb-files-and-fedora-x86_64-debian-armhf-arm6.patch --]
[-- Type: text/plain, Size: 4563 bytes --]

From 75a51ffbf1e47c36dd53d2e0bc42b8ec100baa8d Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Thu, 28 Apr 2022 15:58:06 +0200
Subject: [PATCH] Update gdb files and fedora-x86_64,
 debian-armhf/arm64,fedora-s390x builders

---
 builder/master.cfg | 51 ++++++++++++++++++++++++++++++++++++++++++----
 htdocs/index.html  | 10 +++++++++
 2 files changed, 57 insertions(+), 4 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 0c39365..75b2801 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -379,9 +379,20 @@ c['schedulers'].append(binutils_scheduler)
 
 # Only trigger scheduler for changes to gdb (or deps)
 gdb_files = ["bfd/",
-             "gdb/", "gdbserver/", "gdbsupport/",
-             "gnulib/", "sim/",
-             "include/", "libiberty/", "opcodes/",
+             "config/",
+             "gdb/",
+             "gdbserver/",
+             "gdbsupport/",
+             "include/",
+             "intl/",
+             "libbacktrace/",
+             "libctf/",
+             "libdecnumber/",
+             "libiberty/",
+             "opcodes/",
+             "readline/",
+             "sim/",
+             "zlib/",
              "configure", "Makefile.in"]
 
 def gdbImportant(change):
@@ -397,7 +408,11 @@ gdb_scheduler = schedulers.SingleBranchScheduler(
         change_filter=util.ChangeFilter(project="binutils-gdb",
                                         branch="master"),
         fileIsImportant=gdbImportant,
-        builderNames=["gdb-centos-x86_64"])
+        builderNames=["gdb-centos-x86_64",
+                      "gdb-fedora-x86_64",
+                      "gdb-debian-armhf",
+                      "gdb-debian-arm64",
+                      "gdb-fedora-s390x"])
 c['schedulers'].append(gdb_scheduler)
 
 # A scheduler for everything binutils-gdb without filters
@@ -1189,6 +1204,34 @@ gdb_centos_x86_64_builder = util.BuilderConfig(
         factory=gdb_factory)
 c['builders'].append(gdb_centos_x86_64_builder)
 
+gdb_fedora_x86_64_builder = util.BuilderConfig(
+	name="gdb-fedora-x86_64",
+        workernames=["fedora-x86_64"],
+        tags=["gdb", "fedora", "x86_64"],
+        factory=gdb_factory)
+c['builders'].append(gdb_fedora_x86_64_builder)
+
+gdb_debian_armhf_builder = util.BuilderConfig(
+	name="gdb-debian-armhf",
+        workernames=["debian-armhf"],
+        tags=["gdb", "debian", "armhf"],
+        factory=gdb_factory)
+c['builders'].append(gdb_debian_armhf_builder)
+
+gdb_debian_arm64_builder = util.BuilderConfig(
+	name="gdb-debian-arm64",
+        workernames=["debian-arm64"],
+        tags=["gdb", "debian", "arm64"],
+        factory=gdb_factory)
+c['builders'].append(gdb_debian_arm64_builder)
+
+gdb_fedora_s390x_builder = util.BuilderConfig(
+	name="gdb-fedora-s390x",
+        workernames=["fedora-s390x"],
+        tags=["gdb", "fedora", "s390x"],
+        factory=gdb_factory)
+c['builders'].append(gdb_fedora_s390x_builder)
+
 
 # binutils-gdb build steps, factory and builders
 # just a native build
diff --git a/htdocs/index.html b/htdocs/index.html
index b08a17b..b98b00a 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -129,8 +129,10 @@
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=gccrust">gccrust</a></td>
+<!--
 	  <td><a href="/buildbot/#builders/gccrust-debian-arm64">debian-arm64<br>
 	      <img src="/buildbot/badges/gccrust-debian-arm64.svg"></a></td>
+-->
 	  <td><a href="/buildbot/#builders/gccrust-fedora-x86_64">fedora-x86_64<br>
 	      <img src="/buildbot/badges/gccrust-fedora-x86_64.svg"></a></td>
 	  <td><a href="/buildbot/#builders/gccrust-debian-i386">debian-i386<br>
@@ -148,6 +150,14 @@
 	  <td><a href="/buildbot/#/builders?tags=gdb">gdb</a></td>
 	  <td><a href="/buildbot/#builders/gdb-centos-x86_64">centos-x86_64<br>
               <img src="/buildbot/badges/gdb-centos-x86_64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/gdb-fedora-x86_64">fedora-x86_64<br>
+              <img src="/buildbot/badges/gdb-fedora-x86_64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/gdb-debian-armhf">debian-armhf<br>
+              <img src="/buildbot/badges/gdb-debian-armhf.svg"></a></td>
+	  <td><a href="/buildbot/#builders/gdb-debian-arm64">debian-arm64<br>
+              <img src="/buildbot/badges/gdb-debian-arm64.svg"></a></td>
+	  <td><a href="/buildbot/#builders/gdb-fedora-s390x">fedora-s390x<br>
+              <img src="/buildbot/badges/gdb-fedora-s390x.svg"></a></td>
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=libabigail">libabigail</a></td>
-- 
2.27.0

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Thomas Fitzsimmons]

Hi Mark,

Mark Wielaard <mark@klomp.org> writes:

> On Thu, Apr 28, 2022 at 01:23:44PM +0100, Luis Machado wrote:
>> It would make sense to build-test gdb if any of the following changes:
>
> Thanks, updated the gdb_files list.  Also added fedora-x86_64,
> debian-armhf, debian-arm64 and fedora-s390 builders.  See attached.
>
> Tom, would it be OK to add a debian-ppc64 builder for gdb? As you can
> see in the master.cfg it only does a build of gdb and gdbserver for
> now, no tests are run. So it just would make sure that things
> build. We'll might add tests later, if we can figure out a sane subset
> that should always be green (and won't take more than a couple of
> minutes to run). BTW. I am also working on adding more builders for
> binutils (gas, ld, binutils) but the current testsuite doesn't seem
> clean on anything than latest fedora-x86_64.

OK, let's try it.  The debian-ppc64 builder has 8 GiB of free disk
space.

Thomas

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

Hi Tom,

On Thu, Apr 28, 2022 at 10:47:52AM -0400, Thomas Fitzsimmons wrote:
> Mark Wielaard <mark@klomp.org> writes:
> > On Thu, Apr 28, 2022 at 01:23:44PM +0100, Luis Machado wrote:
> >> It would make sense to build-test gdb if any of the following changes:
> >
> > Thanks, updated the gdb_files list.  Also added fedora-x86_64,
> > debian-armhf, debian-arm64 and fedora-s390 builders.  See attached.
> >
> > Tom, would it be OK to add a debian-ppc64 builder for gdb? As you can
> > see in the master.cfg it only does a build of gdb and gdbserver for
> > now, no tests are run. So it just would make sure that things
> > build. We'll might add tests later, if we can figure out a sane subset
> > that should always be green (and won't take more than a couple of
> > minutes to run). BTW. I am also working on adding more builders for
> > binutils (gas, ld, binutils) but the current testsuite doesn't seem
> > clean on anything than latest fedora-x86_64.
> 
> OK, let's try it.  The debian-ppc64 builder has 8 GiB of free disk
> space.

OK, gdb-debian-ppc64 added (see attached).

BTW. You should be able to add builders to your own worker by
commiting the master.cfg changes to the builder.git repo.

You might want to experiment with an binutils builder.  But I am still
experimenting to see which part of the testsuite actually passes on
the different distro/architectures.

Cheers,

Mark

[-- Attachment #2: 0001-Add-gdb-debian-ppc64-builder.patch --]
[-- Type: text/plain, Size: 1949 bytes --]

From c3d73b2d46b2b9e4f4e2a49bca9e4b6841c08abb Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Thu, 28 Apr 2022 18:24:09 +0200
Subject: [PATCH] Add gdb-debian-ppc64 builder

---
 builder/master.cfg | 9 ++++++++-
 htdocs/index.html  | 2 ++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index 3bf1e2a..ca123b5 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -413,7 +413,8 @@ gdb_scheduler = schedulers.SingleBranchScheduler(
                       "gdb-fedora-x86_64",
                       "gdb-debian-armhf",
                       "gdb-debian-arm64",
-                      "gdb-fedora-s390x"])
+                      "gdb-fedora-s390x",
+                      "gdb-debian-ppc64"])
 c['schedulers'].append(gdb_scheduler)
 
 # A scheduler for everything binutils-gdb without filters
@@ -1233,6 +1234,12 @@ gdb_fedora_s390x_builder = util.BuilderConfig(
         factory=gdb_factory)
 c['builders'].append(gdb_fedora_s390x_builder)
 
+gdb_debian_ppc64_builder = util.BuilderConfig(
+	name="gdb-debian-ppc64",
+        workernames=["debian-ppc64"],
+        tags=["gdb", "debian", "ppc64"],
+        factory=gdb_factory)
+c['builders'].append(gdb_debian_ppc64_builder)
 
 # binutils-gdb build steps, factory and builders
 # just a native build
diff --git a/htdocs/index.html b/htdocs/index.html
index b98b00a..9cf76b7 100644
--- a/htdocs/index.html
+++ b/htdocs/index.html
@@ -158,6 +158,8 @@
               <img src="/buildbot/badges/gdb-debian-arm64.svg"></a></td>
 	  <td><a href="/buildbot/#builders/gdb-fedora-s390x">fedora-s390x<br>
               <img src="/buildbot/badges/gdb-fedora-s390x.svg"></a></td>
+	  <td><a href="/buildbot/#builders/gdb-debian-ppc64">debian-ppc64<br>
+              <img src="/buildbot/badges/gdb-debian-ppc64.svg"></a></td>
 	</tr>
 	<tr>
 	  <td><a href="/buildbot/#/builders?tags=libabigail">libabigail</a></td>
-- 
2.27.0

binutils builder status (Was: Adding binutils to the GNU Toolchain buildbot on sourceware)

[Mark Wielaard]

Hi binutils hackers (dropped gdb),

https://builder.sourceware.org now has 6 binutils builders:
https://builder.sourceware.org/buildbot/#/builders?tags=binutils
debian-amd64, fedora-x86_64, fedora-s390x, debian-ppc64, debian-armhf
and debian-arm64.  I like to add a fedora-ppc64le one once that one
has enough disk space.

The first two are configured with --enable-targets=all. The others
not because they are too slow and/or --enable-targets=all causes
extra failures.

All build all-gas all-ld all-bintuils all-gold fine.  fedora-x86_64
and binutils-debian-ppc64 pass the whole testsuite check-gas check-ld
check-binutils.

The gas and binutils testsuites seem clean on all builders. But the ld
testsuite does see some unexpected failures or passes on some
builders. It would be great if we could fix these. If not it might
make sense to run the ld testsuite separately.

Note that you can see the used linux kernel, gcc, binutils versions on
the workers page:
https://builder.sourceware.org/buildbot/#/workers

binutils-fedora-s390x
https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-s390x
has 2 unexpected ld failures:
FAIL: Run pr19719 fun undefined
FAIL: pr26580-3

binutils-debian-amd64
https://builder.sourceware.org/buildbot/#/builders/binutils-debian-amd64
has 2 unexpected ld failures:
FAIL: Run p_align-1b with PIE
FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE

binutils-debian-arm64
https://builder.sourceware.org/buildbot/#/builders/binutils-debian-arm64
has 1 unexpected success:
XPASS: Run pr19719 fun undefined

binutils-debian-armhf
https://builder.sourceware.org/buildbot/#/builders/binutils-debian-armhf
has 8 unexpected ld failures and 7 unexpected successes
XPASS: Run pr19719 fun undefined
FAIL: Common symbol override ifunc test 1a
FAIL: Common symbol override ifunc test 1b
FAIL: Run pr18841 with libpr18841b.so
FAIL: Run pr18841 with libpr18841c.so
FAIL: Run pr18841 with libpr18841bn.so (-z now)
FAIL: Run pr18841 with libpr18841cn.so (-z now)
FAIL: Run pr23169a
FAIL: Run pr23169d
XPASS: visibility (hidden_undef) (non PIC)
XPASS: visibility (hidden_undef) (non PIC, load offset)
XPASS: visibility (hidden_undef) (PIC main, non PIC so)
XPASS: visibility (protected_undef) (non PIC)
XPASS: visibility (protected_undef) (non PIC, load offset)
XPASS: visibility (protected_undef) (PIC main, non PIC so)

The binutils-debian-armhf builder is also the slowest (takes 15
minutes). The rest take a few minutes. They should sent email once a
new failure occurs (or if one if the currently failing builders starts
passing).

There is also one build and check everything builder
https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-fedrawhide-x86_64
I haven't looked at the test results yet, but they are all stored in
the bunsendb.git for later analysis. This builder doesn't sent emails
on bad builds. It also takes a very long time to run (from 1 to 7 hours).

Cheers,

Mark

Re: binutils builder status (Was: Adding binutils to the GNU Toolchain buildbot on sourceware)

[Jan Beulich]

On 01.05.2022 18:37, Mark Wielaard wrote:
> Hi binutils hackers (dropped gdb),
> 
> https://builder.sourceware.org now has 6 binutils builders:
> https://builder.sourceware.org/buildbot/#/builders?tags=binutils
> debian-amd64, fedora-x86_64, fedora-s390x, debian-ppc64, debian-armhf
> and debian-arm64.  I like to add a fedora-ppc64le one once that one
> has enough disk space.
> 
> The first two are configured with --enable-targets=all. The others
> not because they are too slow and/or --enable-targets=all causes
> extra failures.
> 
> All build all-gas all-ld all-bintuils all-gold fine.  fedora-x86_64
> and binutils-debian-ppc64 pass the whole testsuite check-gas check-ld
> check-binutils.
> 
> The gas and binutils testsuites seem clean on all builders. But the ld
> testsuite does see some unexpected failures or passes on some
> builders. It would be great if we could fix these. If not it might
> make sense to run the ld testsuite separately.
> 
> Note that you can see the used linux kernel, gcc, binutils versions on
> the workers page:
> https://builder.sourceware.org/buildbot/#/workers
> 
> binutils-fedora-s390x
> https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-s390x
> has 2 unexpected ld failures:
> FAIL: Run pr19719 fun undefined
> FAIL: pr26580-3
> 
> binutils-debian-amd64
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-amd64
> has 2 unexpected ld failures:
> FAIL: Run p_align-1b with PIE
> FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE
> 
> binutils-debian-arm64
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-arm64
> has 1 unexpected success:
> XPASS: Run pr19719 fun undefined
> 
> binutils-debian-armhf
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-armhf
> has 8 unexpected ld failures and 7 unexpected successes
> XPASS: Run pr19719 fun undefined
> FAIL: Common symbol override ifunc test 1a
> FAIL: Common symbol override ifunc test 1b
> FAIL: Run pr18841 with libpr18841b.so
> FAIL: Run pr18841 with libpr18841c.so
> FAIL: Run pr18841 with libpr18841bn.so (-z now)
> FAIL: Run pr18841 with libpr18841cn.so (-z now)
> FAIL: Run pr23169a
> FAIL: Run pr23169d
> XPASS: visibility (hidden_undef) (non PIC)
> XPASS: visibility (hidden_undef) (non PIC, load offset)
> XPASS: visibility (hidden_undef) (PIC main, non PIC so)
> XPASS: visibility (protected_undef) (non PIC)
> XPASS: visibility (protected_undef) (non PIC, load offset)
> XPASS: visibility (protected_undef) (PIC main, non PIC so)
> 
> The binutils-debian-armhf builder is also the slowest (takes 15
> minutes). The rest take a few minutes. They should sent email once a
> new failure occurs (or if one if the currently failing builders starts
> passing).
> 
> There is also one build and check everything builder
> https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-fedrawhide-x86_64
> I haven't looked at the test results yet, but they are all stored in
> the bunsendb.git for later analysis. This builder doesn't sent emails
> on bad builds. It also takes a very long time to run (from 1 to 7 hours).

I've received the first email from that build, as it looks. May I please
ask that this be Cc-ed to individuals (rather than putting them on To: )?
I view To: as reasonable only if there was breakage which was possible to
associate with a particular commit at quite high a probability (which
would require either bisection of there being merely a single commit on
top of what was tested in the prior run).

To: otoh quite likely wants to include binutils@ ?

Thanks, Jan

Re: binutils builder status (Was: Adding binutils to the GNU Toolchain buildbot on sourceware)

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 4729 bytes --]

Hi Jan,

On Thu, May 05, 2022 at 09:53:25AM +0200, Jan Beulich wrote:
> On 01.05.2022 18:37, Mark Wielaard wrote:
> > The gas and binutils testsuites seem clean on all builders. But the ld
> > testsuite does see some unexpected failures or passes on some
> > builders. It would be great if we could fix these. If not it might
> > make sense to run the ld testsuite separately.
> > 
> > Note that you can see the used linux kernel, gcc, binutils versions on
> > the workers page:
> > https://builder.sourceware.org/buildbot/#/workers
> > 
> > binutils-fedora-s390x
> > https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-s390x
> > has 2 unexpected ld failures:
> > FAIL: Run pr19719 fun undefined
> > FAIL: pr26580-3
> > 
> > binutils-debian-amd64
> > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-amd64
> > has 2 unexpected ld failures:
> > FAIL: Run p_align-1b with PIE
> > FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE
> > 
> > binutils-debian-arm64
> > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-arm64
> > has 1 unexpected success:
> > XPASS: Run pr19719 fun undefined
> > 
> > binutils-debian-armhf
> > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-armhf
> > has 8 unexpected ld failures and 7 unexpected successes
> > XPASS: Run pr19719 fun undefined
> > FAIL: Common symbol override ifunc test 1a
> > FAIL: Common symbol override ifunc test 1b
> > FAIL: Run pr18841 with libpr18841b.so
> > FAIL: Run pr18841 with libpr18841c.so
> > FAIL: Run pr18841 with libpr18841bn.so (-z now)
> > FAIL: Run pr18841 with libpr18841cn.so (-z now)
> > FAIL: Run pr23169a
> > FAIL: Run pr23169d
> > XPASS: visibility (hidden_undef) (non PIC)
> > XPASS: visibility (hidden_undef) (non PIC, load offset)
> > XPASS: visibility (hidden_undef) (PIC main, non PIC so)
> > XPASS: visibility (protected_undef) (non PIC)
> > XPASS: visibility (protected_undef) (non PIC, load offset)
> > XPASS: visibility (protected_undef) (PIC main, non PIC so)
> > 
> > The binutils-debian-armhf builder is also the slowest (takes 15
> > minutes). The rest take a few minutes. They should sent email once a
> > new failure occurs (or if one if the currently failing builders starts
> > passing).
> > 
> > There is also one build and check everything builder
> > https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-fedrawhide-x86_64
> > I haven't looked at the test results yet, but they are all stored in
> > the bunsendb.git for later analysis. This builder doesn't sent emails
> > on bad builds. It also takes a very long time to run (from 1 to 7 hours).
> 
> I've received the first email from that build, as it looks. May I please
> ask that this be Cc-ed to individuals (rather than putting them on To: )?
> I view To: as reasonable only if there was breakage which was possible to
> associate with a particular commit at quite high a probability (which
> would require either bisection of there being merely a single commit on
> top of what was tested in the prior run).
> 
> To: otoh quite likely wants to include binutils@ ?

I think the email you received was for the "recovered"
gdb-fedora-s390x builder. This builder failed from the start since the
fix on Friday:
https://builder.sourceware.org/buildbot/#builders/gdb-fedora-s390x

The buildbot assumed everybody who had done a commit since the first
fail should hear the happy news that the build now passes. Although it
is good news, this was probably too many people.

I changed the reporters so only people who committed a change that
caused a build to go from success to failure will get a problem report
email. Then I added a new reporter that just reports to the
mailinglist for any "changed" build (from success to failure or from
failure to success).

That way you should only be on the To: when you most likely caused the
problem. There is one small issue with binutils and gdb because they
share a source repository. You might be included in a report about gdb
or binutils even if you meant to only change one or the other
project. The buildbot knows which directories are part of which
project, but some overlap and it doesn't know that a change it didn't
explicitly tests when the build for one project fails couldn't have
caused that failure.

You can write more complicated reporters in python if you want:
https://docs.buildbot.net/current/manual/configuration/report_generators/
https://docs.buildbot.net/current/manual/configuration/reporters/mail_notifier.html

And as you can see from the attached patch I am probably not a great
python programmer. If someones wants to try to clean the master.cfg
file up a little to remove all the replications, please do!

Cheers,

Mark

[-- Attachment #2: 0001-Change-mail-reports-to-only-sent-problems-to-interes.patch --]
[-- Type: text/x-diff, Size: 10265 bytes --]

From c3de0dd3c59818819b9d09c7544ca2feb370770d Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sat, 7 May 2022 22:02:49 +0200
Subject: [PATCH] Change mail reports to only sent problems to interested
 users.

Report for changes (failure to success and success to failure) go to
the mailinglist only. Do this by having two reporters. One sents email
only to interested users, the other only the the extra recipients (the
mailing list).
---
 builder/master.cfg | 145 ++++++++++++++++++++++++++++++++++++---------
 1 file changed, 118 insertions(+), 27 deletions(-)

diff --git a/builder/master.cfg b/builder/master.cfg
index c131974..988e584 100644
--- a/builder/master.cfg
+++ b/builder/master.cfg
@@ -1584,6 +1584,7 @@ c['services'].append(mn)
 
 # Sent reports about sourceware buildbot updates to overseers
 # 2022-05-02 ... don't bother overseers@ anymore about this
+# Just sent a report to the interested user (committer).
 generator_sourceware = reporters.BuildStatusGenerator(tags=["sourceware"])
 mn_sourceware = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
@@ -1591,97 +1592,187 @@ mn_sourceware = reporters.MailNotifier(
         generators=[generator_sourceware])
 c['services'].append(mn_sourceware)
 
+# Only sent problem reports to overseers
+generator_sourceware_problem = reporters.BuildStatusGenerator(
+        mode=('problem',), tags=["sourceware"])
+mn_sourceware_problem = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['overseers@sourceware.org'],
+        generators=[generator_sourceware_problem])
+c['services'].append(mn_sourceware_problem)
 
-# Report for the whole bzip2 tagged builder set
+# Problem report for the whole bzip2 tagged builder set
 generator_bzip2 = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['bzip2'])
+        mode=('problem',), tags=['bzip2'])
 mn_bzip2 = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['bzip2-devel@sourceware.org'],
         generators=[generator_bzip2])
 c['services'].append(mn_bzip2)
 
-# Report for the whole debugedit tagged builder set
+# Change report for the whole bzip2 tagged builder set
+generator_bzip2_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['bzip2'])
+mn_bzip2_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['bzip2-devel@sourceware.org'],
+        generators=[generator_bzip2_change])
+c['services'].append(mn_bzip2_change)
+
+# Problem report for the whole debugedit tagged builder set
 generator_debugedit = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['debugedit'])
+        mode=('problem',), tags=['debugedit'])
 mn_debugedit = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['debugedit@sourceware.org'],
         generators=[generator_debugedit])
 c['services'].append(mn_debugedit)
 
-# Report for the whole dwz tagged builder set
+# Change report for the whole debugedit tagged builder set
+generator_debugedit_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['debugedit'])
+mn_debugedit_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['debugedit@sourceware.org'],
+        generators=[generator_debugedit_change])
+c['services'].append(mn_debugedit_change)
+
+# Problem report for the whole dwz tagged builder set
 generator_dwz = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['dwz'])
+        mode=('problem',), tags=['dwz'])
 mn_dwz = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['dwz@sourceware.org'],
         generators=[generator_dwz])
 c['services'].append(mn_dwz)
 
-# Report for the whole elfutils tagged builder set
+# Change report for the whole dwz tagged builder set
+generator_dwz_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['dwz'])
+mn_dwz_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['dwz@sourceware.org'],
+        generators=[generator_dwz_change])
+c['services'].append(mn_dwz_change)
+
+# Problem report for the whole elfutils tagged builder set
 generator_elfutils = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['elfutils'])
+        mode=('problem',), tags=['elfutils'])
 mn_elfutils = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['elfutils-devel@sourceware.org'],
         generators=[generator_elfutils])
 c['services'].append(mn_elfutils)
 
-# Report for the whole gccrust tagged builder set
+# Change report for the whole elfutils tagged builder set
+generator_elfutils_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['elfutils'])
+mn_elfutils_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['elfutils-devel@sourceware.org'],
+        generators=[generator_elfutils_change])
+c['services'].append(mn_elfutils_change)
+
+# Problem report for the whole gccrust tagged builder set
 generator_gccrust = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['gccrust'])
+        mode=('problem',), tags=['gccrust'])
 mn_gccrust = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['gcc-rust@gcc.gnu.org'],
         generators=[generator_gccrust])
 c['services'].append(mn_gccrust)
 
-# Report for the whole binutils tagged builder set
+# Change report for the whole gccrust tagged builder set
+generator_gccrust_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['gccrust'])
+mn_gccrust_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['gcc-rust@gcc.gnu.org'],
+        generators=[generator_gccrust_change])
+c['services'].append(mn_gccrust_change)
+
+# Problem report for the whole binutils tagged builder set
 generator_binutils = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['binutils'])
+        mode=('problem',), tags=['binutils'])
 mn_binutils = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['binutils@sourceware.org'],
         generators=[generator_binutils])
 c['services'].append(mn_binutils)
 
-# Report for the whole gdb tagged builder set
+# Change report for the whole binutils tagged builder set
+generator_binutils_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['binutils'])
+mn_binutils_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['binutils@sourceware.org'],
+        generators=[generator_binutils_change])
+c['services'].append(mn_binutils_change)
+
+# Problem report for the whole gdb tagged builder set
 generator_gdb = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['gdb'])
+        mode=('problem',), tags=['gdb'])
 mn_gdb = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['gdb-testers@sourceware.org'],
         generators=[generator_gdb])
 c['services'].append(mn_gdb)
 
-# Report for the whole libabigail tagged builder set
+# Change report for the whole gdb tagged builder set
+generator_gdb_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['gdb'])
+mn_gdb_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['gdb-testers@sourceware.org'],
+        generators=[generator_gdb_change])
+c['services'].append(mn_gdb_change)
+
+# Problem report for the whole libabigail tagged builder set
 generator_libabigail = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['libabigail'])
+        mode=('problem',), tags=['libabigail'])
 mn_libabigail = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['libabigail@sourceware.org'],
         generators=[generator_libabigail])
 c['services'].append(mn_libabigail)
 
-# Report for the whole valgrind tagged builder set
+# Changereport for the whole libabigail tagged builder set
+generator_libabigail_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['libabigail'])
+mn_libabigail_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['libabigail@sourceware.org'],
+        generators=[generator_libabigail_change])
+c['services'].append(mn_libabigail_change)
+
+# Problem report for the whole valgrind tagged builder set
 generator_valgrind = reporters.BuildSetStatusGenerator(
-        mode=('change',), tags=['valgrind'])
+        mode=('problem',), tags=['valgrind'])
 mn_valgrind = reporters.MailNotifier(
         fromaddr="builder@sourceware.org",
         sendToInterestedUsers=True,
-        extraRecipients=['valgrind-testresults@lists.sourceforge.net'],
         generators=[generator_valgrind])
 c['services'].append(mn_valgrind)
 
+# Change report for the whole valgrind tagged builder set
+generator_valgrind_change = reporters.BuildSetStatusGenerator(
+        mode=('change',), tags=['valgrind'])
+mn_valgrind_change = reporters.MailNotifier(
+        fromaddr="builder@sourceware.org",
+        sendToInterestedUsers=False,
+        extraRecipients=['valgrind-testresults@lists.sourceforge.net'],
+        generators=[generator_valgrind_change])
+c['services'].append(mn_valgrind_change)
+
 
 ####### PROJECT IDENTITY
 
-- 
2.30.2

Re: binutils builder status (Was: Adding binutils to the GNU Toolchain buildbot on sourceware)

[Mark Wielaard]

Hi,

A small update. Announcing two new builders.

On Sun, 2022-05-01 at 18:37 +0200, Mark Wielaard wrote:
> https://builder.sourceware.org now has 6 binutils builders:
> https://builder.sourceware.org/buildbot/#/builders?tags=binutils
> debian-amd64, fedora-x86_64, fedora-s390x, debian-ppc64, debian-armhf
> and debian-arm64.  I like to add a fedora-ppc64le one once that one
> has enough disk space.
> 
> The first two are configured with --enable-targets=all. The others
> not because they are too slow and/or --enable-targets=all causes
> extra failures.
> 
> All build all-gas all-ld all-bintuils all-gold fine.  fedora-x86_64
> and binutils-debian-ppc64 pass the whole testsuite check-gas check-ld
> check-binutils.
> 
> The gas and binutils testsuites seem clean on all builders. But the ld
> testsuite does see some unexpected failures or passes on some
> builders. It would be great if we could fix these. If not it might
> make sense to run the ld testsuite separately.
> 
> Note that you can see the used linux kernel, gcc, binutils versions on
> the workers page:
> https://builder.sourceware.org/buildbot/#/workers
> 
> binutils-fedora-s390x
> https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-s390x
> has 2 unexpected ld failures:
> FAIL: Run pr19719 fun undefined
> FAIL: pr26580-3
> 
> binutils-debian-amd64
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-amd64
> has 2 unexpected ld failures:
> FAIL: Run p_align-1b with PIE
> FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE
> 
> binutils-debian-arm64
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-arm64
> has 1 unexpected success:
> XPASS: Run pr19719 fun undefined
> 
> binutils-debian-armhf
> https://builder.sourceware.org/buildbot/#/builders/binutils-debian-armhf
> has 8 unexpected ld failures and 7 unexpected successes
> XPASS: Run pr19719 fun undefined
> FAIL: Common symbol override ifunc test 1a
> FAIL: Common symbol override ifunc test 1b
> FAIL: Run pr18841 with libpr18841b.so
> FAIL: Run pr18841 with libpr18841c.so
> FAIL: Run pr18841 with libpr18841bn.so (-z now)
> FAIL: Run pr18841 with libpr18841cn.so (-z now)
> FAIL: Run pr23169a
> FAIL: Run pr23169d
> XPASS: visibility (hidden_undef) (non PIC)
> XPASS: visibility (hidden_undef) (non PIC, load offset)
> XPASS: visibility (hidden_undef) (PIC main, non PIC so)
> XPASS: visibility (protected_undef) (non PIC)
> XPASS: visibility (protected_undef) (non PIC, load offset)
> XPASS: visibility (protected_undef) (PIC main, non PIC so)
> 
> The binutils-debian-armhf builder is also the slowest (takes 15
> minutes). The rest take a few minutes. They should sent email once a
> new failure occurs (or if one if the currently failing builders starts
> passing).

All the above failures are still an issue, but there is now a 
binutils-fedora-ppc64le which is fully green:
https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-ppc64le

So binutils-debian-ppc64, binutils-fedora-ppc64le and binutils-fedora-
x86_64 are fully green. But binutils-debian-amd64, binutils-debian-
arm64, binutils-debian-armhf and binutils-fedora-s390x show one or more
make check failures (but all build):
https://builder.sourceware.org/buildbot/#/builders?tags=binutils

> There is also one build and check everything builder
> https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-fedrawhide-x86_64
> I haven't looked at the test results yet, but they are all stored in
> the bunsendb.git for later analysis. This builder doesn't sent emails
> on bad builds. It also takes a very long time to run (from 1 to 7
> hours).

Frank seems to have gotten a handle on this and full builds/tests now
take ~1 hour. He also added a builder that uses clang so you can easily
compare test results (in bunsendb.git, no logs yet in the buildbot
itself) against an alternative compiler:
https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-clang-fedrawhide-x86_64

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Nick Alcock]

On 28 Apr 2022, Luis Machado via Binutils outgrape:
> It would make sense to build-test gdb if any of the following changes:
[...]
> libctf/

This won't do much with gdb without a (much) newer compiler. (I
build-test gdb before every libctf commit -- every commit that I do,
anyway. I do actual gdb test runs rather less often. I should fix that,
at least for the CTF tests...)

It doesn't change anywhere near as often as GDB though so the cost of
adding it is low.

-- 
NULL && (void)

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Nick,

On Thu, Apr 28, 2022 at 06:50:02PM +0100, Nick Alcock wrote:
> On 28 Apr 2022, Luis Machado via Binutils outgrape:
> > It would make sense to build-test gdb if any of the following changes:
> [...]
> > libctf/
> 
> This won't do much with gdb without a (much) newer compiler. (I
> build-test gdb before every libctf commit -- every commit that I do,
> anyway. I do actual gdb test runs rather less often. I should fix that,
> at least for the CTF tests...)
> 
> It doesn't change anywhere near as often as GDB though so the cost of
> adding it is low.

How much newer compiler are we talking? There are other builders that
use workers which might have newer gcc installed.

A change in libctf is tested against a build of gdb (but no tests
yet).  Should a change in libctf also cause a check of a build of
gas/ld/binutils/gold and tests of gas/ld/binutils?

Are there any specific ctf tests that aren't part of the above
testsuite? If so would it make sense to make libctf its own "project"
so that a change to libctf (and any depended directoy/config files)
would cause a "just build libctf and run the ctf testsuite"?

Maybe we should add something similar for gprof, gprofng and sim?

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Nick Alcock]

On 29 Apr 2022, Mark Wielaard uttered the following:

> Hi Nick,
>
> On Thu, Apr 28, 2022 at 06:50:02PM +0100, Nick Alcock wrote:
>> On 28 Apr 2022, Luis Machado via Binutils outgrape:
>> > It would make sense to build-test gdb if any of the following changes:
>> [...]
>> > libctf/
>> 
>> This won't do much with gdb without a (much) newer compiler. (I
>> build-test gdb before every libctf commit -- every commit that I do,
>> anyway. I do actual gdb test runs rather less often. I should fix that,
>> at least for the CTF tests...)
>> 
>> It doesn't change anywhere near as often as GDB though so the cost of
>> adding it is low.
>
> How much newer compiler are we talking? There are other builders that
> use workers which might have newer gcc installed.

Oh, quite a lot newer -- CTF support in GCC has been in progress since
the GCC 10 days, but it only landed in GCC proper after GCC 11 branched,
so it'll be in GCC 12. It's not in a released GCC yet, but we're
counting the days :)

> A change in libctf is tested against a build of gdb (but no tests
> yet).  Should a change in libctf also cause a check of a build of
> gas/ld/binutils/gold and tests of gas/ld/binutils?

That's probably fairly pointless without GCC 12 in the mix. Even then,
checking ld and GDB is worthwhile but not the others: gas doesn't depend
on it at all, gold doesn't have CTF dedup support (yet: I do mean to add
it), and binutils's objcopy CTF support is only tested by the ld
testsuite's usage of objcopy.

Also, note that GDB doesn't have very many CTF tests yet: you might want
to build it but not run the *entire* testsuite, just tests matching
*ctf*, just to save time, since the GDB testsuite is so huge. Changes to
the CTF code aren't going to break the rest of GDB (I hope!). Mind you,
if it *does*, finding that out is exactly what buildbots are for.
I'm vacillating.

> Are there any specific ctf tests that aren't part of the above
> testsuite? If so would it make sense to make libctf its own "project"
> so that a change to libctf (and any depended directoy/config files)
> would cause a "just build libctf and run the ctf testsuite"?

Hm. That's probably a good idea -- most of the libctf tests require a
compiler supporting -gctf, but there *are* a few that test the writable
dict API (the same one used by ld to emit CTF after deduplication).
They're not exactly comprehensive tests (they're checking for write-time
regressions that are not triggered by ld's usage of libctf) but they do
cover a surprising percentage of libctf nonetheless. It even tests some
of the ctf_link API, though not the complicated deduplicating part. And
libctf does build quickly :)

> Maybe we should add something similar for gprof, gprofng and sim?

gprof doesn't seem to have any tests. I suppoe it might be worth just
making sure changes don't break compilation. gprofng too, but maybe a
litlelater, once the more obvious compilation problems are knocked out
of it. (And definitely run its testsuite. gprofng already has a few
tests and will certainly acquire more with time. Its arch requirements
are fairly tight but x86 is a common arch so probably not actually
*problematic*.)

-- 
NULL && (void)

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Mark Wielaard]

Hi Nick,

On Sat, Apr 30, 2022 at 01:12:54AM +0100, Nick Alcock via Overseers wrote:
> On 29 Apr 2022, Mark Wielaard uttered the following:
> > On Thu, Apr 28, 2022 at 06:50:02PM +0100, Nick Alcock wrote:
> >> On 28 Apr 2022, Luis Machado via Binutils outgrape:
> >> > It would make sense to build-test gdb if any of the following changes:
> >> [...]
> >> > libctf/
> >> 
> >> This won't do much with gdb without a (much) newer compiler. (I
> >> build-test gdb before every libctf commit -- every commit that I do,
> >> anyway. I do actual gdb test runs rather less often. I should fix that,
> >> at least for the CTF tests...)
> >> 
> >> It doesn't change anywhere near as often as GDB though so the cost of
> >> adding it is low.
> >
> > How much newer compiler are we talking? There are other builders that
> > use workers which might have newer gcc installed.
> 
> Oh, quite a lot newer -- CTF support in GCC has been in progress since
> the GCC 10 days, but it only landed in GCC proper after GCC 11 branched,
> so it'll be in GCC 12. It's not in a released GCC yet, but we're
> counting the days :)

OK, and it looks like Fedora 36 will also be released in the next
couple of weeks using GCC12. So once one of the fedora workers is
upgraded to Fedora 36 we can add a new libctf specific builder.

> > A change in libctf is tested against a build of gdb (but no tests
> > yet).  Should a change in libctf also cause a check of a build of
> > gas/ld/binutils/gold and tests of gas/ld/binutils?
> 
> That's probably fairly pointless without GCC 12 in the mix. Even then,
> checking ld and GDB is worthwhile but not the others: gas doesn't depend
> on it at all, gold doesn't have CTF dedup support (yet: I do mean to add
> it), and binutils's objcopy CTF support is only tested by the ld
> testsuite's usage of objcopy.

Even so it seems important to test that everything at least builds
when libctf is changed and that none of the tests fail. Even if there
aren't that many to begin with. I have added libctf/ to the binutils
files for now.

> Also, note that GDB doesn't have very many CTF tests yet: you might want
> to build it but not run the *entire* testsuite, just tests matching
> *ctf*, just to save time, since the GDB testsuite is so huge. Changes to
> the CTF code aren't going to break the rest of GDB (I hope!). Mind you,
> if it *does*, finding that out is exactly what buildbots are for.
> I'm vacillating.

For now we don't have any "stable subset" of gdb tests that we run. If
the ctf-constvars.exp and ctf-ptype.exp are stable (and quick) maybe
they could be part of that stable set?
 
> > Are there any specific ctf tests that aren't part of the above
> > testsuite? If so would it make sense to make libctf its own "project"
> > so that a change to libctf (and any depended directoy/config files)
> > would cause a "just build libctf and run the ctf testsuite"?
> 
> Hm. That's probably a good idea -- most of the libctf tests require a
> compiler supporting -gctf, but there *are* a few that test the writable
> dict API (the same one used by ld to emit CTF after deduplication).
> They're not exactly comprehensive tests (they're checking for write-time
> regressions that are not triggered by ld's usage of libctf) but they do
> cover a surprising percentage of libctf nonetheless. It even tests some
> of the ctf_link API, though not the complicated deduplicating part. And
> libctf does build quickly :)

OK. Lets look at that in a couple of weeks when we have a builder that
can use a GCC12 using distro.

Cheers,

Mark

Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Nick Alcock]

On 30 Apr 2022, Mark Wielaard said:

> On Sat, Apr 30, 2022 at 01:12:54AM +0100, Nick Alcock via Overseers wrote:
>> > How much newer compiler are we talking? There are other builders that
>> > use workers which might have newer gcc installed.
>> 
>> Oh, quite a lot newer -- CTF support in GCC has been in progress since
>> the GCC 10 days, but it only landed in GCC proper after GCC 11 branched,
>> so it'll be in GCC 12. It's not in a released GCC yet, but we're
>> counting the days :)
>
> OK, and it looks like Fedora 36 will also be released in the next
> couple of weeks using GCC12. So once one of the fedora workers is
> upgraded to Fedora 36 we can add a new libctf specific builder.

Excellent!

>> > A change in libctf is tested against a build of gdb (but no tests
>> > yet).  Should a change in libctf also cause a check of a build of
>> > gas/ld/binutils/gold and tests of gas/ld/binutils?
>> 
>> That's probably fairly pointless without GCC 12 in the mix. Even then,
>> checking ld and GDB is worthwhile but not the others: gas doesn't depend
>> on it at all, gold doesn't have CTF dedup support (yet: I do mean to add
>> it), and binutils's objcopy CTF support is only tested by the ld
>> testsuite's usage of objcopy.
>
> Even so it seems important to test that everything at least builds
> when libctf is changed and that none of the tests fail. Even if there
> aren't that many to begin with. I have added libctf/ to the binutils
> files for now.

That seems perfectly reasonable. (ld/ is directly affected, as are
objcopy and objdump.)

>> Also, note that GDB doesn't have very many CTF tests yet: you might want
>> to build it but not run the *entire* testsuite, just tests matching
>> *ctf*, just to save time, since the GDB testsuite is so huge. Changes to
>> the CTF code aren't going to break the rest of GDB (I hope!). Mind you,
>> if it *does*, finding that out is exactly what buildbots are for.
>> I'm vacillating.
>
> For now we don't have any "stable subset" of gdb tests that we run. If
> the ctf-constvars.exp and ctf-ptype.exp are stable (and quick) maybe
> they could be part of that stable set?

Oh definitely. They're both stable and quick.

>> Hm. That's probably a good idea -- most of the libctf tests require a
>> compiler supporting -gctf, but there *are* a few that test the writable
>> dict API (the same one used by ld to emit CTF after deduplication).
>> They're not exactly comprehensive tests (they're checking for write-time
>> regressions that are not triggered by ld's usage of libctf) but they do
>> cover a surprising percentage of libctf nonetheless. It even tests some
>> of the ctf_link API, though not the complicated deduplicating part. And
>> libctf does build quickly :)
>
> OK. Lets look at that in a couple of weeks when we have a builder that
> can use a GCC12 using distro.

I'd expect everything to both run and pass (and do so quickly) on a
Linux system with GCC 12.

-- 
NULL && (void)