public inbox for gdb@sourceware.org
 help / color / mirror / Atom feed
From: Mark Wielaard <mark@klomp.org>
To: Szabolcs Nagy <szabolcs.nagy@arm.com>
Cc: binutils@sourceware.org, gdb@sourceware.org,
	buildbot@sourceware.org,
	 Thomas Fitzsimmons <fitzsim@fitzsim.org>,
	fche@redhat.com
Subject: Re: buildbot users try branch builders
Date: Tue, 21 Jun 2022 19:30:01 +0200	[thread overview]
Message-ID: <60db1ab53cd9e65c9c22b11bbfd5706ebcc85a90.camel@klomp.org> (raw)
In-Reply-To: <YrGdzvnHZZl6Eo6n@arm.com>

Hi Szabolcs,

On Tue, 2022-06-21 at 11:30 +0100, Szabolcs Nagy wrote:
> The 06/20/2022 12:48, Mark Wielaard wrote:
> > The armhf and arm64 builders have left out for now since they are a
> > bit too slow. We'll hopefully get a 8 core Ampare machine from the
> > Oregon State University this week so we can also enable those arm
> > builders.
> 
> what is the requirement for the machine? i have an aarch64 machine
> for public gnu tools ci, and i can set up build slaves there if
> there are instructions how to do it.

The requirements are not that big, in theory that tinker-board and
odroid-board are fine as buildbot-workers. But only for smaller builds.
To actually run some of the larger builds it should have at least 8GB
and 4cpus. More is better of course. Especially for the larger projects
if you want to do full test runs and not just "quick" CI builds.

Disk again depends on how many builds/projects you want to support. It
is really beneficial to have a bigger ~5GB ccache for repeat builds.
binutils takes ~2G (plus another ~2G for the try builder), similar for
gdb and libabigail, gccrust is ~4.5G, valgrind ~600MB, elfutils ~300MB,
bzip2, dwz are just a couple of MB. If we want to add glibc and enable
try builders for more projects I estimate 30GB to 50GB. The gccrust
builder does just a --disable-bootstrap build, so a full gcc builder
would add another 10G at least (maybe 2 or 3 times if we want to add
try builders). If you like to do container builds then those are almost
full distros, so add at least another 4GB per container/distro type (we
currently support 4). So you can get away with ~15GB, but might need up
to ~300GB if you want to do more and larger builds across multiple
distros.

There is a little documentation on adding workers (they used to be
called slaves, but buildbot renamed them some time ago) in the README:
https://sourceware.org/git/?p=builder.git;a=blob;f=README;hb=HEAD#l111
But that is mostly from the side of the buildbot master.

The easiest setup is one with a dedicated distro install with a
buildbot-worker: 
https://docs.buildbot.net/current/manual/installation/worker.html

All you really need to do is make sure a full distro is installed with
all development packages. You can take a peek at one of the container
files to see which: 
https://sourceware.org/git/?p=builder.git;a=tree;f=builder/containers;hb=HEAD
Then create a user which can run the buildbot-worker.
Please contact the buildbot@sourceware.org mailinglist for connection
info.

Another way is making sure you have docker installed (podman sadly
seems incompatible at the moment) and a builder user that is in the
docker group which can be accessed through ssh to create full container
images. But that is a bit more work (and currently depends on some
specific user uid to make in/out container sharing easier). Please
again contact the buildbot@sourceware.org mailinglist for more info.

There are several people on the list who have setup buildbot-workers
who can help. And then we can coordinate which projects/builders you
want to support. https://sourceware.org/mailman/listinfo/buildbot

Cheers,

Mark

  reply	other threads:[~2022-06-21 17:30 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-20 10:48 Mark Wielaard
2022-06-21 10:30 ` Szabolcs Nagy
2022-06-21 17:30   ` Mark Wielaard [this message]
2022-06-22  6:32 ` Jan Beulich
2022-06-22  7:30   ` Andreas Schwab
2022-06-22 10:51   ` Mark Wielaard
2022-06-22 11:04     ` Andreas Schwab
2022-07-04  6:39       ` Jan Beulich
2022-07-04 14:10         ` Andreas Schwab

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=60db1ab53cd9e65c9c22b11bbfd5706ebcc85a90.camel@klomp.org \
    --to=mark@klomp.org \
    --cc=binutils@sourceware.org \
    --cc=buildbot@sourceware.org \
    --cc=fche@redhat.com \
    --cc=fitzsim@fitzsim.org \
    --cc=gdb@sourceware.org \
    --cc=szabolcs.nagy@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).