Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org

[Mark Wielaard <mark@klomp.org>]

Hi Lance (and hi builder hackers, this is the OSUOSL request),

On Tue, Jun 07, 2022 at 08:52:56AM -0700, Lance Albertson via RT wrote:
> On Mon Jun 06 15:31:05 2022, mark@klomp.org wrote:
> > Project Name: sourceware
> > Project URL: builder.sourceware.org
> > 
> > Software License: GPL
> > 
> > Estimated size of user community:
> >
> > ~500 committers across all projects, but only ~3 buildbot admins might need
> > direct access.
> > 
> > Other pertinent information about your project:
> > sourceware provides hosting for the GNU Toolchain, gcc, binutils, gdb, glibc,
> > but also various other related projects like valgrind, elfutils, bzip2,
> > libabigail, dwz, debugedit. builder.sourceware.org is our buildbot based CI
> > for these projects.
> 
> Very cool!

Thanks. Note, all configuration is in our public repo:
https://sourceware.org/git/builder.git

> > We have native/VM workers for ppc64le, s390x, ppc64, i386, arm64 and armhf for
> > debian, fedora and centos and x86_64 container builders for fedora, debian and
> > opensuse. Doing hundreds of builds. But we are hitting limits for projects
> > with large testssuite (gcc, glibc and gdb in particular).
> > 
> > The sourceware overseers, who do the technical management of all of sourceware
> > can be reached at overseers@sourceware.org. The builder maintainers can be
> > reached at buildbot@sourceware.org. Both are public mailinglists:
> > https://sourceware.org/mailman/listinfo/overseers
> > https://sourceware.org/mailman/listinfo/buildbot
> 
> Should either of these email addresses be included on this request?

Yes. I didn't know if it was appropriate to include a public list on
osuosl tickets. But I added the builder project email address
buildbot@sourceware.org to this email.

> > The type(s) of server(s) or service(s) you are requesting hosting for, if
> > applicable:
> >
> > We are looking for a larger x86_64 server with 16 to 32 cores, 32G memory and
> > 200GB diskspace on which we would like to run a container based
> > buildbot-worker (our current container hosts use Fedora Core with docker, but
> > any OS which supports docker and user namespaces would do).
> 
> I believe we had some hardware around for this and we shouldn't have a problem
> with installing Fedora on it if that's what you want.

Fedora would be fine. But for the other container builders we are
actually using Fedora CoreOS, which is an auto updating variant (which
is great for our zero maintenance infrastructure). We could install
that ourselves. Or go with plain Fedora if we make sure it also
autoupdates.

> We typically name systems
> as <short project name>-<hostname>.osuosl.org. So I'd assume we'd go with
> sourceware-<hostname>.osuosl.org. Do you have a hostname in mind?

sourceware-builder.osuosl.org but note that we don't need a public
reachable host. All that is needed is a port to access it through ssh,
which doesn't need to be port 22.

> We'll also need an ssh key.

Mine is:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNz openpgp:0xC8437776

Which is derived from my pgp key:
https://gnu.wildebeest.org/blog/mjw/2019/02/17/new-pgp-key/

If that provides me admin access I would use it to create a builder
user with uid 1001 in the docker group with an authorized key to
access the docker socket to create the buildbot-worker
containers. Using a separate account and ssh authorization makes sure
it can only be used for the container builder.

> > Also our current arm64 worker is a single odroid N2+ board which is really
> > underpowered. If you could provide a real arm64 server (does not have to
> > support containers, we would just install a buildbot-worker directly under a
> > user account) that would be wonderful.
> 
> We have an aarch64 cluster running on server class hardware [1] where we can
> easily spin up VMs. I'd recommend you fill out the form there since we have a
> separate onboarding / approval process for that.
> 
> If you still have issues submitting the request using that form, just send send
> an email using the information requested to aarch64-hosting-request@osuosl.org.
> 
> [1] https://osuosl.org/services/aarch64/

I'll fill out that form.

Thanks,

Mark