public inbox for buildbot@sourceware.org
 help / color / mirror / Atom feed
* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]   ` <rt-4.0.4-32453-1654617175-469.32563-6-0@osuosl.org>
@ 2022-06-07 22:24     ` Mark Wielaard
       [not found]       ` <rt-4.0.4-80639-1654640709-839.32563-6-0@osuosl.org>
  0 siblings, 1 reply; 14+ messages in thread
From: Mark Wielaard @ 2022-06-07 22:24 UTC (permalink / raw)
  To: Lance Albertson via RT; +Cc: buildbot

Hi Lance (and hi builder hackers, this is the OSUOSL request),

On Tue, Jun 07, 2022 at 08:52:56AM -0700, Lance Albertson via RT wrote:
> On Mon Jun 06 15:31:05 2022, mark@klomp.org wrote:
> > Project Name: sourceware
> > Project URL: builder.sourceware.org
> > 
> > Software License: GPL
> > 
> > Estimated size of user community:
> >
> > ~500 committers across all projects, but only ~3 buildbot admins might need
> > direct access.
> > 
> > Other pertinent information about your project:
> > sourceware provides hosting for the GNU Toolchain, gcc, binutils, gdb, glibc,
> > but also various other related projects like valgrind, elfutils, bzip2,
> > libabigail, dwz, debugedit. builder.sourceware.org is our buildbot based CI
> > for these projects.
> 
> Very cool!

Thanks. Note, all configuration is in our public repo:
https://sourceware.org/git/builder.git

> > We have native/VM workers for ppc64le, s390x, ppc64, i386, arm64 and armhf for
> > debian, fedora and centos and x86_64 container builders for fedora, debian and
> > opensuse. Doing hundreds of builds. But we are hitting limits for projects
> > with large testssuite (gcc, glibc and gdb in particular).
> > 
> > The sourceware overseers, who do the technical management of all of sourceware
> > can be reached at overseers@sourceware.org. The builder maintainers can be
> > reached at buildbot@sourceware.org. Both are public mailinglists:
> > https://sourceware.org/mailman/listinfo/overseers
> > https://sourceware.org/mailman/listinfo/buildbot
> 
> Should either of these email addresses be included on this request?

Yes. I didn't know if it was appropriate to include a public list on
osuosl tickets. But I added the builder project email address
buildbot@sourceware.org to this email.

> > The type(s) of server(s) or service(s) you are requesting hosting for, if
> > applicable:
> >
> > We are looking for a larger x86_64 server with 16 to 32 cores, 32G memory and
> > 200GB diskspace on which we would like to run a container based
> > buildbot-worker (our current container hosts use Fedora Core with docker, but
> > any OS which supports docker and user namespaces would do).
> 
> I believe we had some hardware around for this and we shouldn't have a problem
> with installing Fedora on it if that's what you want.

Fedora would be fine. But for the other container builders we are
actually using Fedora CoreOS, which is an auto updating variant (which
is great for our zero maintenance infrastructure). We could install
that ourselves. Or go with plain Fedora if we make sure it also
autoupdates.

> We typically name systems
> as <short project name>-<hostname>.osuosl.org. So I'd assume we'd go with
> sourceware-<hostname>.osuosl.org. Do you have a hostname in mind?

sourceware-builder.osuosl.org but note that we don't need a public
reachable host. All that is needed is a port to access it through ssh,
which doesn't need to be port 22.

> We'll also need an ssh key.

Mine is:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNz openpgp:0xC8437776

Which is derived from my pgp key:
https://gnu.wildebeest.org/blog/mjw/2019/02/17/new-pgp-key/

If that provides me admin access I would use it to create a builder
user with uid 1001 in the docker group with an authorized key to
access the docker socket to create the buildbot-worker
containers. Using a separate account and ssh authorization makes sure
it can only be used for the container builder.

> > Also our current arm64 worker is a single odroid N2+ board which is really
> > underpowered. If you could provide a real arm64 server (does not have to
> > support containers, we would just install a buildbot-worker directly under a
> > user account) that would be wonderful.
> 
> We have an aarch64 cluster running on server class hardware [1] where we can
> easily spin up VMs. I'd recommend you fill out the form there since we have a
> separate onboarding / approval process for that.
> 
> If you still have issues submitting the request using that form, just send send
> an email using the information requested to aarch64-hosting-request@osuosl.org.
> 
> [1] https://osuosl.org/services/aarch64/

I'll fill out that form.

Thanks,

Mark


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]       ` <rt-4.0.4-80639-1654640709-839.32563-6-0@osuosl.org>
@ 2022-06-16 17:40         ` Lance Albertson via RT
  2022-06-16 22:49           ` Mark Wielaard
       [not found]         ` <rt-4.0.4-135343-1655762795-1742.32563-6-0@osuosl.org>
  1 sibling, 1 reply; 14+ messages in thread
From: Lance Albertson via RT @ 2022-06-16 17:40 UTC (permalink / raw)
  To: mark; +Cc: buildbot

On Tue Jun 07 15:25:09 2022, mark@klomp.org wrote:
> Hi Lance (and hi builder hackers, this is the OSUOSL request),

> > > The sourceware overseers, who do the technical management of all of
> > > sourceware can be reached at overseers@sourceware.org. The builder
> > > maintainers can be reached at buildbot@sourceware.org. Both are public
> > > mailinglists:
> > > https://sourceware.org/mailman/listinfo/overseers
> > > https://sourceware.org/mailman/listinfo/buildbot
> >
> > Should either of these email addresses be included on this request?
> 
> Yes. I didn't know if it was appropriate to include a public list on osuosl
> tickets. But I added the builder project email address buildbot@sourceware.org
> to this email.

We've done it with other projects so it shouldn't be a problem as long as you're
OK with it.

> > > The type(s) of server(s) or service(s) you are requesting hosting for, if
> > > applicable:
> > >
> > > We are looking for a larger x86_64 server with 16 to 32 cores, 32G memory
> > > and 200GB diskspace on which we would like to run a container based
> > > buildbot-worker (our current container hosts use Fedora Core with docker,
> > > but any OS which supports docker and user namespaces would do).
> >
> > I believe we had some hardware around for this and we shouldn't have a
> > problem with installing Fedora on it if that's what you want.
> 
> Fedora would be fine. But for the other container builders we are actually
> using Fedora CoreOS, which is an auto updating variant (which is great for our
> zero maintenance infrastructure). We could install that ourselves. Or go with
> plain Fedora if we make sure it also autoupdates.

I haven't had a chance to get in the data center to allocate a system. I'm
hoping to get to that next week. Sorry for the delay! I'll try and get Fedora
CoreOS installed.

> > We typically name systems as <short project name>-<hostname>.osuosl.org. So
> > I'd assume we'd go with sourceware-<hostname>.osuosl.org. Do you have a
> > hostname in mind?
> 
> sourceware-builder.osuosl.org but note that we don't need a public reachable
> host. All that is needed is a port to access it through ssh, which doesn't
> need to be port 22.

I'll likely go with sourceware-builder1.osuosl.org so we can easily increment it
if we add more hosts.

> > We'll also need an ssh key.
> 
> Mine is:
> ssh-rsa
> AAAAB3NzaC1yc2EAAAADAQABAAABAQCgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNz
> openpgp:0xC8437776
> 
> Which is derived from my pgp key:
> https://gnu.wildebeest.org/blog/mjw/2019/02/17/new-pgp-key/
> 
> If that provides me admin access I would use it to create a builder user with
> uid 1001 in the docker group with an authorized key to access the docker
> socket to create the buildbot-worker containers. Using a separate account and
> ssh authorization makes sure it can only be used for the container builder.

OK sounds good.

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-06-16 17:40         ` Lance Albertson via RT
@ 2022-06-16 22:49           ` Mark Wielaard
  2022-06-16 22:49             `  via RT
       [not found]             ` <rt-4.0.4-176523-1655419769-1468.32563-6-0@osuosl.org>
  0 siblings, 2 replies; 14+ messages in thread
From: Mark Wielaard @ 2022-06-16 22:49 UTC (permalink / raw)
  To: Lance Albertson via RT; +Cc: buildbot

[-- Attachment #1: Type: text/plain, Size: 2436 bytes --]

Hi Lance,

On Thu, Jun 16, 2022 at 10:40:42AM -0700, Lance Albertson via RT wrote:
> On Tue Jun 07 15:25:09 2022, mark@klomp.org wrote:
> > Yes. I didn't know if it was appropriate to include a public list on osuosl
> > tickets. But I added the builder project email address buildbot@sourceware.org
> > to this email.
> 
> We've done it with other projects so it shouldn't be a problem as long as you're
> OK with it.

Yes, this makes sure all builder contributors know what is being setup.

> > Fedora would be fine. But for the other container builders we are actually
> > using Fedora CoreOS, which is an auto updating variant (which is great for our
> > zero maintenance infrastructure). We could install that ourselves. Or go with
> > plain Fedora if we make sure it also autoupdates.
> 
> I haven't had a chance to get in the data center to allocate a system. I'm
> hoping to get to that next week. Sorry for the delay! I'll try and get Fedora
> CoreOS installed.

If you could use something like the attached butane/ignition file that
would be great. You might need to change the disk device name if it
isn't /dev/vda but everything else (except the hostname) is generic
and allows the buildbot to connect to it through ssh directly (might
be on a different port behind NAT).

This is all the state/config needed. Everything else (docker images,
git repos, buildbot workers, builds, etc) can/will be regenerated when
needed. So we don't need any backups, because the whole setup can be
regenerated by this butane.yaml file.

Assuming you are using openstack, see also
https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/

You don't need to use my ssh key for the core user. With this setup I
don't really need access. You can assign a different authorized key if
you want to have access yourself.

But we can also go with plain Fedora and then I'll setup docker and
the builder user up myself.

> > > We typically name systems as <short project name>-<hostname>.osuosl.org. So
> > > I'd assume we'd go with sourceware-<hostname>.osuosl.org. Do you have a
> > > hostname in mind?
> >
> > sourceware-builder.osuosl.org but note that we don't need a public reachable
> > host. All that is needed is a port to access it through ssh, which doesn't
> > need to be port 22.
> 
> I'll likely go with sourceware-builder1.osuosl.org so we can easily increment it
> if we add more hosts.

I like that :)

Thanks,

Mark

[-- Attachment #2: osuosl-butane.yaml --]
[-- Type: text/plain, Size: 2572 bytes --]

variant: fcos
version: 1.4.0
storage:
  disks:
  - device: /dev/vda
    wipe_table: false
    partitions:
    # 10GB root (already exists, resize)
    - number: 4
      label: root
      size_mib: 10240
      resize: true
    # 8GB swap
    - number: 5
      label: swap
      size_mib: 8192
    # Everything else for /var (which includes everything writable)
    - size_mib: 0
      label: var
  filesystems:
    - device: /dev/disk/by-partlabel/swap
      format: swap
      wipe_filesystem: true
      with_mount_unit: true
    - path: /var
      device: /dev/disk/by-partlabel/var
      # We can select the filesystem we'd like.
      format: ext4
      wipe_filesystem: true
      # Ask Butane to generate a mount unit for us so that this filesystem
      # gets mounted in the real root.
      with_mount_unit: true
  files:
    - path: /etc/hostname
      mode: 0644
      contents:
        inline: sourceware-builder1
systemd:
  units:
    # Create and set selinux context for shared builder dir
    - name: builder-shared-dir.service
      enabled: true
      contents: |
        [Unit]
        Description=Builder shared dir
        Wants=network-online.target
        After=network-online.target
        Before=zincati.service
        ConditionPathExists=!/var/lib/%N.stamp

        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStart=/bin/mkdir -p /home/builder/shared
        ExecStart=/bin/chown builder.builder /home/builder/shared
        ExecStart=/bin/chcon -t container_file_t /home/builder/shared
        ExecStart=/bin/touch /var/lib/%N.stamp

        [Install]
        WantedBy=multi-user.target
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNz
    - name: builder
      uid: 1001
      groups: [docker]
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnB//uE66TX+nmLuox3ETfZ7dvjUl2mlwN4Q+107EWCuP05eKxcahyzwenmzODzzC8/hUVeUtuCJJyMW8+CpqCluHd2bEUT9WIzP1C/T22jBUbhY7zXLufSYfKmZe9zHNUizEbqbGAfR01D3jv3wis8JNOq/yopGLKJmAPd/Ye8XN5lsfAUTREIF+xygjxSb6SZPU9UvKlJmnwZoCN2Fd1u4IhSWBcD3dyZyirD80ZLGsSdOUmzX2Z2NaSI5tG6MxjYtQZbzmsG/Z0HY1Y/KiSolX989+7dMKMy2V911SkS6Cx9aVuSY+ig9Rq6GpfjGkBnHrNCCeVFe8PlYJl6DXZ

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-06-16 22:49           ` Mark Wielaard
@ 2022-06-16 22:49             `  via RT
       [not found]             ` <rt-4.0.4-176523-1655419769-1468.32563-6-0@osuosl.org>
  1 sibling, 0 replies; 14+ messages in thread
From:  via RT @ 2022-06-16 22:49 UTC (permalink / raw)
  Cc: buildbot

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

Hi Lance,

On Thu, Jun 16, 2022 at 10:40:42AM -0700, Lance Albertson via RT wrote:
> On Tue Jun 07 15:25:09 2022, mark@klomp.org wrote:
> > Yes. I didn't know if it was appropriate to include a public list on osuosl
> > tickets. But I added the builder project email address buildbot@sourceware.org
> > to this email.
> 
> We've done it with other projects so it shouldn't be a problem as long as you're
> OK with it.

Yes, this makes sure all builder contributors know what is being setup.

> > Fedora would be fine. But for the other container builders we are actually
> > using Fedora CoreOS, which is an auto updating variant (which is great for our
> > zero maintenance infrastructure). We could install that ourselves. Or go with
> > plain Fedora if we make sure it also autoupdates.
> 
> I haven't had a chance to get in the data center to allocate a system. I'm
> hoping to get to that next week. Sorry for the delay! I'll try and get Fedora
> CoreOS installed.

If you could use something like the attached butane/ignition file that
would be great. You might need to change the disk device name if it
isn't /dev/vda but everything else (except the hostname) is generic
and allows the buildbot to connect to it through ssh directly (might
be on a different port behind NAT).

This is all the state/config needed. Everything else (docker images,
git repos, buildbot workers, builds, etc) can/will be regenerated when
needed. So we don't need any backups, because the whole setup can be
regenerated by this butane.yaml file.

Assuming you are using openstack, see also
https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/

You don't need to use my ssh key for the core user. With this setup I
don't really need access. You can assign a different authorized key if
you want to have access yourself.

But we can also go with plain Fedora and then I'll setup docker and
the builder user up myself.

> > > We typically name systems as <short project name>-<hostname>.osuosl.org. So
> > > I'd assume we'd go with sourceware-<hostname>.osuosl.org. Do you have a
> > > hostname in mind?
> >
> > sourceware-builder.osuosl.org but note that we don't need a public reachable
> > host. All that is needed is a port to access it through ssh, which doesn't
> > need to be port 22.
> 
> I'll likely go with sourceware-builder1.osuosl.org so we can easily increment it
> if we add more hosts.

I like that :)

Thanks,

Mark

[-- Attachment #2: osuosl-butane.yaml --]
[-- Type: text/plain, Size: 2572 bytes --]

variant: fcos
version: 1.4.0
storage:
  disks:
  - device: /dev/vda
    wipe_table: false
    partitions:
    # 10GB root (already exists, resize)
    - number: 4
      label: root
      size_mib: 10240
      resize: true
    # 8GB swap
    - number: 5
      label: swap
      size_mib: 8192
    # Everything else for /var (which includes everything writable)
    - size_mib: 0
      label: var
  filesystems:
    - device: /dev/disk/by-partlabel/swap
      format: swap
      wipe_filesystem: true
      with_mount_unit: true
    - path: /var
      device: /dev/disk/by-partlabel/var
      # We can select the filesystem we'd like.
      format: ext4
      wipe_filesystem: true
      # Ask Butane to generate a mount unit for us so that this filesystem
      # gets mounted in the real root.
      with_mount_unit: true
  files:
    - path: /etc/hostname
      mode: 0644
      contents:
        inline: sourceware-builder1
systemd:
  units:
    # Create and set selinux context for shared builder dir
    - name: builder-shared-dir.service
      enabled: true
      contents: |
        [Unit]
        Description=Builder shared dir
        Wants=network-online.target
        After=network-online.target
        Before=zincati.service
        ConditionPathExists=!/var/lib/%N.stamp

        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStart=/bin/mkdir -p /home/builder/shared
        ExecStart=/bin/chown builder.builder /home/builder/shared
        ExecStart=/bin/chcon -t container_file_t /home/builder/shared
        ExecStart=/bin/touch /var/lib/%N.stamp

        [Install]
        WantedBy=multi-user.target
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgVJJnY8zh8uHn8d/E7p4j+9ueTvTHMRYOS0kkGhHBC7JmxCw6/EvbnbTsI0CQeyIJHlmPIqDVgRVjijcTWacd3vIdazzH9sqs65nl49yMnA23tIya4VWlbHC3J4x/LL84A4GaJO/FVF2vv6hVg3IGbopp5KX+pr6s56TiWddSDqMgjb7rSzjWuNyRK75ToctL7Y/Zn6st3ZioO7LXq3ghkWf8JR7ZaUFIY6P1qS5heiCHP0PxQJSrtpYzH3rKJoHpIkjxnsB/sD0C05cAdlzXBTUVTNLY+DPlQ7FeRkG+VK91briG4tvQ8ohhEiC9HuJu1AKMNWBZ9qeUwsXaJvNz
    - name: builder
      uid: 1001
      groups: [docker]
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnB//uE66TX+nmLuox3ETfZ7dvjUl2mlwN4Q+107EWCuP05eKxcahyzwenmzODzzC8/hUVeUtuCJJyMW8+CpqCluHd2bEUT9WIzP1C/T22jBUbhY7zXLufSYfKmZe9zHNUizEbqbGAfR01D3jv3wis8JNOq/yopGLKJmAPd/Ye8XN5lsfAUTREIF+xygjxSb6SZPU9UvKlJmnwZoCN2Fd1u4IhSWBcD3dyZyirD80ZLGsSdOUmzX2Z2NaSI5tG6MxjYtQZbzmsG/Z0HY1Y/KiSolX989+7dMKMy2V911SkS6Cx9aVuSY+ig9Rq6GpfjGkBnHrNCCeVFe8PlYJl6DXZ

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]             ` <rt-4.0.4-176523-1655419769-1468.32563-6-0@osuosl.org>
@ 2022-06-20 19:36               ` Lance Albertson via RT
  2022-06-20 22:06                 ` Mark Wielaard
  0 siblings, 1 reply; 14+ messages in thread
From: Lance Albertson via RT @ 2022-06-20 19:36 UTC (permalink / raw)
  To: mark; +Cc: buildbot

On Thu Jun 16 15:49:29 2022, mark@klomp.org wrote:
> > > Fedora would be fine. But for the other container builders we are actually
> > > using Fedora CoreOS, which is an auto updating variant (which is great for
> > > our zero maintenance infrastructure). We could install that ourselves.  Or
> > > go with plain Fedora if we make sure it also autoupdates.
> >
> > I haven't had a chance to get in the data center to allocate a system. I'm
> > hoping to get to that next week. Sorry for the delay! I'll try and get
> > Fedora CoreOS installed.
> 
> If you could use something like the attached butane/ignition file that would
> be great. You might need to change the disk device name if it isn't /dev/vda
> but everything else (except the hostname) is generic and allows the buildbot
> to connect to it through ssh directly (might be on a different port behind
> NAT).
> 
> This is all the state/config needed. Everything else (docker images, git
> repos, buildbot workers, builds, etc) can/will be regenerated when needed. So
> we don't need any backups, because the whole setup can be regenerated by this
> butane.yaml file.
> 
> Assuming you are using openstack, see also
> https://docs.fedoraproject.org/en-US/fedora-coreos/provisioning-openstack/
> 
> You don't need to use my ssh key for the core user. With this setup I don't
> really need access. You can assign a different authorized key if you want to
> have access yourself.
> 
> But we can also go with plain Fedora and then I'll setup docker and the
> builder user up myself.

I believe we were going to allocate a new phyiscal system for this instead of
allocating VM(s) for it on our OpenStack. Is that still correct? I was planning
on using the bare metal installation guide [1] once I got hardware allocated.

It doesn't look like it'll be that much work to install for us but I'll let you
know if we run into any issues.

I hope to get this online by the end of the week.

[1] https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-06-20 19:36               ` Lance Albertson via RT
@ 2022-06-20 22:06                 ` Mark Wielaard
  2022-06-20 22:06                   `  via RT
  0 siblings, 1 reply; 14+ messages in thread
From: Mark Wielaard @ 2022-06-20 22:06 UTC (permalink / raw)
  To: Lance Albertson via RT; +Cc: buildbot

Hi Lance,

On Mon, Jun 20, 2022 at 12:36:22PM -0700, Lance Albertson via RT wrote:
> I believe we were going to allocate a new phyiscal system for this instead of
> allocating VM(s) for it on our OpenStack. Is that still correct? I was planning
> on using the bare metal installation guide [1] once I got hardware allocated.

The same file can be used for bare metal. You only have to change the
disk device entry from /dev/vda to /dev/sda. And you have to provide
the ignition file through an URL that the system can reach.

BTW. The osuosl-butane.yaml file is a butane yaml file, while the install
wants an ignition json file. You transform the yaml file into a json
file using the butane binary, see
https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/

> It doesn't look like it'll be that much work to install for us but I'll let you
> know if we run into any issues.

I'll try to be available on irc libera.chat in #osuosl

> I hope to get this online by the end of the week.
> 
> [1] https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/

Thanks,

Mark


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-06-20 22:06                 ` Mark Wielaard
@ 2022-06-20 22:06                   `  via RT
  0 siblings, 0 replies; 14+ messages in thread
From:  via RT @ 2022-06-20 22:06 UTC (permalink / raw)
  Cc: buildbot

Hi Lance,

On Mon, Jun 20, 2022 at 12:36:22PM -0700, Lance Albertson via RT wrote:
> I believe we were going to allocate a new phyiscal system for this instead of
> allocating VM(s) for it on our OpenStack. Is that still correct? I was planning
> on using the bare metal installation guide [1] once I got hardware allocated.

The same file can be used for bare metal. You only have to change the
disk device entry from /dev/vda to /dev/sda. And you have to provide
the ignition file through an URL that the system can reach.

BTW. The osuosl-butane.yaml file is a butane yaml file, while the install
wants an ignition json file. You transform the yaml file into a json
file using the butane binary, see
https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/

> It doesn't look like it'll be that much work to install for us but I'll let you
> know if we run into any issues.

I'll try to be available on irc libera.chat in #osuosl

> I hope to get this online by the end of the week.
> 
> [1] https://docs.fedoraproject.org/en-US/fedora-coreos/bare-metal/

Thanks,

Mark



^ permalink raw reply	[flat|nested] 14+ messages in thread

* [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]         ` <rt-4.0.4-135343-1655762795-1742.32563-6-0@osuosl.org>
@ 2022-08-05  0:29           ` Lance Albertson via RT
  2022-08-05  1:30             ` Mark Wielaard
  0 siblings, 1 reply; 14+ messages in thread
From: Lance Albertson via RT @ 2022-08-05  0:29 UTC (permalink / raw)
  To: mark; +Cc: buildbot

> > I believe we were going to allocate a new phyiscal system for this instead
> > of allocating VM(s) for it on our OpenStack. Is that still correct? I was
> > planning on using the bare metal installation guide [1] once I got hardware
> > allocated.
> 
> The same file can be used for bare metal. You only have to change the disk
> device entry from /dev/vda to /dev/sda. And you have to provide the ignition
> file through an URL that the system can reach.
> 
> BTW. The osuosl-butane.yaml file is a butane yaml file, while the install
> wants an ignition json file. You transform the yaml file into a json file
> using the butane binary, see
> https://docs.fedoraproject.org/en-US/fedora-coreos/producing-ign/

I _finally_ got this system online. You should be able to ssh to it as
sourceware-builder1.osuosl.org.

A few changes I made to the ignition file:

- Change the console to go to ttyS1 so IPMI works properly
- Add OSL unmanaged ssh key to core user
- Setup IPv6 as a static IP address

Can you please confirm you have access and let me know how it looks?

Thanks-

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-08-05  0:29           ` Lance Albertson via RT
@ 2022-08-05  1:30             ` Mark Wielaard
  2022-08-05  1:31               `  via RT
       [not found]               ` <rt-4.0.4-195587-1659663068-623.32563-6-0@osuosl.org>
  0 siblings, 2 replies; 14+ messages in thread
From: Mark Wielaard @ 2022-08-05  1:30 UTC (permalink / raw)
  To: Lance Albertson via RT; +Cc: buildbot

Hi Lance,

On Thu, Aug 04, 2022 at 05:29:12PM -0700, Lance Albertson via RT wrote:
> Can you please confirm you have access and let me know how it looks?

It looks great. It already did its first build:
https://builder.sourceware.org/buildbot/#/workers/26
Which also means it could create the container image without issue.

Would it be possible to enable a few more cpus? Currently there are 8
online and 8 offline. For the larger testsuite from gcc it really
helps to be able to run more tests in parallel (test time scales
linear with the number of cpus).

Thanks,

Mark

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-08-05  1:30             ` Mark Wielaard
@ 2022-08-05  1:31               `  via RT
       [not found]               ` <rt-4.0.4-195587-1659663068-623.32563-6-0@osuosl.org>
  1 sibling, 0 replies; 14+ messages in thread
From:  via RT @ 2022-08-05  1:31 UTC (permalink / raw)
  Cc: buildbot

Hi Lance,

On Thu, Aug 04, 2022 at 05:29:12PM -0700, Lance Albertson via RT wrote:
> Can you please confirm you have access and let me know how it looks?

It looks great. It already did its first build:
https://builder.sourceware.org/buildbot/#/workers/26
Which also means it could create the container image without issue.

Would it be possible to enable a few more cpus? Currently there are 8
online and 8 offline. For the larger testsuite from gcc it really
helps to be able to run more tests in parallel (test time scales
linear with the number of cpus).

Thanks,

Mark



^ permalink raw reply	[flat|nested] 14+ messages in thread

* [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]               ` <rt-4.0.4-195587-1659663068-623.32563-6-0@osuosl.org>
@ 2022-08-05 15:52                 ` Lance Albertson via RT
  2022-08-05 21:17                   ` Mark Wielaard
  0 siblings, 1 reply; 14+ messages in thread
From: Lance Albertson via RT @ 2022-08-05 15:52 UTC (permalink / raw)
  To: mark; +Cc: buildbot

On Thu Aug 04 18:31:08 2022, mark@klomp.org wrote:
> On Thu, Aug 04, 2022 at 05:29:12PM -0700, Lance Albertson via RT wrote:
> > Can you please confirm you have access and let me know how it looks?
> 
> It looks great. It already did its first build:
> https://builder.sourceware.org/buildbot/#/workers/26 Which also means it could
> create the container image without issue.

Awesome!

> Would it be possible to enable a few more cpus? Currently there are 8 online
> and 8 offline. For the larger testsuite from gcc it really helps to be able to
> run more tests in parallel (test time scales linear with the number of cpus).

Resolved via IRC:

08:39 < Ramereth> mjw: hmm, hyperthreading is enabled. I wonder if the kernel
      cpu mitigations that are enabled by default in the kernel are turning it
      off. Do you want me to adjust the kernel parameters to not do that?
08:40 < mjw> Ramereth, if you could and know how then yes please, since the same
      user (builder) runs everything there really isn't any data leak to
      prevent.
08:41 < Ramereth> ok, give me a moment to do that
08:41 < mjw> Or if there is, there are much simpler ways to leak things than
      trying to extract data through subtle timing issues exposed by
      hyperthreading :)
08:46 < Ramereth> CPU(s):                  16
08:46 < Ramereth> mjw: looks better
08:47 < mjw> Cool!\
08:47 < Ramereth> FWIW I just updated the ignition file in case we end up
      rebuilding it later to make sure that's the case again


FWIW I adjusted this on the running system by doing the following:

rpm-ostree kargs --replace=mitigations=auto,nosmt=off

And then rebooted the system.

Is there anything else you need for this?

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab 

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-08-05 15:52                 ` Lance Albertson via RT
@ 2022-08-05 21:17                   ` Mark Wielaard
  2022-08-05 21:17                     `  via RT
       [not found]                     ` <rt-4.0.4-171867-1659734276-616.32563-6-0@osuosl.org>
  0 siblings, 2 replies; 14+ messages in thread
From: Mark Wielaard @ 2022-08-05 21:17 UTC (permalink / raw)
  To: Lance Albertson via RT; +Cc: buildbot

Hi Lance,

On Fri, Aug 05, 2022 at 08:52:58AM -0700, Lance Albertson via RT wrote:
> Is there anything else you need for this?

No, looks great. I'll play with it this weekend to enable some larger
testsuites using all 16 CPUs.

Thanks,

Mark

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
  2022-08-05 21:17                   ` Mark Wielaard
@ 2022-08-05 21:17                     `  via RT
       [not found]                     ` <rt-4.0.4-171867-1659734276-616.32563-6-0@osuosl.org>
  1 sibling, 0 replies; 14+ messages in thread
From:  via RT @ 2022-08-05 21:17 UTC (permalink / raw)
  Cc: buildbot

Hi Lance,

On Fri, Aug 05, 2022 at 08:52:58AM -0700, Lance Albertson via RT wrote:
> Is there anything else you need for this?

No, looks great. I'll play with it this weekend to enable some larger
testsuites using all 16 CPUs.

Thanks,

Mark



^ permalink raw reply	[flat|nested] 14+ messages in thread

* [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org
       [not found]                     ` <rt-4.0.4-171867-1659734276-616.32563-6-0@osuosl.org>
@ 2022-08-05 21:52                       ` Lance Albertson via RT
  0 siblings, 0 replies; 14+ messages in thread
From: Lance Albertson via RT @ 2022-08-05 21:52 UTC (permalink / raw)
  To: mark; +Cc: buildbot

On Fri Aug 05 14:17:56 2022, mark@klomp.org wrote:
> On Fri, Aug 05, 2022 at 08:52:58AM -0700, Lance Albertson via RT wrote:
> > Is there anything else you need for this?
> 
> No, looks great. I'll play with it this weekend to enable some larger
> testsuites using all 16 CPUs.

Excellent. Going ahead and closing this ticket then.

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab 

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2022-08-05 21:52 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <RT-Ticket-32563@osuosl.org>
     [not found] ` <Yp5+pqvhR8WYiaqq@wildebeest.org>
     [not found]   ` <rt-4.0.4-32453-1654617175-469.32563-6-0@osuosl.org>
2022-06-07 22:24     ` [support.osuosl.org #32563] Request CI hosting for builder.sourceware.org Mark Wielaard
     [not found]       ` <rt-4.0.4-80639-1654640709-839.32563-6-0@osuosl.org>
2022-06-16 17:40         ` Lance Albertson via RT
2022-06-16 22:49           ` Mark Wielaard
2022-06-16 22:49             `  via RT
     [not found]             ` <rt-4.0.4-176523-1655419769-1468.32563-6-0@osuosl.org>
2022-06-20 19:36               ` Lance Albertson via RT
2022-06-20 22:06                 ` Mark Wielaard
2022-06-20 22:06                   `  via RT
     [not found]         ` <rt-4.0.4-135343-1655762795-1742.32563-6-0@osuosl.org>
2022-08-05  0:29           ` Lance Albertson via RT
2022-08-05  1:30             ` Mark Wielaard
2022-08-05  1:31               `  via RT
     [not found]               ` <rt-4.0.4-195587-1659663068-623.32563-6-0@osuosl.org>
2022-08-05 15:52                 ` Lance Albertson via RT
2022-08-05 21:17                   ` Mark Wielaard
2022-08-05 21:17                     `  via RT
     [not found]                     ` <rt-4.0.4-171867-1659734276-616.32563-6-0@osuosl.org>
2022-08-05 21:52                       ` Lance Albertson via RT

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).