buildbot status

buildbot status

[Roland McGrath]

I've just created a new Bugzilla component called "buildbot".  I decided it
would be useful to have this as distinct from the "admin" catch-all so that
someone interested in improving the bot situation could have a clear queue
of items to look at.  I added a mention of the component on the Buildbot
wiki page.  That wiki page could still use some more love.

Not filed in Bugzilla, but now fixed on the bots, is an issue that Tulio
raised a while back and that has been making the i686 and x86_64 builds
essentially useless for some months now.

The i686 and x86_64 builders were both stuck because I'd left changes in
their working directories from past manual debugging attempts.  This was
making the git commands in the "sync" step do basically nothing, causing
the "sync" step to show up as red but not preventing the build and test
steps from proceeding with a source directory that wasn't getting updated.
I've now made the "sync" step more robust to such local turds so it
aggressively wipes out everything from the source directory except exactly
what's in the git trunk.

I filed https://sourceware.org/bugzilla/show_bug.cgi?id=19760 for the other
issue that Tulio raised, which is races between bot slave triggers and more
commits coming in.  We don't yet have a working fix for that, but it is a
much less serious issue than the failing-sync problem.

For the new bugzilla component, I'm automatically CC'd but there is no
default assignee.  I didn't want to imply that I will necessarily jump on
every buildbot bug filed myself, though I will try to keep track of it.
Indeed I would very much like to encourage more folks to get involved with
improving the bot stuff.


Thanks,
Roland

Re: buildbot status

[Carlos O'Donell]

On 03/03/2016 05:38 PM, Roland McGrath wrote:
> I've just created a new Bugzilla component called "buildbot".  I decided it
> would be useful to have this as distinct from the "admin" catch-all so that
> someone interested in improving the bot situation could have a clear queue
> of items to look at.  I added a mention of the component on the Buildbot
> wiki page.  That wiki page could still use some more love.
> 
> Not filed in Bugzilla, but now fixed on the bots, is an issue that Tulio
> raised a while back and that has been making the i686 and x86_64 builds
> essentially useless for some months now.
> 
> The i686 and x86_64 builders were both stuck because I'd left changes in
> their working directories from past manual debugging attempts.  This was
> making the git commands in the "sync" step do basically nothing, causing
> the "sync" step to show up as red but not preventing the build and test
> steps from proceeding with a source directory that wasn't getting updated.
> I've now made the "sync" step more robust to such local turds so it
> aggressively wipes out everything from the source directory except exactly
> what's in the git trunk.
> 
> I filed https://sourceware.org/bugzilla/show_bug.cgi?id=19760 for the other
> issue that Tulio raised, which is races between bot slave triggers and more
> commits coming in.  We don't yet have a working fix for that, but it is a
> much less serious issue than the failing-sync problem.
> 
> For the new bugzilla component, I'm automatically CC'd but there is no
> default assignee.  I didn't want to imply that I will necessarily jump on
> every buildbot bug filed myself, though I will try to keep track of it.
> Indeed I would very much like to encourage more folks to get involved with
> improving the bot stuff.

I'd love to just repeat the same process Tulio used :-)

Tulio,

Any status on updating the wiki page with more instructions?

https://sourceware.org/glibc/wiki/Buildbot

-- 
Cheers,
Carlos.

Re: buildbot status

[Tulio Magno Quites Machado Filho]

Carlos O'Donell <carlos@redhat.com> writes:

> On 03/03/2016 05:38 PM, Roland McGrath wrote:
>> For the new bugzilla component, I'm automatically CC'd but there is no
>> default assignee.  I didn't want to imply that I will necessarily jump on
>> every buildbot bug filed myself, though I will try to keep track of it.
>> Indeed I would very much like to encourage more folks to get involved with
>> improving the bot stuff.
>
> I'd love to just repeat the same process Tulio used :-)
>
> Tulio,
>
> Any status on updating the wiki page with more instructions?

Sorry.  Give me a few more days.
I'm validating the steps in a new BuildSlave.

> https://sourceware.org/glibc/wiki/Buildbot

I'll update this wiki page as I complete each of the steps.

-- 
Tulio Magno

Re: buildbot status

[Tulio Magno Quites Machado Filho]

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com> writes:

> Carlos O'Donell <carlos@redhat.com> writes:
>
>> On 03/03/2016 05:38 PM, Roland McGrath wrote:
>>> For the new bugzilla component, I'm automatically CC'd but there is no
>>> default assignee.  I didn't want to imply that I will necessarily jump on
>>> every buildbot bug filed myself, though I will try to keep track of it.
>>> Indeed I would very much like to encourage more folks to get involved with
>>> improving the bot stuff.
>>
>> I'd love to just repeat the same process Tulio used :-)
>>
>> Tulio,
>>
>> Any status on updating the wiki page with more instructions?
>
> Sorry.  Give me a few more days.
> I'm validating the steps in a new BuildSlave.
>
>> https://sourceware.org/glibc/wiki/Buildbot
>
> I'll update this wiki page as I complete each of the steps.

Most of the steps to create a BuildSlave are already documented in our wiki
page.  However, I still have to review step 7, which is the process
to create a patch adding the BuildSlave.  I don't believe it makes sense to
automate this step, but we should have at least some hints on what needs to
be changed.

I'm attaching an Ansible playbook as RFC that does the exact same thing
documented there.

-- 
Tulio Magno

[-- Attachment #2: glibc-buildslave.yml --]
[-- Type: application/octet-stream, Size: 4611 bytes --]

---
# This playbook prepares a glibc BuildSlave.

# Steps to use this playbook:
#
# 1. Set the values of the following variables:
#    - slave.name
#    - slave.user
#    - slave.pass
#    - admin.name
#    - admin.mail
#
# 2. Set your hosts file, e.g.:
#  [buildslaves]
#  myserver.domain
#
# 3. Call ansible-playbook, e.g.:
#  ansible-playbook -i hosts glibc-buildslave.yml -K
#
# TODO: Add support to setup multiple BuildSlaves at the same time.
#       This will require to use more host_vars/ and one file / host,
#       making it more complex to share this playbook.

- name: Setup a glibc BuildSlave
  hosts: buildslaves
  vars:
    - slave:
        # Name of the BuildSlave.
        name: replaceme
        # Username that will run the BuildSlave.
        user: glibc-bot
        # Password that has been 
        pass: replaceme
    - admin:
        # Your name.
        name: Replace Me
        # Your email address.
        mail: mail@example.com
    # Address of the master.  Usually, you shouldn't change it.
    - master_addr: 127.0.0.1:9991 #130.211.48.148:9991
    # Default value for keepalive.  Usually, you shouldn't change it.
    - keepalive_default: 200

  tasks:
    - name: Create BuildSlave user
      user: name="{{slave.user}}"
      become: true

    - name: Install dependencies
      package: name="{{item}}" state=latest
      with_items:
        - gcc
        - make
        - buildbot-slave
        - git
        - binutils
        - texinfo
        - gawk
        - perl
        - sed
      become: true

    - name: Install specific packages for Debian-based distros
      package: name="{{item}}" state=latest
      with_items:
        - g++
        - libc6-dev
      when: ansible_os_family == "Debian"
      become: true

    - name: Install specific packages for RedHat or Suse-based distros
      package: name="{{item}}" state=latest
      with_items:
        - gcc-c++
        - glibc-static
      when: ansible_os_family == "RedHat" or ansible_os_family == "Suse"
      become: true

    - name: Get user home
      shell: echo $HOME
      changed_when: false
      register: home_dir
      become: true
      become_user: "{{slave.user}}"

    - name: Create the BuildSlave
      command: >
        buildslave create-slave
        "{{slave.name}}"
        "{{master_addr}}"
        "{{slave.name}}"
        "{{slave.pass}}"
      args:
        chdir: "{{home_dir.stdout}}"
      become: true
      become_user: "{{slave.user}}"

    - name: Ensure keepalive value
      lineinfile: >
        dest="{{home_dir.stdout}}/{{slave.name}}/buildbot.tac"
        regexp="^keepalive = "
        line="keepalive = {{keepalive_default}}"
      become: true
      become_user: "{{slave.user}}"

    - name: Set Admin information
      shell: echo "{{admin.name}} <{{admin.mail}}" > admin
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"

    - name: Add LSB description
      shell: echo "{{ansible_lsb.description}}" > host
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"

    - name: Add kernel information
      shell: uname -srv >> host
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"

    - name: Add null line
      shell: echo >> host
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"

    - name: Add gcc version
      shell: >
        gcc -v 2>&1
        | tail -n 1 -
        | sed 's/gcc version \([^ ]\+\) .*/g++ \1/' >> host
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"

    - name: Add Binutils version
      shell: >
        objdump -v 2>&1
        | head -n 1 -
        | sed 's/.* version \([^ -]\+\).*/binutils \1/' >> host
      args:
        chdir: "{{home_dir.stdout}}/{{slave.name}}/info"
      become: true
      become_user: "{{slave.user}}"


# TODO: Prepare a patch adding your BuildSlave to the master settings.

    # - name: Start the BuildSlave
    #   command: buildslave start
    #   args:
    #     chdir: "{{home_dir.stdout}}/{{slave.name}}"
    #   become: true
    #   become_user: "{{slave.user}}"


    - name: Use cron to restart the BuildSlave after reboots
      cron: >
        name="Start the BuildSlave"
        special_time=reboot
        job="buildslave start {{home_dir.stdout}}/{{slave.name}}"
      become: true
      become_user: "{{slave.user}}"

Re: buildbot status

[Carlos O'Donell]

On 04/19/2016 11:09 AM, Tulio Magno Quites Machado Filho wrote:
> Tulio Magno Quites Machado Filho <tuliom@linux.vnet.ibm.com> writes:
> 
>> Carlos O'Donell <carlos@redhat.com> writes:
>>
>>> On 03/03/2016 05:38 PM, Roland McGrath wrote:
>>>> For the new bugzilla component, I'm automatically CC'd but there is no
>>>> default assignee.  I didn't want to imply that I will necessarily jump on
>>>> every buildbot bug filed myself, though I will try to keep track of it.
>>>> Indeed I would very much like to encourage more folks to get involved with
>>>> improving the bot stuff.
>>>
>>> I'd love to just repeat the same process Tulio used :-)
>>>
>>> Tulio,
>>>
>>> Any status on updating the wiki page with more instructions?
>>
>> Sorry.  Give me a few more days.
>> I'm validating the steps in a new BuildSlave.
>>
>>> https://sourceware.org/glibc/wiki/Buildbot
>>
>> I'll update this wiki page as I complete each of the steps.
> 
> Most of the steps to create a BuildSlave are already documented in our wiki
> page.  However, I still have to review step 7, which is the process
> to create a patch adding the BuildSlave.  I don't believe it makes sense to
> automate this step, but we should have at least some hints on what needs to
> be changed.

As long as you have hints, thats the best.

> I'm attaching an Ansible playbook as RFC that does the exact same thing
> documented there.

Awesome!

Cheers,
Carlos.
 


-- 
Cheers,
Carlos.