Hi there, 0. Introduction =============== I am happy to announce that I have finally finished configuring our BuildBot instance to accept "try jobs". What this means is that a developer will be able to upload a patch to our BuildBot, and this patch will then be tested against a set of builders we have there. In the paragraphs below I will explain how this feature works, who can use it (and how), and try to answer a few questions that I think you will have. 1. Concepts =========== As you know, our BuildBot master runs on my personal server, and we have several "slaves" running on different locations. The master doesn't do any builds itself, but rather controls all the details about the builds that need to be performed. The slaves do not take any decision by themselves; they just do what the master tells them to do. On top of the slaves, there is an abstraction called "builder". The builder is basically a recipe of how to do a build. This includes the compilation flags used, environment variables, and testsuite options. A slave can have multiple builders; a builder can also be assigned to multiple slaves (for example, when we have more than one slave with the same hardware configuration, as is the case of the Fedora x86_64 slaves). Then, we have the schedulers. They are responsible for monitoring events and triggering builds. Our main scheduler monitors our repository, and triggers a full build on all builders whenever a new commit has been pushed. We also have "cron job" schedulers, used to trigger racy testcase detection builds every week at a certain time. And now, we have the "try job" scheduler, responsible for triggering a build on a set of builders when a developer has submitted a patch for testing. 2. Build priority ================= Before the try scheduler, builds were basically treated in a "first come, first served" basis. No special priority was assigned to a build. Now, I decided to try a new algorithm. Considering that a developer usually want to test patches when they are almost ready for submission, it would be really annoying if she had to wait in line until her build is processed; it is not rare to see some builders having 10 or more builds in the queue. So, my decision was to "move" the try builds to the top of the queue. This will obviously delay the testing of the official repository, but I am hoping that the impact will not be very big. If it is, then this decision will have to be revisited. 3. Security =========== I am currently talking to the buildslave owners off-list in order to make sure that we are on the same page here, but there are a few considerations to be made about the security of a try job. The obvious one is that a developer may upload a malicious patch to be "tested", and there is no easy way for BuildBot to detect this. Since the BuildBot master does not build anything, it would be the slaves who would suffer From this kind of attack. But there is another thing to consider. In order to be able to upload a patch to be tested, the developer must have some kind of access to the BuildBot master. Yes, my personal server. No, I am not happy about this. I will write more on the authentication mechanism below. So, after giving a long thought about this problem, I decided that I will only provide accounts to: - Global maintainers. - Someone I know personally, or that I feel confident enough that is a trustworthy person. (yeah, this is very subjective, but it is my machine :-/) If you do not fit into one of these categories, you can ask someone from one of the above categories to vouch for you. This person will need to contact me. 4. Authentication ================= The BuildBot try command can use SSH for authentication, so that is what I have chosen. Basically, you will need to send me your *public* SSH key; no passwords are needed/accepted. You will be using a common user to log in, and will not receive a PTY (i.e., will not have shell access). If you ever used gitolite, then you know the drill. If not, it's pretty simple. I will provide a configuration file below. 5. Installing buildbot ====================== In order to use the new feature, you will have to install the "buildbot" package. You don't need the "slave" package! On Debian, an "apt-get install buildbot" should do the trick. On Fedora, "dnf install buildbot". However, Fedora enables the BuildBot systemd service after the installation; I strongly recommend you disable that, unless you intend to run a BuildBot master on your machine. 6. The "buildbot try" command ============================= After installing the package, the only command you need to know is "buildbot try". It will be responsible for generating a patch out of the git tree you're in (assuming you don't provide a patch explicitly), connecting to the server and submitting the patch to be tested. I strongly recommend that you read the command's documentation at: However, the usage is pretty basic. There are two modes: you can either be in a local branch and ask "buildbot try" to automatically figure out the patch to be tested, or you can provide the patch file explicitly. In the first mode, "buildbot try" will generate a diff of your local branch against your "master" branch, and will instruct the BuildBot master to test the patch *using the master's branch HEAD revision*. This means that if you want to generate a patch to be tested against the latest commit of the official repository, you have to do a "git pull" on your master branch first (and rebase your local branch, of course). The second mode accepts a patch file via the "--diff" parameter. Originally, "buildbot try" will not specify a revision against which the patch needs to be tested; however, I have modified that behaviour on the server side, and the provided patch will always be tested against the latest commit of the official repository. 7. Configuration file for "buildbot try" ======================================== Here is the configuration file that you need to use for the "buildbot try" command to work. This file needs to be named <$HOME/.buildbot/options>. $> cat $HOME/.buildbot/options # The method of connection to the BuildBot master. try_connect = 'ssh' # The host of the BuildBot master. try_host = 'gdb-build.sergiodj.net' # Our username. We will always use this username! try_username = 'gdb-buildbot' # The jobdir on BuildBot master (i.e., the directory where "buildbot # try" will put the request files. try_jobdir = 'gdb/try_ssh_jobdir' # The VCS we are using for the project. try_vc = 'git' # The branch against which we will try our patches. Currently, we # only support 'master'. try_branch = 'master' # Who am I? # NOTE: YOU NEED TO USE THE FORMAT "NAME " IN THIS FIELD! try_who = 'John Smith ' # The builders I want to test my patch in. # The list of available builder can be found in: # try_builders = [ "Fedora-x86_64-m64", "Fedora-x86_64-m32", "Fedora-x86_64-native-gdbserver-m64", "Fedora-x86_64-native-gdbserver-m32", "Fedora-x86_64-native-extended-gdbserver-m64", "Fedora-x86_64-native-extended-gdbserver-m32", "Fedora-x86_64-cc-with-index", "Fedora-i686" ] $> It is good to mention that most of these options can be overriden by command line options, so you can (for example) choose to test your patch against only a subset of the available try_builders. 8. Notifications ================ The "buildbot try" command offers a way to wait for the builds to complete on the command line, but this doesn't work when you're using SSH. However, you will receive e-mails from our BuildBot notifying you about the results of all of your try builds. The e-mails will be very similar to those sent to the gdb-testers mailing list. 9. Build specifics ================== As mentioned above, the try builds will almost always be performed against the latest commit on the official repository. They will also run using the very same flags that are being used for each builder (unfortunately, there is no way to change these flags on try builds). And the results will be compared against the last results that our BuildBot has on file. For example, if our BuildBot has just tested the commit 0abcd on a certain builder, and then you request it to test your patch on the same builder, BuildBot will run the full build of your patch, generate the testsuite logs, and compare them against the testsuite logs generated by testing 0abcd. The known XFAIL files for that specific builder will also be taken into account, i.e., they will be discarded when comparing the testsuite logs. Last, but not least, your try build will generate its own testsuite logs, which will be recorded in that builder's git repository, available at: 10. More info ============= I guess this e-mail already covers quite a lot, but I also intend to populate our wiki with more info. The page will be: And, of course, I am open to suggestions for improvements (can't promise I'll be able to dedicate much time these days, though). ---- I guess that is it. Please let me know if you have any questions, and I will be happy to (try to) answer! Cheers, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible http://sergiodj.net/