From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 32334 invoked by alias); 5 Apr 2007 20:48:46 -0000 Received: (qmail 32319 invoked by uid 22791); 5 Apr 2007 20:48:44 -0000 X-Spam-Status: No, hits=-1.1 required=5.0 tests=AWL,BAYES_50 X-Spam-Check-By: sourceware.org Received: from rgminet01.oracle.com (HELO rgminet01.oracle.com) (148.87.113.118) by sourceware.org (qpsmtpd/0.31) with ESMTP; Thu, 05 Apr 2007 21:48:39 +0100 Received: from rgmsgw02.us.oracle.com (rgmsgw02.us.oracle.com [138.1.186.52]) by rgminet01.oracle.com (Switch-3.2.4/Switch-3.1.6) with ESMTP id l35Kmaim016262; Thu, 5 Apr 2007 14:48:36 -0600 Received: from ca-server1.us.oracle.com (ca-server1.us.oracle.com [139.185.48.5]) by rgmsgw02.us.oracle.com (Switch-3.2.4/Switch-3.2.4) with ESMTP id l35KmZ7r009563 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO); Thu, 5 Apr 2007 14:48:36 -0600 Received: from kvanhees by ca-server1.us.oracle.com with local (Exim 4.63) (envelope-from ) id 1HZYsl-0000kd-9j; Thu, 05 Apr 2007 13:48:35 -0700 Date: Thu, 05 Apr 2007 20:48:00 -0000 From: Kris Van Hees To: Mark Wielaard Cc: Kris Van Hees , Frysk Mailing List Subject: Re: ANNOUNCEMENT: Release of the automated build-and-test system Message-ID: <20070405204835.GA28259@ca-server1.us.oracle.com> References: <20070329050103.GA6571@ca-server1.us.oracle.com> <1175517820.4460.39.camel@dijkstra.wildebeest.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1175517820.4460.39.camel@dijkstra.wildebeest.org> User-Agent: Mutt/1.5.11 X-Brightmail-Tracker: AAAAAQAAAAI= X-Brightmail-Tracker: AAAAAQAAAAI= X-Whitelist: TRUE X-Whitelist: TRUE X-IsSubscribed: yes Mailing-List: contact frysk-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: frysk-owner@sourceware.org X-SW-Source: 2007-q2/txt/msg00014.txt.bz2 Checking in from vacation (hence the late response)... On Mon, Apr 02, 2007 at 02:43:39PM +0200, Mark Wielaard wrote: > Hoi Kris, > > On Wed, 2007-03-28 at 22:01 -0700, Kris Van Hees wrote: > > I am happy to announce that the automated build-and-test system is > > available for general participation. This is a major milestone in the > > development of this system, because it allows direct (and safe) > > contributions to the overall coverage that automated tests can provide > > by anyone interested in donating some cycles on their system(s). > > Thanks! Doing a test build now. Great! If you are able to commit a few cycles every night or so, please let me know (and provide me with the uname -a output for the system you can run it on) and I will send you a password to enable you to upload results. > > You can find more information (and instructions for download and > > installation) at: > > > > http://build.alchar.org/~aedil/ > > > > Please note that this system is currently in *beta* mode, so things are > > still changing in various places, and the central reporting is not yet > > operational (although upload of logs is). I look forward to comments. > > It is pretty generic, but also very centralistic. Could the build > scripts, or at least the frysk_*.pkg files be stored in frysk CVS? > Likewise for mailing/uploading the results. That way a project could > handle its own tests and result processing. The system is not meant to be Frysk specific and truly intended to be outside of Frysk CVS for all of its functions. If I were to put the frysk_*.pkg files in Frysk CVS, then people would not be able to use those files without having the build_farm scripts, and the other way around. It is meant to be an independent unit that pulls the source tree of Frysk (and any other project we define in it), and performs a build-and-test run on it. The project (Frysk in this case) is still responsible for defining tests in its testsuite. And (of course) there is also a project specific piece on the reporting server that parses the project specific data from the logs. But I do think that there is more value in keeping that outside of Frysk rather than putting it in Frysk CVS. The reporting piece has more dependencies on the build_farm stuff than it has on any of the supported projects. Another reason for being centralistic is that reporting is meant to be done on a central system, where support for a variety of projects can be coordinated. The goal is to have an automated build-and-test system that is very lightweight, easy to deploy, and with minimal dependencies. > Asking contributors to request a password is an extra step that I would > not add as long as you can identify different build hosts by email > address or ip-address. Lowering barriers to contribute is good. See for > example how gcc does it, you can do an anonymous checkout and then run > the test scripts that will email the results to a public list so > everybody can easily contribute and get at the raw results: > (See the various build scripts and the test_summary script) > http://gcc.gnu.org/viewcvs/trunk/contrib/ > http://gcc.gnu.org/ml/gcc-testresults/current/ The main reason for putting up a small barrier is that we obviously are facing potential malicious uploads. While a password-protection is only a limited level of security, it does pose enough of a barrier to avoid the majority of pests. Given that the central server will provide access to the results, both in parsed and raw format, everybody should be able to get to that data at will. The advantage of using an upload area and a central reporting server is that we have a place where (rather large) logs are stored. If a bug is found in the parsing scripts, it is easy enough to re-run reporting on older logs. Doing this on a central server also allows us to automatically run comparisons between various build-and-test runs. In all, it offers (limited) data mining possibilities that otherwise would only be available if we were to have nodes mail the full logs, and that would be rather heavy. > You say that you keep version information as a list of CVS revision ids. > Wouldn't a date/time suffice? Or just a revision tag in the tree? Well, perhaps. However, note that CVS keeps revisions on a per-file basis, and that commits are not exactly atomic (when multiple files are commited at the same time). Given the granularity of date/time stamps, there is a potential for differences between the tree used for the build and one to re-create the build. What we do is actually a combination. The revision list contains a date/time stamp as well (in UTC), which is used as the first indicator of a tree revision. After doing a checkout using the date/time stamp found in the rev file, the per-file revision ids are compared between the checked out tree and the rev file, and any discrepancies are resolved using the revision id from the rev file. that guarantees that the tree will be identicial to the one used to generate the rev file. Using a revision tag in the tree would work if we were having nodes do coordinated builds, i.e. builds of specific revision tags, like one per night. The trouble is that we'd be cluttering up the CVS space with tons and tons of tags that have only a limited use in time, and we also end up having nodes perform build-and-test runs on revisions that may already be known to be bad. By having every node set their own schedule for when they run tests, and have them test against the current CVS HEAD, we cover a lot more ground. > BTW. There are no distribution terms on these sources, could you put > them under the GPL? Thanks for catching this. The scripts have now been released under the GPLv2. Cheers, Kris