Re: Again the build is broken :(

[Kris Van Hees <kris.van.hees@oracle.com>]

Well, 

1) If the time at which the automated builds run is significant in any way, we
   have a problem with our development procedures, because the quality of code
   (and the commits) should obviously not be time dependent.
   Also note that the builds are currently being executed at the following time
   slots:

	coldstone	3:34 am EST
	ca-tools2	4:45 am EST

   Given that the entire build-and-test cycle currently takes about 1 hour,
   that puts the completion of the latter right before EST dawn.

   And you can of course schedule your own builds using the automated build and
   test system at any time you want.  In fact, I'd highly recommend doing that,
   because it helps out everyone by increasing the overall build-and-test
   coverage.

2) I think you would definitely make a great contribution to the testing of
   frysk if you could create a per-commit test system.  I also believe it would
   make a nice complement to the current build-and-test system.  I look forward
   to hearing (and seeing) more about this.

   I'd like to point out though that there is nothing fuzzy about the point in
   time where a specific build was done by the automated build-and-test system.
   It is in fact possible (quite easily) to reconstruct the exact source tree
   that was used for the build (using CVS, so there is full access to commit
   log messages, etc...)

3) I don't think anyone has been reporting disaster in the event of build
   failures due to commits breaking the build, but I do believe that it is
   worth mentioning when it happens.  At a minimum, it raises the point that
   we are not as successful as we could be in making sure that commits are not
   breaking the builds.  Also, almost every single build failure *has* been
   discussed on IRC as soon as it was found (and resolution usually followed
   fairly shortly thereafter).  In this particular case, there was no IRC
   discussion (or at least, not initiated by me) simply because I was unable to
   connect to IRC at that given time.  Rather than not mentioning anything,
   sending mail to the list seemed quite reasonable under those circumstances.

	Cheers,
	Kris

> Elena Zannoni wrote:
>       
>> Sure, the two things you mention are not mutually exclusive.
>> However there is a cost to identifying broken builds too, and it seems 
>> that Mark is drawing the
>> short straw frequently, since he is usually the first to correct said 
>> oversights. It takes away some
>> of his time from development.
>>         
> Quite frequently, during the American day, we encounter and quick fix 
> similar issues (just today there was another "oops"), and we're 
> successfully and co-operatively managing these hiccups through our IRC 
> chatter and/or through bug reports and commits.
>
> However, we do need to be careful. Both Pearly and Mark pick up what I'll 
> call the night shift (from my tz pov) and so occasionally might be first 
> to encounter a problem also encountered by this build system. There I 
> think the most important thing is for us to be careful that we don't 
> message an expectation that Pearly and/or Mark are some how expected to 
> down-tools and focus all energies on getting it fixed.  As with us during 
> the day, back-date the check-out for a few hours 'til things are resolved; 
> for mark using mecurial, this is trivial.
>
> Can I suggest:
>
> - Moving the build farm's time to run just before US dawn so results are 
> better timed for us waking up; or better ...
>
> - setting up a test system that makes available results from individual 
> commits and not fuzzy dates
>
> - accept that an occasional build failure in the build farm does not 
> require an immediate post about the sky falling; I for instance would only 
> be concerned if the build failed consistently and for an identical reason 
> across two work days; and then my first response is still going to be to 
> fix it.
>
> Andrew
>
>
>       
>> I haven't suggested that you or anybody checks every combination
>> before checking stuff in. What I have suggested is that, like we used to 
>> do once upon a time, we
>> stick with as few development platforms as we can get away with in order 
>> to minimize the
>> oversights. So if the platforms supported are FC6 and F7, let's stick 
>> with those and make
>> everybody's life easier. If somebody wants to add FC5 to the test grid, 
>> please do so and contribute
>> the tests results so that they can be uploaded. Any takers?