Re: Automated build-and-test summary report (2007/05/23)

[Kris Van Hees <kris.van.hees@oracle.com>]

I somehow fail to see the value of crippling the system by reducing the
amount of useful reporting, when it should be quite easy for you to
obtain the information you want based on the more detailed, more
frequent reports.  The point of the automated build-and-test system is
to perform the build-and-test runs, and collect the output for reporting
purposes.  The point of the system is not to do project-specific analysis
based on that data.  All the data (even the raw data) is available to
you from the the web pages generated by the automated build-and-test
system.

Also note that I am a bit reluctant to make significant changes to a
generic system (i.e. that is *not* frysk specific) that has yet to
receive any outside contribution in terms of build-and-test resources.
Note that the system is in fact generic enough to allow you to do
exactly what you want on your own system, i.e. you could use it to have
nightly (or even more frequent) build-and-test runs executed, capture
the logs and parse them at your heart's content.  Or let the central
reporting system process them, and you pull them back off the system for
your own reporting.

The opportunities are virtually endless...  let's not limit the
flexibility it offers for a single very specific request that could
easily be implemented on top of the existing system.

	Cheers,
	Kris

On Thu, May 24, 2007 at 01:35:50PM -0400, Andrew Cagney wrote:
> Hi Kris,
> 
> Is it possible to separate out the building and reporting functions so 
> that there's a weekly summary showing trends?  In general, crasher bugs 
> are found quickly and directly by running the test-suite locally.  Its 
> the more obscure failures, that are missed by normal testing, and 
> require drilling down through change/build history that are really 
> helped by this system.
> 
> Andrew
> 
> Kris Van Hees wrote:
> >I may be missing something, but I honestly do not understand the issue
> >here.  Obviously, an automated build-and-test system that performs
> >nightly builds is hardly any use if the results are only emailed out
> >once a week.  The continuing existence of tests that show intermittent
> >failures, and the continued failure of the dist-builds (that you in fact
> >suggested be added to the list of configuration) indicates that we (as
> >developers) are not detecting and reporting all problems.  The automated
> >system is performing a full-scale test, and extracts the relevant output
> >from the tests, consolidating those in clear reports.
> >
> >However, if receiving a daily report in email places an undue burden on
> >you, I can certainly disable the scheduling of nightly builds for Frysk.
> >There is no point in running tests when you don't want the results
> >distributed to the developers.
> >
> >	Cheers,
> >	Kris
> >
> >On Wed, May 23, 2007 at 11:24:35AM -0400, Andrew Cagney wrote:
> >  
> >>Kris,
> >>
> >>Does this really need to be semi-daily?  I thought it was happily 
> >>posting weekly summaries.  As developers we can more directly detect and 
> >>report immediate problems.
> >>
> >>Andrew
> >>
> >>    
>