Re: Preparing for the GDB 5.0 / GDB 2000 / GDB2k release

Re: Preparing for the GDB 5.0 / GDB 2000 / GDB2k release

[Tom Tromey]

Benjamin> note that this is somewhat related to a criticism of the
Benjamin> sourceforge site: the build machine is x86-linux only.

They have a build machine?  I didn't know that!
Now it is clear that we must do this.

Benjamin> ??-linux

ARM.

Tom

Re: Preparing for the GDB 5.0 / GDB 2000 / GDB2k release

[Benjamin Kosnik]

note that this is somewhat related to a criticism of the sourceforge 
site: the build machine is x86-linux only. 

If sourceware was to provide say an

alpha-linux
powerpc-linux
x86-linux
??-linux
(? solaris26?)

build machines, and tie in an automated test program a la nightly testing 
or the mozilla thing, well. .. .  that would be something that would kick 
ass.

-benjamin

On 8 Feb 2000, Tom Tromey wrote:

> Jason> Maybe it would be better to get some test results from a
> Jason> variety of common Unix platforms and decide based on how things
> Jason> look.  NB cygwin support in binutils is noticably broken -- it
> Jason> will take at least a little work to get that resolved.
> 
> It would be really cool if sourceware/Red Hat had automated
> multi-platform testing for hosted projects.  Or maybe security
> concerns outweigh the coolness factor.
> 
> Tom
> 

Providing automated testing for projects on sourceware

[Jason Molenda]

On Tue, Feb 08, 2000 at 11:02:49AM -0700, Tom Tromey wrote:

> It would be really cool if sourceware/Red Hat had automated
> multi-platform testing for hosted projects.  Or maybe security
> concerns outweigh the coolness factor.

There are two sources of test results:

1.  Get volunteers around the net to do the testing.  They mail in
    test results in a standard format and there are scripts and things
    to munge this into a useful summary.

    GCC is doing this, I think it is mostly the work of Marc
    Lehmann.  He seems to be inputting the test results that are
    sent in into an SQL database.  He uses metahtml to generate the
    http://gcc.gnu.org/testresults/ and gcc.gnu.org/benchresults web
    pages based on the data in this database.  For an example, look at
    this summary of the new failures that occured in the gcc-2000-01-28
    snapshot:

	http://gcc.gnu.org/testresults/cs866.html

    There are no security problems here for us; people e-mail in their
    test results to the test collector program and the test collector
    program presumably does not us gets().

2.  Do it ourselves.

    I imagine we could find some cash to buy test hardware.  The hardware
    could do regular checkouts of repositories on sourceware, build them
    and run the testsuites, and transmit the results back to sourceware
    for summarization.

    From a security standpoint, the test-boxes would have to be carefully
    isolated from other machines.  We'd have to assume that an evil dude
    will check in naughty code to a repository just before the test box
    starts its checkout, and therefore the testboxes may be compromised.
    I don't think we'd have to go as far as isolating them on their
    own subnet--you can only get in to sourceware via Kerberos or SSH
    authentication so there aren't any cleartext passwords that could
    be sniffed off the network.  

    The testbox would mail its summaries back to sourceware, again keeping
    the possibility for crackers escaping the testbox to a minimum.


As you can see, there's no real difference in the implementation for
these two approaches--a combination of the two is entirely possible
and preferrable.  #2 will guarantee that tests are run regularly on a
certain host or set of hosts.  #1 will help supplement that with a lot
of weirdo hardware that we don't have locally.

You'll notice that I'm assuming some consistency in the way the test
results are reported.  This is true for all of our Cygnus stuff that
uses dejagnu, but other projects are going to use arbitrary formats for
their test results.  Possibly some kind of interchange format could be
defined and projects using other test reporting formats could rewrite
their test results into the interchange format before they get read in
to the test result tracking database on sourceware.

I'm sure this goes without saying, but there is no way I'll even think
about this stuff before I leave Cygnus--this is clearly a forward
looking thing that will take some real time and effort to happen.

Jason

Re: Preparing for the GDB 5.0 / GDB 2000 / GDB2k release

[Benjamin Kosnik]

from a recent exchange with Chip Salzenberg, whereby I was shut down in 
my attempt to get VA to pimp out hardware via this very issue:

--chip:
There's no procedure (AFAIK).  But what about shell.sourceforge.net?
It's a dual PIII/600, and I can get you an expanded quota.  Just sign
up as a SourceForge developer, which automatically provides shell
access, and I'll do the rest.  (<URL: http://sourceforge.net >)

... But of course that would only be really useful if you have fast
access.  You're not stuck with a 56K modem, are you?
--end chip


> ARM.

even better let's get intel to donate some of that flashy strongarm2 stuff.

one can dream. . .

-benjamin

Re: Preparing for the GDB 5.0 / GDB 2000 / GDB2k release

[Tom Tromey]

Jason> Maybe it would be better to get some test results from a
Jason> variety of common Unix platforms and decide based on how things
Jason> look.  NB cygwin support in binutils is noticably broken -- it
Jason> will take at least a little work to get that resolved.

It would be really cool if sourceware/Red Hat had automated
multi-platform testing for hosted projects.  Or maybe security
concerns outweigh the coolness factor.

Tom