Re: gdb test results web page

Re: gdb test results web page

[Michael Elizabeth Chastain]

Tom Tromey writes:
Tom> Can you suggest how to do this?  I don't see a way to tell the two
Tom> kinds of files apart.

Here are some differences to key on for gdb.sum versus gdb.log:

  A gdb.log file has several '^Using' lines in the first few lines.

  A gdb.log file ends with the line '^runtest completed ...'

You could also parse the '^# of ...' totals near the end of the file,
and compare them with the count you get by grepping.  If the counts do
not match you could reject the file.  That would protect against lots of
kinds of user errors, script shortcomings, and bogus input files.

Tom> Hmm.  One approach would be to fix dejagnu so that pass/fail
Tom> indications always start on a new line.  At least, this doesn't seem
Tom> unreasonable to me.

Ouch, an interface problem.  You are suggesting we change this:

  Program exited with code 01.^M
  (gdb) PASS: gdb.base/a2-run.exp: run "run" with no args

To this:

  Program exited with code 01.^M
  (gdb)
  PASS: gdb.base/a2-run.exp: run "run" with no args

I have two concerns.  First, if we make this change in Dejagnu, it affects
all tools tested with Dejagnu.  And second, it means that nobody can enter
any test results until this change gets made and propagated in Dejagnu.

So I'm more inclined to say: leave the sender [dejagnu] alone, and change
the receiver to be more liberal.

Michael

Re: gdb test results web page

[Tom Tromey]

>>>>> "Michael" == Michael Elizabeth Chastain <chastain@cygnus.com> writes:

Michael> I have two concerns.  First, if we make this change in
Michael> Dejagnu, it affects all tools tested with Dejagnu.  And
Michael> second, it means that nobody can enter any test results until
Michael> this change gets made and propagated in Dejagnu.

I think the current behavior is just a dejagnu bug.  I doubt anybody
relies on "PASS" or "FAIL" not appearing at the start of a line.  And
I doubt it was ever intended to work the way it does -- dejagnu wasn't
that carefully built.

Second, I don't think propagation is a big problem.  I thought
everybody was supposed to use the dejagnu in the src repository.

If we were dealing with non-free software I would agree with you.  But
I think with free software we can afford to have a higher standard --
fix the bug where it lies, not work around it elsewhere.

Tom

Re: gdb test results web page

[Christopher Faylor]

On Thu, Jan 18, 2001 at 11:36:20AM -0700, Tom Tromey wrote:
>>>>>> "Michael" == Michael Elizabeth Chastain <chastain@cygnus.com> writes:
>
>Michael> I have two concerns.  First, if we make this change in
>Michael> Dejagnu, it affects all tools tested with Dejagnu.  And
>Michael> second, it means that nobody can enter any test results until
>Michael> this change gets made and propagated in Dejagnu.
>
>I think the current behavior is just a dejagnu bug.  I doubt anybody
>relies on "PASS" or "FAIL" not appearing at the start of a line.  And
>I doubt it was ever intended to work the way it does -- dejagnu wasn't
>that carefully built.

It is not that hard to modify terf to find the PASS/FAIL/XFAIL in other
places than at the beginning of the line.  I've just checked in a change
to do this.

Do you think this is a bad idea, Tom?  If so, I'll revert the change.

cgf

Re: gdb test results web page

[Tom Tromey]

>>>>> "Chris" == Christopher Faylor <cgf@redhat.com> writes:

Chris> Do you think this is a bad idea, Tom?  If so, I'll revert the
Chris> change.

I doubt it is actively bad.
I've just reached my limit of catering to dejagnu brokenness.
Perhaps I'll send a dejagnu patch.

Tom

Re: gdb test results web page

[Andrew Cagney]

A quick thanks, I'll update the web pages.

Also a follow-up.  This was an unofficial item on the GDB 5.1 release
criteria.  There had been a number of requests for a mechanism that
allowed people to publish test results - this is it.

When it comes to 5.1 it should be possible to just look at the test
results pages and provided they are ok things can be released. (well
sounds good in theory :-)

	Andrew



Christopher Faylor wrote:
> 
> I've been meaning to announce the existence of the gdb test results web
> page for some time.  It is here:
> 
> http://sources.redhat.com/cgi-bin/gdb.cgi
> 
> This web page uses some cool software, written by Tom Tromey that helps
> in the categorization and comparison of test results.  You can upload
> your own test results here by clicking on the "upload a testsuite log
> file" link or by sending email to gdb-testresults@sources.redhat.com.
> 
> We're thinking of setting up a project on sources.redhat.com devoted to
> the "Terf" software which handles this web page so that people can
> improve it.  If anyone is interested in joining this project let me
> know.  You'll need to be fairly proficient in perl, since Terf is written
> in perl.
> 
> cgf

Re: gdb test results web page

[Tom Tromey]

>>>>> "Michael" == Michael Elizabeth Chastain <chastain@cygnus.com> writes:

Michael> The upload page should say more clearly whether it wants a
Michael> "gdb.log" file or a "gdb.sum" file.  And if it does not
Michael> handle both types, then it needs to reject files of the wrong
Michael> type.

Can you suggest how to do this?  I don't see a way to tell the two
kinds of files apart.

Tom

Re: gdb test results web page

[Tom Tromey]

>>>>> "Michael" == Michael Elizabeth Chastain <chastain@cygnus.com> writes:

Michael> It appears that the results page counts only results that
Michael> start in column 1.  That's fine for "gdb.sum" files, but it
Michael> misses 90% of the results in "gdb.log" files.  It looks like
Michael> every entry in the current database suffers from this
Michael> problem.

Hmm.  One approach would be to fix dejagnu so that pass/fail
indications always start on a new line.  At least, this doesn't seem
unreasonable to me.

Michael> The upload page should say more clearly whether it wants a
Michael> "gdb.log" file or a "gdb.sum" file.  And if it does not
Michael> handle both types, then it needs to reject files of the wrong
Michael> type.

It wants .log files.

Michael> I have a lot more opinions but I better shut up before I get
Michael> recruited!

Indeed.

Tom

Re: gdb test results web page

[Michael Elizabeth Chastain]

It looks promising, but I think it's not usable yet.

The upload page says:

  UPLOAD FOR GDB
  Enter the file name of a test suite log file to upload:
  --------------------------------------------------------
  |                                                      |
  --------------------------------------------------------

So I gave it my "gdb.log" file for 2001-01-15.

Now the results page says there were 375 PASS, 7 FAIL, 0 XPASS, and 0 XFAIL
for this test run.  That's way too few total tests.

It appears that the results page counts only results that start in
column 1.  That's fine for "gdb.sum" files, but it misses 90% of the
results in "gdb.log" files.  It looks like every entry in the current
database suffers from this problem.

The upload page should say more clearly whether it wants a "gdb.log"
file or a "gdb.sum" file.  And if it does not handle both types, then
it needs to reject files of the wrong type.

I have a lot more opinions but I better shut up before I get recruited!

Michael Chastain
<chastain@redhat.com>
"love without fear"

gdb test results web page

[Christopher Faylor]

I've been meaning to announce the existence of the gdb test results web
page for some time.  It is here:

http://sources.redhat.com/cgi-bin/gdb.cgi

This web page uses some cool software, written by Tom Tromey that helps
in the categorization and comparison of test results.  You can upload
your own test results here by clicking on the "upload a testsuite log
file" link or by sending email to gdb-testresults@sources.redhat.com.

We're thinking of setting up a project on sources.redhat.com devoted to
the "Terf" software which handles this web page so that people can
improve it.  If anyone is interested in joining this project let me
know.  You'll need to be fairly proficient in perl, since Terf is written
in perl.

cgf