moving testcase submissions to testsuite

moving testcase submissions to testsuite

[Robert Lipe]

Mike Simons wrote:

>   Speaking of testcases... do all of the short code bug examples reports 
> posted to this list turn themselves into additional testsuite cases?

Unfortunately, bits in the $MAIL stream don't magically mutate into
source code bits in gcc/testsuite/*.  A human has to do it.  Sometimes
a human will do it - I've noticed that Mark Mitchell in partcular is
really good about "here's a test case that fails, here's a patch to fix
it, and here's a testsuite entry to ensure that this never fails again".

It doesn't look like anyone has been scanning the lists and jamming them
into testcases.  This would be a lovely task for someone with a few
hours on their hands to help improve the quality of the compiler even if
you're not an industrial grade RTL wizard.

I've just picked through the EGCS-BUGS archives and pulled out all the
messages that look like they contain something that might be useful
to stick into testsuite cases.  (These are the ones that weren't
installation or bootstrap problem or followed up by "your code is
broken", etc.) There are still something like 800 of them.  After I get
them whittled down a little more I will try to work out a strategy for
reducing the working set down to a more manageable number.  I would
welcome some help in the process. 

A depressing percentage of the cases seemed to be g++ aborting when
faced with some of those newfangled language features.   That's just
an impression.

>   If not, how should they be packaged so they *are* added to the 
> regression suite?  (I don't know how dejagnu works)

Look in gcc/testsuite/*/* for examples.  "When in dejagnu, acts as the 
dejagnudists do."

>   Is posting information about failed tests useful?  (test name, it's 
> exit code... maybe a backtrace)

If you have a target that isn't well covered by the regular testers,
there's probably some value in it.  There are a couple folks that have
automated the build/test process routinely post the results.


RJL