Test suite for new front end: DejaGNU or QMtest

Test suite for new front end: DejaGNU or QMtest

[Steven Bosscher]

Hello,

Now that G95 generates some code, we'll soon need to be able to run test
cases. Eventually we want to integrate G95 in GCC, so I guess we should
write test cases for the testing framework that GCC will use in the
future.

GCC uses DejaGNU now, but I read in some post that g++ will use QMtest
in the (near?) future, and that other parts of GCC will use it, too. So
should a new front-end test suite use DejaGNU or QMtest?

Greetz
Steven

Re: Test suite for new front end: DejaGNU or QMtest

[Daniel Jacobowitz]

On Sun, Jun 30, 2002 at 11:54:08PM +0200, Steven Bosscher wrote:
> Hello,
> 
> Now that G95 generates some code, we'll soon need to be able to run test
> cases. Eventually we want to integrate G95 in GCC, so I guess we should
> write test cases for the testing framework that GCC will use in the
> future.
> 
> GCC uses DejaGNU now, but I read in some post that g++ will use QMtest
> in the (near?) future, and that other parts of GCC will use it, too. So
> should a new front-end test suite use DejaGNU or QMtest?

As far as I know there is no intention to replace the DejaGNU testsuite
in the short- to medium-term - only long-term if QMtest proves viable. 
So for the moment I recommend continuing with DejaGNU, since that
framework exists now, and since converting over seems to be a
relatively simple process.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: Test suite for new front end: DejaGNU or QMtest

[Mark Mitchell]

--On Sunday, June 30, 2002 09:19:16 PM -0400 Daniel Jacobowitz 
<drow@mvista.com> wrote:

> On Sun, Jun 30, 2002 at 11:54:08PM +0200, Steven Bosscher wrote:
>> Hello,
>>
>> Now that G95 generates some code, we'll soon need to be able to run test
>> cases. Eventually we want to integrate G95 in GCC, so I guess we should
>> write test cases for the testing framework that GCC will use in the
>> future.
>>
>> GCC uses DejaGNU now, but I read in some post that g++ will use QMtest
>> in the (near?) future, and that other parts of GCC will use it, too. So
>> should a new front-end test suite use DejaGNU or QMtest?
>
> As far as I know there is no intention to replace the DejaGNU testsuite
> in the short- to medium-term - only long-term if QMtest proves viable.
> So for the moment I recommend continuing with DejaGNU, since that
> framework exists now, and since converting over seems to be a
> relatively simple process.

As the prime proponent of QMTest, I agree with most of what Daniel says.

The state that we are in with QMTest is that most people seem moderately
pleased with the tool -- modulo the fact that all the right test classes
for supporting complicated cross environments are not in place.

We don't know yet whether CodeSourcery (or someone else) will do the work
required to implement that support.

DejaGNU is certainly the GCC standard at this point.

On the other hand, conversion is not entirely trivial; it takes work to
write test classes that can understand the DejaGNU code.  Things will be
simpler if you modify the existing DejaGNU code as little as possible;
the more your tests look like the C or C++ tests the simpler the
conversion will be.  (And, the more consistent the tests will be internally,
even if we never convert!)

So, I would suggest that you write the tests using the "dg" style of
DejaGNU test, as found for the tests in the "g++.dg" subdirectory.

--
Mark Mitchell                   mark@codesourcery.com
CodeSourcery, LLC               http://www.codesourcery.com