Re: Testsuite (was: New features of old-dejagnu.exp)

Re: Testsuite (was: New features of old-dejagnu.exp)

[N8TM]

In a message dated 10/29/98 2:22:58 AM Pacific Standard Time,
pfeifer@dbai.tuwien.ac.at writes:

>  o the structure of the suite
>     (Where to add test cases? Naming conventions?)
>  
>   o the features of the testsuite
>     (Part of which you have described in your mail -- thanks!)
>  
>   o how to extend the testsuite (with additional features)
>  
Yes, I am making a variant on egcs/testsuite for use in testing "any" Fortran
compiler, and have made some progress.  I would appreciated such information.

Re: Testsuite (was: New features of old-dejagnu.exp)

[Gerald Pfeifer]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 5499 bytes --]

[ Please remove egcs@cygnus.com from replies. ]

On Wed, 4 Nov 1998, Mike Stump wrote:
>>  o the features of the testsuite
>>    (Part of which you have described in your mail -- thanks!)
> :-) I use grep and memory.  If someone wants to doc it up, I'd be
> happy to describe stuff.

As it was me who asked and you spent time responding and none else
seemed to bother, I felt to obliged to give it a first shot. :-)

This is by *no* means perfect, but I'd like to see this serving as a
starting point.

Okay to install?

Gerald


1998-11-07  Gerald Pfeifer  <pfeifer@dbai.tuwien.ac.at>

	* README: New file. General information about the testsuite and 
	new description of the various C++ test subdirectories.
	* README.g++: Eliminate obsolete information, update and move most
	relevant stuff to README.

Index: README
===================================================================
RCS file: README
diff -N README
*** /dev/null	Mon Dec 31 20:00:00 1979
--- README	Sat Nov  7 12:42:08 1998
***************
*** 0 ****
--- 1,45 ----
+ This is a collection of tests for the C and C++ frontends of EGCS. For
+ further information please check README.gcc and README.g++, though the
+ latter may not be up-to-date any more.
+ 
+ The driver that runs this testsuite is called DejaGnu and you will
+ need a current DejaGnu snapshot, which is available from
+ ftp://egcs.cygnus.com/pub/egcs/infrastructure , for example.
+ 
+ Most of these tests were originally developed and/or collected by
+ Cygnus Support, but these days many are being added by the EGCS
+ developers.
+ 
+ These tests are included "as is". If any of them fails, do not report
+ a bug.  Bug reports for DejaGnu can go to bug-dejagnu@prep.ai.mit.edu.
+ Discussion and comments about this testsuite should be sent to
+ egcs@cygnus.com; additions and changes to should go to sent to
+ egcs-patches@cygnus.com.
+ 
+ The entire testsuite is invoked by `make check` at the top level of
+ the EGCS tree. `make check-g++` runs the C++ testsuite only.
+ 
+ 
+ STRUCTURE OF THE TESTSUITE
+ 
+ All C++ tests reside in subdirectories of g++.old-deja, based on the
+ following structure:
+ 
+   g++.benjamin	Tests by Benjamin Koz
+   g++.bob
+   g++.brendan	Tests by Brendan Kehoe
+   g++.bugs
+   g++.eh  	Tests for exception handling
+   g++.ext  	Tests for g++ extensions
+   g++.gb  	Tests by Gerald Baumgartner
+   g++.jason	Tests by Jason Merill
+   g++.jeff	Tests by Jeffrey A Law
+   g++.martin	Tests by Martin v. Löwis
+   g++.mike	Tests by Mike Stump
+   g++.niklas	Tests by Niklas Hallqvist
+   g++.ns  	Tests for namespaces
+   g++.other
+   g++.pt  	Tests for templates
+   g++.rfg
+   g++.robertl	Tests from egcs-bugs@cygnus.com, gathered by Robert Lipe
+ 	
Index: README.g++
===================================================================
RCS file: /egcs/carton/cvsfiles/egcs/gcc/testsuite/README.g++,v
retrieving revision 1.1.1.1
diff -c -3 -p -r1.1.1.1 README.g++
*** README.g++	1997/08/19 07:34:41	1.1.1.1
--- README.g++	1998/11/07 20:42:08
***************
*** 1,9 ****
! This file needs a little updating, but is pretty close.  Here are some
! random last minute notes:
  
- make check and make check-g++ aren't in FSF's gcc yet, I am planning
- on getting this work into the tree some time...
- 
  > and the command
  
  >       runtest --tool g++ --srcdir ./testsuite
--- 1,6 ----
! Here are some random last minute notes by this file's original maintainer,
! Mike Stump <mrs@cygnus.com>:
  
  > and the command
  
  >       runtest --tool g++ --srcdir ./testsuite
*************** on getting this work into the tree some 
*** 16,44 ****
  site.exp updated to have right CXXFLAGS and CXX, but not at the FSF yet.
  
  
-   This is a collection of tests for g++, the FSF's C++ compiler. The
- driver that runs the testsuite is called DejaGnu.  If you do not have
- DejaGnu yet, you will need to get it ( ftp://ftp.cygnus.com/pub/dejagnu ).
- You will need a snapshot of DejaGnu of 02/21/97 or later.  The tests
- were developed and/or collected by Cygnus Support. These tests are
- included "as is". If any of the tests fail, don't report a bug.  Bug
- reports for DejaGnu can go to bug-dejagnu@prep.ai.mit.edu.  Discussion
- and comments about this testsuite can be sent to me, at the address
- below.
- 
-   Since these are part of the development source tree at Cygnus, this
- tar file is supposed to be untarred so that the testsuite directory is
- in the gcc source tree. Most of the time DejaGnu is run by hand using
- "runtest", but as we believe in solid testing, we've integrated it
- into the Makefile as a target. We normally use the testsuite by doing
- a "make all; make check-g++" at the top level and building and testing
- our whole tree.  If you have both the gcc testsuite and the g++
- testsuite, you can check both with the "make check" command.
- 
-   DejaGnu can be run either installed, or uninstalled. Usually it gets
- installed so that anyone can just run the tests without having to
- configure or build DejaGnu.
- 
    To run the tests, first change to the gcc-2.7.1 directory, then type:
  
  	runtest --tool g++ --srcdir ./testsuite
--- 13,18 ----
*************** runtest         - Is the name used to in
*** 57,64 ****
                    containing the sources of the testsuite. This is
                    ./testsuite if you are in the directory that has the
                    testsuite directory.
- 
- 
- 					Mike Stump
- 					mrs@cygnus.com
- 					FSF C++ developer
--- 31,33 ----

Re: Testsuite (was: New features of old-dejagnu.exp)

[N8TM]

In a message dated 11/5/98 9:03:50 AM Pacific Standard Time,
pfeifer@dbai.tuwien.ac.at writes:

> >> IMHO, testcases for extensions should be labeled as such by a comment
>  >> somewhere. Not that I'm against this.
>  > They're all in the g++.ext directory.  ext stands for extensions :-)
There are at least 4 g77 test cases which rely on extensions to Fortran.
-pedantic does produce appropriate remarks, but I'd prefer to see them noted
either in comments in the source or by the extension directory scheme.  

Re: Testsuite (was: New features of old-dejagnu.exp)

[Alexandre Oliva]

On Nov  4, 1998, Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at> wrote:

> On 29 Oct 1998, Alexandre Oliva wrote:

>> They're all in the g++.ext directory.  ext stands for extensions :-)

> ...and I checked and failed to find the above documented anywhere.

It's not documented, AFAIK.  I just figured it out by looking at the
testcases within that directory, they're all testing extensions, and
all containg `// Special g++ Options:' so that the default `-ansi
-pedantic-errors' is not used.

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:oliva@gnu.org mailto:aoliva@acm.org
http://www.dcc.unicamp.br/~oliva
Universidade Estadual de Campinas, SP, Brasil

Re: Testsuite (was: New features of old-dejagnu.exp)

[Mike Stump]

> Date: Wed, 28 Oct 1998 15:59:35 +0100 (MET)
> From: Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at>

> One general note, though: Where is stuff like that documented?

You have read the dejagnu manual, right?  Some is there.

In the testsuite.  :-)/2

It was meant to be so simple as to be obvious.  And for the other
stuff, asking and having the experts say sorry, can't do that...  :-(
or explain how to do it.

>  o the structure of the suite
>    (Where to add test cases? Naming conventions?)

You can add C++ testcases anywhere under g++.old-deja.  Convension,
yes, what ever you think works best on the type of tests your putting
in.  My convention was to use net_n_.C, where n is previous n + 1.
That was fine until I figured out that was a stupid idea, then I
copied Jason (or was it Brendan), and went with short ids that
identify areas of the compiler.  eh for exception handling, rtti for
run time tpe information, conv for conversions and so on.  That is
still the one I use today.

For simplicity, I just put all my testcases in g++.mike.  For others,
g++.{other,pt,eh}.

>  o the features of the testsuite
>    (Part of which you have described in your mail -- thanks!)

:-) I use grep and memory.  If someone wants to doc it up, I'd be
happy to describe stuff.

Re: Testsuite (was: New features of old-dejagnu.exp)

[Gerald Pfeifer]

On 29 Oct 1998, Alexandre Oliva wrote:
>> [ Testsuite documentation]
>> One general note, though: Where is stuff like that documented?
> In the case of the C++ testsuite, in old-dejagnu.exp.
>> Perhaps your mail can serve as a starting point?
> Go ahead! :-)

Unfortunately I have a definite lack of knowledge in this area (plus that
probably is enough work for now), but I'd really hope that someone -- not
neccessarily you -- has both time and energy to improve the testsuite
documentation.

For example, in some later mail you wrote...

>> IMHO, testcases for extensions should be labeled as such by a comment
>> somewhere. Not that I'm against this.
> They're all in the g++.ext directory.  ext stands for extensions :-)

...and I checked and failed to find the above documented anywhere.

Gerald
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

Re: Testsuite (was: New features of old-dejagnu.exp)

[Alexandre Oliva]

Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at> writes:

> On 27 Oct 1998, Alexandre Oliva wrote:

>> the new features I have introduced in old-dejagnu.exp, used for the C++
>> testsuite

> One general note, though: Where is stuff like that documented?

In the case of the C++ testsuite, in old-dejagnu.exp.

> Perhaps your mail can serve as a starting point?

Go ahead! :-)

-- 
Alexandre Oliva
mailto:oliva@dcc.unicamp.br mailto:oliva@gnu.org mailto:aoliva@acm.org
http://www.dcc.unicamp.br/~oliva
Universidade Estadual de Campinas, SP, Brasil

Testsuite (was: New features of old-dejagnu.exp)

[Gerald Pfeifer]

On 27 Oct 1998, Alexandre Oliva wrote:
> Mike Stump reminded me that I should have made more alard :-) about
> the new features I have introduced in old-dejagnu.exp, used for the C++
> testsuite.  The new features allow:

Great!


One general note, though: Where is stuff like that documented?

As far as I can see (and I also asked for documentation a couple of
months ago) there is practically no documentation at all for the
testsuite, especially about

 o the structure of the suite
   (Where to add test cases? Naming conventions?)

 o the features of the testsuite
   (Part of which you have described in your mail -- thanks!)

 o how to extend the testsuite (with additional features)


Personally I think that it would be highly desirable to start some
documentation, so that at least future changes can be described at
a single place.

Perhaps your mail can serve as a starting point?

Gerald
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/