[Bug ada/5909] Ada has no test suite.

[Bug ada/5909] Ada has no test suite.

[dhazeghi at yahoo dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From dhazeghi at yahoo dot com  2003-07-06 02:19 -------
Any news about the Ada testsuite? Is ACATS likely to make the gcc testsuite before 3.4? Thanks.

[Bug ada/5909] Ada has no test suite.

[guerby at acm dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From guerby at acm dot org  2003-07-06 06:24 -------
ACATS won't make it into 3.4, you can still
run it by downloading it from
<http://perso.wanadoo.fr/guerby/>
and following the instructions, current
version as of 20030706 is 0.7.

[Bug ada/5909] Ada has no test suite.

[pinskia at physics dot uc dot edu]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909


pinskia at physics dot uc dot edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|0000-00-00 00:00:00         |2003-07-19 17:16:38
               date|                            |


------- Additional Comments From pinskia at physics dot uc dot edu  2003-07-19 17:16 -------
Ada really should have a test suite so that changes to the backend get exersied on Ada.

[Bug ada/5909] Ada has no test suite.

[charlet at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From charlet at gcc dot gnu dot org  2003-10-22 13:29 -------
I've started from Laurent's set up to integrate the ACATS test suite as
part of 'make check'.

I have a complete set up which is working. Since Laurent and I know nothing
about dejagnu, we're simply using sh to run the test suite, which is
working just fine.

The integration follows other gcc requirements: using make check will
run the acats test suite if Ada is enabled, and make check-ada will
work on a non installed version of the compiler.

If this is acceptable, the ACATS test suite along with the makefile
changes could be incorporated as soon as practical (e.g. probably
after the 3.4 branch). I am using make check-ada (part of make check)
before any of my check ins.

Arno

[Bug ada/5909] Ada has no test suite.

[jsm at polyomino dot org dot uk]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From jsm at polyomino dot org dot uk  2003-10-22 14:05 -------
Subject: Re:  Ada has no test suite.

On Wed, 22 Oct 2003, charlet at gcc dot gnu dot org wrote:

> I have a complete set up which is working. Since Laurent and I know nothing
> about dejagnu, we're simply using sh to run the test suite, which is
> working just fine.

But is it working for testing cross-compilers where programs for the
target need to be run on a simulator, or on an embedded board, or ...?  
And does it generate .sum files in the Dejagnu format for the use of the
automated regression testers (this point is critical to get much of the
benefit of the testing, a sh script can always be committed first then
converted to a Dejagnu harness later to take advantage of all the support
Dejagnu has for testing cross-compilers)?

Does your test setup support additionally "ordinary" testsuite directories
for gradually accumulating tests in the style of
gcc.c-torture/{compile,execute} (test compilation / execution works with
multiple optimization options) or gcc.dg (test that error/warning messages
are as expected)?  There are many such tests that Kenner has posted to the 
lists over the years (for back-end bugs shown up by Ada testcases) that 
should be gathered into the testsuite.

[Bug ada/5909] Ada has no test suite.

[charlet at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From charlet at gcc dot gnu dot org  2003-10-22 16:33 -------
<<
But is it working for testing cross-compilers where programs for the
target need to be run on a simulator, or on an embedded board, or ...?  
>>

Yes, it has support for cross testing, although I haven't used it
myself. Maybe Laurent can comment on this part.

Basically there's a customization section where you can specify how
to load/run an executable.

<<
And does it generate .sum files in the Dejagnu format for the use of the
automated regression testers (this point is critical to get much of the
benefit of the testing, a sh script can always be committed first then
converted to a Dejagnu harness later to take advantage of all the support
Dejagnu has for testing cross-compilers)?
>>

No, but if there's a clear documentation about the format of .sum
files, I'd guess this should not be very hard to do.

<<
Does your test setup support additionally "ordinary" testsuite directories
for gradually accumulating tests in the style of
gcc.c-torture/{compile,execute} (test compilation / execution works with
multiple optimization options) or gcc.dg (test that error/warning messages
are as expected)?
>>

The current set up only supports executable tests, but it shouldn't
be too hard to enhance it if needed.

Arno

[Bug ada/5909] Ada has no test suite.

[laurent at guerby dot net]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From laurent at guerby dot net  2003-10-22 17:40 -------
Subject: Re:  Ada has no test suite.


> On Wed, 22 Oct 2003, charlet at gcc dot gnu dot org wrote:
> 
> > I have a complete set up which is working. Since Laurent and I know nothing
> > about dejagnu, we're simply using sh to run the test suite, which is
> > working just fine.

If someone is willing to contribute a dejagnu Ada harness, we're
ready to use it, otherwise we'll do with simple scripts.

> But is it working for testing cross-compilers where programs for the
> target need to be run on a simulator, or on an embedded board, or ...? 

Joel Sherrill did run my original script with a cross-compiler
on a simulator and IIRC he had trouble running dejagnu (much
more complicated to set up).

>  
> And does it generate .sum files in the Dejagnu format for the use of the
> automated regression testers (this point is critical to get much of the
> benefit of the testing, a sh script can always be committed first then
> converted to a Dejagnu harness later to take advantage of all the support
> Dejagnu has for testing cross-compilers)?

If the format and where to put the ".sum" files is documented, that
should be easy to add.

> Does your test setup support additionally "ordinary" testsuite directories
> for gradually accumulating tests in the style of
> gcc.c-torture/{compile,execute} (test compilation / execution works with
> multiple optimization options) or gcc.dg (test that error/warning messages
> are as expected)?  There are many such tests that Kenner has posted to the 
> lists over the years (for back-end bugs shown up by Ada testcases) that 
> should be gathered into the testsuite.

At -O0 the current ACATS run is done in 45 minutes on a P3 1GHz. We can
easily add a loop over compile options, but at some point
there will be some thinking about what we want to require for patch
and what is optional in the GCC testsuites.

ACT also has a small set of tests which are not proprietary code
and could be included, adding a loop over them in the
ACATS script is easy. I'll go back in the list archives
and add them over the time.

Laurent

[Bug ada/5909] Ada has no test suite.

[jsm at polyomino dot org dot uk]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From jsm at polyomino dot org dot uk  2003-10-22 18:55 -------
Subject: Re:  Ada has no test suite.

On Wed, 22 Oct 2003, laurent at guerby dot net wrote:

> Joel Sherrill did run my original script with a cross-compiler
> on a simulator and IIRC he had trouble running dejagnu (much
> more complicated to set up).

For dejagnu we have instructions (simtest-howto.html) that enable someone
with a major patch to test (so needing to test it on three platforms) to
run simulator tests without needing to understand setting up Dejagnu for
cross targets.  Does this Ada harness need another column for that table,
or something more complicated?

> At -O0 the current ACATS run is done in 45 minutes on a P3 1GHz. We can
> easily add a loop over compile options, but at some point
> there will be some thinking about what we want to require for patch
> and what is optional in the GCC testsuites.

I don't think ACATS needs running over multiple options - but where Ada
code has shown up a back end bug (as in many of the testcases Kenner has
sent to the lists), running over multiple options may be more useful.

[Bug ada/5909] Ada has no test suite.

[zack at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From zack at gcc dot gnu dot org  2003-10-22 19:01 -------
Can't you just clone gcc-dg.exp?  I don't see why this is hard.

[Bug ada/5909] Ada has no test suite.

[laurent at guerby dot net]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From laurent at guerby dot net  2003-10-22 20:58 -------
Subject: Re:  Ada has no test suite.

On Wed, 2003-10-22 at 21:01, zack at gcc dot gnu dot org wrote:
> Can't you just clone gcc-dg.exp?  I don't see why this is hard.

gcc-dg.exp is written in a language I don't know, I don't
know how it is called, what are the available tools, how it infers its
list of test, how it understands results, how to precompile common
files or handle special cases.

The sh script is two-third of the size of gcc-dg.exp and
does what is needed. It is understandable by just
about anyone able to write a sh script.
It's a one line change to test post install,
and another line change to test in a simulator,
a few lines to loop over flags if needed.

Sorry to look stupid, gcc-dg.exp is *hard* for me.

As I said, if someone for whom it is "easy"
to make a dg Ada framework contributes it,
I'll use it (I assume there will be
minimal comments on how to add a test
or something to be precompiler).

Sincerely,

Laurent

[Bug ada/5909] Ada has no test suite.

[zack at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From zack at gcc dot gnu dot org  2003-10-22 21:11 -------
Sorry, I didn't realize you were unfamiliar with expect in general.

I suggest you check in an initial cut with the shell script you've got. Some
kind of test suite is better than no test suite at all.  And someone else will
probably then be motivated to convert it to dejagnu.

[Bug ada/5909] Ada has no test suite.

[charlet at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From charlet at gcc dot gnu dot org  2003-10-23 08:29 -------
Thanks for the last suggestion, I believe this will help
moving forward, rather than trying to get a "perfect" solution and get
stuck.

I'm ready to put the current version, although it may be better to
wait for the 3.4 branch I believe.

I'd rather not push too many changes after stage 3, or that will drive
Mark crazy :-) Not mentioning the fact that being a release manager
myself, I certainly agree that we should avoid any possible disruption
during stage 3 and concentrate on stabilizing.

Arno

[Bug ada/5909] Ada has no test suite.

[zack at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From zack at gcc dot gnu dot org  2003-10-23 08:46 -------
I'm not Mark, but I think adding a test suite for Ada should cause minimal
disruption -- certainly far lower risk than the merge you just did.

I also think it would be good to have the test suite included on the 3.4 branch
if at all possible; that makes it much easier to check whether the Ada front end
on the branch works at all, which is very useful from a release engineering
standpoint.

[Bug ada/5909] Ada has no test suite.

[charlet at act-europe dot fr]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From charlet at act-europe dot fr  2003-10-23 08:57 -------
Subject: Re:  Ada has no test suite.

> I'm not Mark, but I think adding a test suite for Ada should cause minimal
> disruption -- certainly far lower risk than the merge you just did.

That's fine with me.

Assuming I have the approval of Mark, I'll add the Ada test suite.

> I also think it would be good to have the test suite included on the 3.4
> branch if at all possible; that makes it much easier to check whether the
> Ada front end on the branch works at all, which is very useful from a
> release engineering standpoint.

That's right.

Arno

[Bug ada/5909] Ada has no test suite.

[mark at codesourcery dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From mark at codesourcery dot com  2003-10-24 15:39 -------
Subject: Re:  Ada has no test suite.

> > I'm not Mark, but I think adding a test suite for Ada should cause
minimal
> > disruption -- certainly far lower risk than the merge you just did.
>
> That's fine with me.
>
> Assuming I have the approval of Mark, I'll add the Ada test suite.

It's OK with me.

Thanks,

-- Mark

[Bug ada/5909] Ada has no test suite.

[cvs-commit at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From cvs-commit at gcc dot gnu dot org  2003-10-27 11:33 -------
Subject: Bug 5909

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	charlet@gcc.gnu.org	2003-10-27 11:33:00

Modified files:
	gcc/testsuite  : ChangeLog 
Added files:
	gcc/testsuite  : README.ada 

Log message:
	PR ada/5909:
	* README.ada, ada/acats: Import ACATS 2.5 for GCC Ada test suite.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/README.ada.diff?cvsroot=gcc&r1=NONE&r2=1.1
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&r1=1.3147&r2=1.3148

[Bug ada/5909] Ada has no test suite.

[cvs-commit at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From cvs-commit at gcc dot gnu dot org  2003-10-27 14:27 -------
Subject: Bug 5909

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	charlet@gcc.gnu.org	2003-10-27 14:27:17

Modified files:
	gcc/ada        : ChangeLog exp_ch3.adb exp_ch5.adb sem_prag.adb 
	                 make.adb prj-env.adb prj-env.ads g-dirope.adb 
	                 Makefile.generic sem_ch12.adb sem_ch8.adb 
	                 sem_ch8.ads Make-lang.in 

Log message:
	* Makefile.generic: Add missing substitution on object_deps handling.
	
	PR ada/5909:
	* Make-lang.in (check-ada): Enable ACATS test suite.
	
	* exp_ch3.adb:
	(Freeze_Array_Type): We do not need an initialization routine for types
	derived from String or Wide_String. They should be treated the same
	as String and Wide_String themselves. This caused problems with the
	use of Initialize_Scalars.
	
	* exp_ch5.adb:
	(Expand_Assign_Record): Do component-wise assignment of non-byte aligned
	composites. This allows use of component clauses that are not byte
	aligned.
	
	* sem_prag.adb:
	(Analyze_Pragma, case Pack): Generate warning and ignore pack if there
	is an attempt to pack an array of atomic objects.
	
	* make.adb, prj-env.adb, prj-env.ads: Minor reformatting
	
	* g-dirope.adb:
	(Basename): Check for drive letters in a pathname only on DOS based OS.
	
	* make.adb:
	(Gnatmake): When unable to change dir to the object dir, display the
	content of the parent dir of the obj dir, to try to understand why this
	happens.
	
	* Make-lang.in: Makefile automatically updated
	
	* sem_ch12.adb:
	(Inline_Instance_Body): Indicate that the save/restore of use_clauses
	should not be done in Save/Restore_Scope_Stack, because it is performed
	locally.
	
	* sem_ch8.adb:
	(Save_Scope_Stack, Restore_Scope_Stack): Add parameter to indicate
	whether use clauses should be removed/restored.
	
	* sem_ch8.ads:
	(Save_Scope_Stack, Restore_Scope_Stack): Add parameter to indicate
	whether use clauses should be removed/restored.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/ChangeLog.diff?cvsroot=gcc&r1=1.363&r2=1.364
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/exp_ch3.adb.diff?cvsroot=gcc&r1=1.9&r2=1.10
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/exp_ch5.adb.diff?cvsroot=gcc&r1=1.9&r2=1.10
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/sem_prag.adb.diff?cvsroot=gcc&r1=1.10&r2=1.11
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/make.adb.diff?cvsroot=gcc&r1=1.18&r2=1.19
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/prj-env.adb.diff?cvsroot=gcc&r1=1.11&r2=1.12
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/prj-env.ads.diff?cvsroot=gcc&r1=1.9&r2=1.10
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/g-dirope.adb.diff?cvsroot=gcc&r1=1.11&r2=1.12
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/Makefile.generic.diff?cvsroot=gcc&r1=1.2&r2=1.3
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/sem_ch12.adb.diff?cvsroot=gcc&r1=1.23&r2=1.24
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/sem_ch8.adb.diff?cvsroot=gcc&r1=1.9&r2=1.10
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/sem_ch8.ads.diff?cvsroot=gcc&r1=1.7&r2=1.8
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ada/Make-lang.in.diff?cvsroot=gcc&r1=1.44&r2=1.45

[Bug ada/5909] Ada has no test suite.

[cvs-commit at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909



------- Additional Comments From cvs-commit at gcc dot gnu dot org  2003-10-27 14:51 -------
Subject: Bug 5909

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	charlet@gcc.gnu.org	2003-10-27 14:51:27

Modified files:
	gcc            : ChangeLog 
	gcc/doc        : sourcebuild.texi 

Log message:
	PR ada/5909:
	* doc/sourcebuild.texi: Document Ada test suite.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&r1=2.1573&r2=2.1574
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/doc/sourcebuild.texi.diff?cvsroot=gcc&r1=1.32&r2=1.33

[Bug ada/5909] Ada has no test suite.

[charlet at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=5909


charlet at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED
   Target Milestone|---                         |3.4


------- Additional Comments From charlet at gcc dot gnu dot org  2003-10-27 15:31 -------
Ada now has a test suite, so I am closing this PR.
Feel free to open new PR for specific improvements/suggestions, that
go beyond the general issue of having an Ada test suite.

Arno