Re: Running the compat testsuite in "non-mirror" mode

[Janis Johnson <janis187@us.ibm.com>]

On Fri, Nov 19, 2004 at 09:17:04AM +0100, Eric Botcazou wrote:
> > I'm sorry, I still don't understand what you mean my mirror and non-mirror
> > modes.
> 
> OK, sorry for being obtuse.
> 
> The compat testsuite is currently automatically run only once, in what I call 
> mirror mode, that is the newly built compiler is tested against (an identical 
> copy of) itself.  While this is useful for catching problems in the 
> argument/return value handling machinery of the back-end, this is only 
> moderately useful for compatibility purposes.  Of course this is 
> customizable, but only externally (i.e. manually or with an external Makefile 
> for example).
> 
> What I would like to have is the possibility to automatically run (i.e. with a 
> bare make -k check-gcc) the compat testsuite twice, once in the mirror mode 
> described above and once in a non-mirror mode.  The latter would mean that 
> the newly built compiler is tested against a slight variation of itself, i.e. 
> typically with a non-default option, that could have an impact on the calling 
> conventions, turned on.  So it would be possible to specify in the compat 
> testsuite harness (e.g. a platform-specific driver) that it should be 
> automatically run twice on a particular platform, in mirror mode and with an 
> hardcoded pair (-mfoo/-mno-foo) of options.

I think I understand now.  For anyone testing a particular target, the
compat tests would be run first using the compiler under test to compile
both parts of each test using default options, and again using different
sets of options for the two parts of each test to make sure those
options don't affect binary compatibility.  Is that right?

This is probably easy to set up in the compat.exp file for each compat
testsuite; I'll take a look.

Janis