Testsuite custom environment

Testsuite custom environment

[Stephen Biggs]

I am trying to run 'make check-gcc' and wish to define my own set of
TORTURE_OPTIONS (specifically to do only "-O0").  I have tried different
things and nothing seems to work.  E.g.:
$ export TORTURE_OPTIONS="{ { -O0 } }" ; make check-gcc
$ make TORTURE_OPTIONS="{ { -O0 } }" check-gcc

The environment variable set is ignored and the full list of options is
applied. What am I doing wrong?  Is this a problem with how I specify it
to the shell?  This forces me to edit gcc/testsuite/lib/c-torture.exp
and put my own options in instead of what is there for TORTURE_OPTIONS.

Also, how do I specify
set_board_info gcc,no_label_values 1;
on the command line?  At the moment, I have a changed
gcc/testsuite/config/default.exp where I add this line to the end, and
this is not right.

Thanks for any advice.

Re: Testsuite custom environment

[Richard Sandiford]

Stephen Biggs <xyzzy@hotpop.com> writes:
> I am trying to run 'make check-gcc' and wish to define my own set of
> TORTURE_OPTIONS (specifically to do only "-O0").  I have tried different
> things and nothing seems to work.  E.g.:
> $ export TORTURE_OPTIONS="{ { -O0 } }" ; make check-gcc
> $ make TORTURE_OPTIONS="{ { -O0 } }" check-gcc
> 
> The environment variable set is ignored and the full list of options is
> applied. What am I doing wrong?  Is this a problem with how I specify it
> to the shell?  This forces me to edit gcc/testsuite/lib/c-torture.exp
> and put my own options in instead of what is there for TORTURE_OPTIONS.

I think you want something like:

    make check-gcc RUNTESTFLAGS="TORTURE_OPTIONS=-O0"

Richard

Re: Testsuite custom environment

[Janis Johnson]

On Tue, May 20, 2003 at 06:47:57PM +0300, Stephen Biggs wrote:
> I am trying to run 'make check-gcc' and wish to define my own set of
> TORTURE_OPTIONS (specifically to do only "-O0").  I have tried different
> things and nothing seems to work.  E.g.:
> $ export TORTURE_OPTIONS="{ { -O0 } }" ; make check-gcc
> $ make TORTURE_OPTIONS="{ { -O0 } }" check-gcc
> 
> The environment variable set is ignored and the full list of options is
> applied. What am I doing wrong?  Is this a problem with how I specify it
> to the shell?  This forces me to edit gcc/testsuite/lib/c-torture.exp
> and put my own options in instead of what is there for TORTURE_OPTIONS.

I've tried lots of different ways to get a variable passed through to
the testing tools and haven't yet figured out how to do it.  For the
binary compatibility tests I gave up and had the Makefile write
particular variables to site.exp; ugly, but it did what I needed.  If
there's a better way to do this, I'd like to know about it, too, and
then we can fix the documentation.
 
> Also, how do I specify
> set_board_info gcc,no_label_values 1;
> on the command line?  At the moment, I have a changed
> gcc/testsuite/config/default.exp where I add this line to the end, and
> this is not right.

Sorry, I don't know anything about this one.

Janis

Re: Testsuite custom environment

[David Edelsohn]

>>>>> Janis Johnson writes:

Janis> I've tried lots of different ways to get a variable passed through to
Janis> the testing tools and haven't yet figured out how to do it.  For the
Janis> binary compatibility tests I gave up and had the Makefile write
Janis> particular variables to site.exp; ugly, but it did what I needed.  If
Janis> there's a better way to do this, I'd like to know about it, too, and
Janis> then we can fix the documentation.
 
Are

	--tool_opts

and 
	RUNTESTFLAGS=""

useful?

	--tool_opts can specify additional options that always are applied
to the commandline when invoking the tool, but not overriding existing
flags.

	RUNTESTFLAGS can specify those options in an environment variable,
e.g., when running from "make check".

David

Re: Testsuite custom environment

[Janis Johnson]

On Tue, May 20, 2003 at 05:15:56PM -0400, David Edelsohn wrote:
> >>>>> Janis Johnson writes:
> 
> Janis> I've tried lots of different ways to get a variable passed through to
> Janis> the testing tools and haven't yet figured out how to do it.  For the
> Janis> binary compatibility tests I gave up and had the Makefile write
> Janis> particular variables to site.exp; ugly, but it did what I needed.  If
> Janis> there's a better way to do this, I'd like to know about it, too, and
> Janis> then we can fix the documentation.
>  
> Are
> 
> 	--tool_opts
> 
> and 
> 	RUNTESTFLAGS=""
> 
> useful?
> 
> 	--tool_opts can specify additional options that always are applied
> to the commandline when invoking the tool, but not overriding existing
> flags.
> 
> 	RUNTESTFLAGS can specify those options in an environment variable,
> e.g., when running from "make check".

RUNTESTFLAGS is used to pass options to runtest; the list of those
options can be seen with runtest --help.  I've used --tool-opts to
specify flags that are passed to the compiler (the "tool") for every
invocation, but haven't figured out how to set environment variables
that will be recognized by the .exp files.  I suppose it would help
to read the documentation for expect.

Janis

Re: Testsuite custom environment

[Janis Johnson]

On Tue, May 20, 2003 at 02:41:41PM -0700, Janis Johnson wrote:
>                                            I suppose it would help
> to read the documentation for expect.

Score!  I created a file ~/.expect.rc with the following:

global TORTURE_OPTIONS
set TORTURE_OPTIONS [list { -O0 }]

and the torture tests are run with only -O0.

Janis

Re: Testsuite custom environment

[Mike Stump]

On Tuesday, May 20, 2003, at 02:59 PM, Janis Johnson wrote:
> On Tue, May 20, 2003 at 02:41:41PM -0700, Janis Johnson wrote:
>>                                            I suppose it would help
>> to read the documentation for expect.
>
> Score!  I created a file ~/.expect.rc with the following:
>
> global TORTURE_OPTIONS
> set TORTURE_OPTIONS [list { -O0 }]
>
> and the torture tests are run with only -O0.

There is also a ~/.dejagnurc file as well.  In addition, ~/site.exp can 
be used in complex ways as I recall.

Re: Testsuite custom environment

[Stephen Biggs]

Thanks everybody, for all the help.

The .expect.rc took care of my TORTURE_OPTIONS problem, but neither
.dejagnurc nor site.exp accept 'set_board_info' as a valid command.


On Wed, 2003-05-21 at 02:21, Mike Stump wrote:
> On Tuesday, May 20, 2003, at 02:59 PM, Janis Johnson wrote:
> > On Tue, May 20, 2003 at 02:41:41PM -0700, Janis Johnson wrote:
> >>                                            I suppose it would help
> >> to read the documentation for expect.
> >
> > Score!  I created a file ~/.expect.rc with the following:
> >
> > global TORTURE_OPTIONS
> > set TORTURE_OPTIONS [list { -O0 }]
> >
> > and the torture tests are run with only -O0.
> 
> There is also a ~/.dejagnurc file as well.  In addition, ~/site.exp can 
> be used in complex ways as I recall.
> 
>