Re: [testsuite] dg-final object-size: fail if file does not exist

[Janis Johnson <janisjo@codesourcery.com>]

On 06/16/2011 10:08 AM, Joseph S. Myers wrote:
> On Thu, 16 Jun 2011, Janis Johnson wrote:
> 
>> Currently the dg-final check "object-size" results in ERROR if the
>> assemble failed and the object file does not exist.  This patch fails
>> the test instead.  OK for trunk?
> 
> The set of testcase names - the things after "PASS: " or "FAIL: " or other 
> statuses - should not depend on the results if comparison is to work well, 
> so
> 
> +       fail "$testcase $output_file does not exist"
> 
> is a bad idea unless there is a corresponding
> 
> 	pass "$testcase $output_file does not exist"
> 
> (obvious nonsense) as the alternative.  Instead you should:
> 
> * Make sure the compilation of the test produced its own PASS or FAIL 
> line.
> 
> * If that failed, report the subsequent test as UNRESOLVED.
> 
> 	unresolved "$testcase object-size $what $cmp $with"
> 

This issue also affects other procedures used from dg-final, including
the scan-assembler and scan-dump variants.  The scan-dump routines
append "dump file does not exist" to the usual FAIL messages and the
scan-assembler routines report ERROR.  I'll fix those after I
understand the correct fix for object-size.  These routines don't have
access to the pass/fail status of the compilation, and the compilation
step doesn't know about dg-final checks.

Pass/fail messages for "object-size text <=  32" are:

PASS: gcc.target/arm/ivopts-6.c object-size text <= 32
FAIL: gcc.target/arm/ivopts-6.c object-size text <= 32

If the file doesn't exist the message could be:

UNRESOLVED: gcc.target/arm/ivopts-6.c object-size text <= 32

Currently there are several possible causes for failure in object-size.
Some are errors in the test itself, like the wrong number of arguments,
but some others could be UNRESOLVED instead of ERROR, such as the "size"
command failing or producing unexpected output.  Can the UNRESOLVED line
include additional information about the reason for the failure, or
should the reason just be in a message in the log file?

Janis