* [testsuite] dg-final object-size: fail if file does not exist @ 2011-06-16 17:07 Janis Johnson 2011-06-16 17:26 ` Joseph S. Myers 2011-06-16 20:38 ` Mike Stump 0 siblings, 2 replies; 7+ messages in thread From: Janis Johnson @ 2011-06-16 17:07 UTC (permalink / raw) To: gcc-patches [-- Attachment #1: Type: text/plain, Size: 177 bytes --] Currently the dg-final check "object-size" results in ERROR if the assemble failed and the object file does not exist. This patch fails the test instead. OK for trunk? Janis [-- Attachment #2: gcc-20110616-1 --] [-- Type: text/plain, Size: 669 bytes --] 2011-06-16 Janis Johnson <janisjo@codesourcery.com> * lib/scanasm.exp (object-size): Fail if object file does not exist. Index: lib/scanasm.exp =================================================================== --- lib/scanasm.exp (revision 175083) +++ lib/scanasm.exp (working copy) @@ -351,6 +351,10 @@ upvar 2 name testcase set testcase [lindex $testcase 0] set output_file "[file rootname [file tail $testcase]].o" + if ![file_on_host exists $output_file] { + fail "$testcase $output_file does not exist" + return + } set output [remote_exec host "$size" "$output_file"] set status [lindex $output 0] if { $status != 0 } { ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-16 17:07 [testsuite] dg-final object-size: fail if file does not exist Janis Johnson @ 2011-06-16 17:26 ` Joseph S. Myers 2011-06-16 23:12 ` Janis Johnson 2011-06-16 20:38 ` Mike Stump 1 sibling, 1 reply; 7+ messages in thread From: Joseph S. Myers @ 2011-06-16 17:26 UTC (permalink / raw) To: Janis Johnson; +Cc: gcc-patches On Thu, 16 Jun 2011, Janis Johnson wrote: > Currently the dg-final check "object-size" results in ERROR if the > assemble failed and the object file does not exist. This patch fails > the test instead. OK for trunk? The set of testcase names - the things after "PASS: " or "FAIL: " or other statuses - should not depend on the results if comparison is to work well, so + fail "$testcase $output_file does not exist" is a bad idea unless there is a corresponding pass "$testcase $output_file does not exist" (obvious nonsense) as the alternative. Instead you should: * Make sure the compilation of the test produced its own PASS or FAIL line. * If that failed, report the subsequent test as UNRESOLVED. unresolved "$testcase object-size $what $cmp $with" -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-16 17:26 ` Joseph S. Myers @ 2011-06-16 23:12 ` Janis Johnson 2011-06-16 23:55 ` Joseph S. Myers 0 siblings, 1 reply; 7+ messages in thread From: Janis Johnson @ 2011-06-16 23:12 UTC (permalink / raw) To: Joseph S. Myers; +Cc: gcc-patches, Mike Stump On 06/16/2011 10:08 AM, Joseph S. Myers wrote: > On Thu, 16 Jun 2011, Janis Johnson wrote: > >> Currently the dg-final check "object-size" results in ERROR if the >> assemble failed and the object file does not exist. This patch fails >> the test instead. OK for trunk? > > The set of testcase names - the things after "PASS: " or "FAIL: " or other > statuses - should not depend on the results if comparison is to work well, > so > > + fail "$testcase $output_file does not exist" > > is a bad idea unless there is a corresponding > > pass "$testcase $output_file does not exist" > > (obvious nonsense) as the alternative. Instead you should: > > * Make sure the compilation of the test produced its own PASS or FAIL > line. > > * If that failed, report the subsequent test as UNRESOLVED. > > unresolved "$testcase object-size $what $cmp $with" > This issue also affects other procedures used from dg-final, including the scan-assembler and scan-dump variants. The scan-dump routines append "dump file does not exist" to the usual FAIL messages and the scan-assembler routines report ERROR. I'll fix those after I understand the correct fix for object-size. These routines don't have access to the pass/fail status of the compilation, and the compilation step doesn't know about dg-final checks. Pass/fail messages for "object-size text <= 32" are: PASS: gcc.target/arm/ivopts-6.c object-size text <= 32 FAIL: gcc.target/arm/ivopts-6.c object-size text <= 32 If the file doesn't exist the message could be: UNRESOLVED: gcc.target/arm/ivopts-6.c object-size text <= 32 Currently there are several possible causes for failure in object-size. Some are errors in the test itself, like the wrong number of arguments, but some others could be UNRESOLVED instead of ERROR, such as the "size" command failing or producing unexpected output. Can the UNRESOLVED line include additional information about the reason for the failure, or should the reason just be in a message in the log file? Janis ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-16 23:12 ` Janis Johnson @ 2011-06-16 23:55 ` Joseph S. Myers 2011-06-17 1:09 ` Janis Johnson 0 siblings, 1 reply; 7+ messages in thread From: Joseph S. Myers @ 2011-06-16 23:55 UTC (permalink / raw) To: Janis Johnson; +Cc: gcc-patches, Mike Stump On Thu, 16 Jun 2011, Janis Johnson wrote: > Currently there are several possible causes for failure in object-size. > Some are errors in the test itself, like the wrong number of arguments, > but some others could be UNRESOLVED instead of ERROR, such as the "size" > command failing or producing unexpected output. Can the UNRESOLVED line > include additional information about the reason for the failure, or > should the reason just be in a message in the log file? My view is that reasons should be separately in the log file; there should be a fixed set of test names, each of which may be PASS, FAIL, UNRESOLVED etc. in a particular test run. (I know the code reporting ICEs in test names doesn't conform to this; properly that should have separate "test for internal compiler error" test names rather than modifying the name of a failing test if it has an ICE.) -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-16 23:55 ` Joseph S. Myers @ 2011-06-17 1:09 ` Janis Johnson 2011-06-18 14:50 ` Mike Stump 0 siblings, 1 reply; 7+ messages in thread From: Janis Johnson @ 2011-06-17 1:09 UTC (permalink / raw) To: Joseph S. Myers; +Cc: gcc-patches, Mike Stump [-- Attachment #1: Type: text/plain, Size: 1396 bytes --] On 06/16/2011 04:18 PM, Joseph S. Myers wrote: > On Thu, 16 Jun 2011, Janis Johnson wrote: > >> Currently there are several possible causes for failure in object-size. >> Some are errors in the test itself, like the wrong number of arguments, >> but some others could be UNRESOLVED instead of ERROR, such as the "size" >> command failing or producing unexpected output. Can the UNRESOLVED line >> include additional information about the reason for the failure, or >> should the reason just be in a message in the log file? > > My view is that reasons should be separately in the log file; there should > be a fixed set of test names, each of which may be PASS, FAIL, UNRESOLVED > etc. in a particular test run. (I know the code reporting ICEs in test > names doesn't conform to this; properly that should have separate "test > for internal compiler error" test names rather than modifying the name of > a failing test if it has an ICE.) This new fix to object-size, used within dg-final, continues to report static test errors as ERROR, but reports problems detected at runtime as UNRESOLVED, writing the reason to the log file. I'm confident that a missing object file should make the test UNRESOLVED, not as sure about problems with the size command on a particular target. With this change the message from object-size is the same for pass/fail/ unresolved. OK for trunk? Janis [-- Attachment #2: gcc-20110616-2 --] [-- Type: text/plain, Size: 3041 bytes --] 2011-06-16 Janis Johnson <janisjo@codesourcery.com> * lib/scanasm.exp (object-size): Move argument processing earlier to report errors before verifying that the file exists. Report problems detected at runtime as unresolved instead of error and report their reasons to the log file. Index: lib/scanasm.exp =================================================================== --- lib/scanasm.exp (revision 175083) +++ lib/scanasm.exp (working copy) @@ -350,11 +350,35 @@ upvar 2 name testcase set testcase [lindex $testcase 0] + + set what [lindex $args 0] + set where [lsearch { text data bss total } $what] + if { $where == -1 } { + error "object-size: illegal argument: $what" + return + } + set cmp [lindex $args 1] + if { [lsearch { < > <= >= == != } $cmp] == -1 } { + error "object-size: illegal argument: $cmp" + return + } + set with [lindex $args 2] + if ![string is integer $with ] { + error "object-size: illegal argument: $with" + return + } + set output_file "[file rootname [file tail $testcase]].o" + if ![file_on_host exists $output_file] { + verbose -log "$testcase: $output_file does not exist" + unresolved "$testcase object-size $what $cmp $with" + return + } set output [remote_exec host "$size" "$output_file"] set status [lindex $output 0] if { $status != 0 } { - error "object-size: $size failed" + verbose -log "$testcase object-size: $size failed" + unresolved "$testcase object-size $what $cmp $with" return } @@ -363,37 +387,21 @@ set line0 [lindex $lines 0] if ![regexp {^\s*text\s+data\s+bss\s+dec\s+hex\s+filename\s*$} $line0] { - error "object-size: $size did not produce expected first line: $line0" + verbose -log "$testcase object-size: $size did not produce expected first line: $line0" + unresolved "$testcase object-size $what $cmp $with" return } set line1 [lindex $lines 1] if ![regexp {^\s*\d+\s+\d+\s+\d+\s+\d+\s+[\da-fA-F]+\s+} $line1] { - error "object-size: $size did not produce expected second line: $line1" + verbose -log "$testcase object-size: $size did not produce expected second line: $line1" + unresolved "$testcase object-size $what $cmp $with" return } - set what [lindex $args 0] - set where [lsearch { text data bss total } $what] - if { $where == -1 } { - error "object-size: illegal argument: $what" - return - } set actual [lindex $line1 $where] verbose -log "$what size is $actual" - set cmp [lindex $args 1] - if { [lsearch { < > <= >= == != } $cmp] == -1 } { - error "object-size: illegal argument: $cmp" - return - } - - set with [lindex $args 2] - if ![string is integer $with ] { - error "object-size: illegal argument: $with" - return - } - if [expr $actual $cmp $with] { pass "$testcase object-size $what $cmp $with" } else { ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-17 1:09 ` Janis Johnson @ 2011-06-18 14:50 ` Mike Stump 0 siblings, 0 replies; 7+ messages in thread From: Mike Stump @ 2011-06-18 14:50 UTC (permalink / raw) To: Janis Johnson; +Cc: Joseph S. Myers, gcc-patches On Jun 16, 2011, at 5:26 PM, Janis Johnson wrote: > This new fix to object-size, used within dg-final, continues to report > static test errors as ERROR, but reports problems detected at runtime > as UNRESOLVED, writing the reason to the log file. I'm confident that > a missing object file should make the test UNRESOLVED, not as sure about > problems with the size command on a particular target. > > With this change the message from object-size is the same for pass/fail/ > unresolved. > > OK for trunk? Ok. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [testsuite] dg-final object-size: fail if file does not exist 2011-06-16 17:07 [testsuite] dg-final object-size: fail if file does not exist Janis Johnson 2011-06-16 17:26 ` Joseph S. Myers @ 2011-06-16 20:38 ` Mike Stump 1 sibling, 0 replies; 7+ messages in thread From: Mike Stump @ 2011-06-16 20:38 UTC (permalink / raw) To: Janis Johnson; +Cc: gcc-patches On Jun 16, 2011, at 9:31 AM, Janis Johnson wrote: > Currently the dg-final check "object-size" results in ERROR if the > assemble failed and the object file does not exist. This patch fails > the test instead. If you can arrange for call fail on only those things that later would have called pass or fail on, that would be better. The idea is that we want to preserve all testcases, but just call pass or fail on each one, as appropriate. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2011-06-18 14:21 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-06-16 17:07 [testsuite] dg-final object-size: fail if file does not exist Janis Johnson 2011-06-16 17:26 ` Joseph S. Myers 2011-06-16 23:12 ` Janis Johnson 2011-06-16 23:55 ` Joseph S. Myers 2011-06-17 1:09 ` Janis Johnson 2011-06-18 14:50 ` Mike Stump 2011-06-16 20:38 ` Mike Stump
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).