Minor testsuite improvement

Minor testsuite improvement

[Jeffrey A Law]

This speeds up native testing of the c-torture execute tests by a few
percent for native tests.

Basically there's an optimization in the test path which greatly helps
embedded or simulated targets, but loses for native targets because the
costs of the various steps is radically different.

        * lib/c-torture.exp: Avoid the "compare executables" optimization
        when testing native.

Index: c-torture.exp
===================================================================
RCS file: /egcs/carton/cvsfiles/egcs/gcc/testsuite/lib/c-torture.exp,v
retrieving revision 1.11
diff -c -3 -p -r1.11 c-torture.exp
*** c-torture.exp	1999/06/30 01:36:43	1.11
--- c-torture.exp	1999/08/31 23:58:48
*************** proc c-torture-execute { src args } {
*** 189,196 ****
              set ignore_me [eval $torture_eval_before_execute]
  	}
  	
  	set skip 0;
! 	if [info exists oldexec] {
  	    if { [remote_file build cmp $oldexec $execname] == 0 } {
  		set skip 1;
  	    }
--- 189,210 ----
              set ignore_me [eval $torture_eval_before_execute]
  	}
  	
+ 
+ 	# Sometimes we end up creating identical executables for two
+ 	# consecutive sets of different of compiler options.
+ 	#
+ 	# In such cases we know the result of this test will be identical
+ 	# to the result of the last test.
+ 	#
+ 	# So in cases where the time to load and run/simulate the test
+ 	# is relatively high, compare the two binaries and avoid rerunning
+ 	# tests if the executables are identical.
+ 	#
+ 	# Do not do this for native testing since the cost to load/execute
+ 	# the test is fairly small and the comparison step actually slows
+ 	# the entire process down because it usually does not "hit".
  	set skip 0;
! 	if { ![isnative] && [info exists oldexec] } {
  	    if { [remote_file build cmp $oldexec $execname] == 0 } {
  		set skip 1;
  	    }

Minor testsuite improvement

[Jeffrey A Law]

This speeds up native testing of the c-torture execute tests by a few
percent for native tests.

Basically there's an optimization in the test path which greatly helps
embedded or simulated targets, but loses for native targets because the
costs of the various steps is radically different.

        * lib/c-torture.exp: Avoid the "compare executables" optimization
        when testing native.

Index: c-torture.exp
===================================================================
RCS file: /egcs/carton/cvsfiles/egcs/gcc/testsuite/lib/c-torture.exp,v
retrieving revision 1.11
diff -c -3 -p -r1.11 c-torture.exp
*** c-torture.exp	1999/06/30 01:36:43	1.11
--- c-torture.exp	1999/08/31 23:58:48
*************** proc c-torture-execute { src args } {
*** 189,196 ****
              set ignore_me [eval $torture_eval_before_execute]
  	}
  	
  	set skip 0;
! 	if [info exists oldexec] {
  	    if { [remote_file build cmp $oldexec $execname] == 0 } {
  		set skip 1;
  	    }
--- 189,210 ----
              set ignore_me [eval $torture_eval_before_execute]
  	}
  	
+ 
+ 	# Sometimes we end up creating identical executables for two
+ 	# consecutive sets of different of compiler options.
+ 	#
+ 	# In such cases we know the result of this test will be identical
+ 	# to the result of the last test.
+ 	#
+ 	# So in cases where the time to load and run/simulate the test
+ 	# is relatively high, compare the two binaries and avoid rerunning
+ 	# tests if the executables are identical.
+ 	#
+ 	# Do not do this for native testing since the cost to load/execute
+ 	# the test is fairly small and the comparison step actually slows
+ 	# the entire process down because it usually does not "hit".
  	set skip 0;
! 	if { ![isnative] && [info exists oldexec] } {
  	    if { [remote_file build cmp $oldexec $execname] == 0 } {
  		set skip 1;
  	    }

Minor testsuite improvement

[Jeff Law]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]


Gary Funck noted that vrp06.c has two tests with the same output.  After 
further investigation it was clear that expected output strings were too 
lenient and were in fact masking a missed optimization.

This patch tightens the expected output from the vrp dump which has the 
side effect of making each test's string unique.  Obviously the masked 
failure is xfailed.

OK for the trunk?

Thanks,
Jeff

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 943 bytes --]

	* gcc/dg/tree-ssa/vrp06.c: Tighten expected output.  Make each
	pass/fail message unique.

diff --git a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
index 1ef6c92..4edca36 100644
--- a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
+++ b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
@@ -25,7 +25,7 @@ foo (int i, int j, int a)
   return i + a + j;
 }
 
-/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1" } } */
-/* { dg-final { scan-tree-dump-times "Folding predicate j_.*to 1" 1 "vrp1" } } */
-/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate i_.*0 to 0" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate j_.*0 to 1" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate i_.*j_.* to 0" 1 "vrp1" { xfail *-*-* } } } */
 /* { dg-final { cleanup-tree-dump "vrp1" } } */

Re: Minor testsuite improvement

[Richard Biener]

On Thu, Jan 10, 2013 at 6:15 AM, Jeff Law <law@redhat.com> wrote:
>
> Gary Funck noted that vrp06.c has two tests with the same output.  After
> further investigation it was clear that expected output strings were too
> lenient and were in fact masking a missed optimization.
>
> This patch tightens the expected output from the vrp dump which has the side
> effect of making each test's string unique.  Obviously the masked failure is
> xfailed.
>
> OK for the trunk?

Hmm, but if the SSA versions are simply i_10 then i_.*0 will still match it
the same?  I think you want the more elaborate 'i_\[0-9\]* . 0' here?

Please also open a bug for the added XFAIL.

Thanks,
Richard.

> Thanks,
> Jeff
>
>         * gcc/dg/tree-ssa/vrp06.c: Tighten expected output.  Make each
>         pass/fail message unique.
>
> diff --git a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
> b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
> index 1ef6c92..4edca36 100644
> --- a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
> +++ b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
> @@ -25,7 +25,7 @@ foo (int i, int j, int a)
>    return i + a + j;
>  }
>
> -/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1"
> } } */
> -/* { dg-final { scan-tree-dump-times "Folding predicate j_.*to 1" 1 "vrp1"
> } } */
> -/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1"
> } } */
> +/* { dg-final { scan-tree-dump-times "Folding predicate i_.*0 to 0" 1
> "vrp1" } } */
> +/* { dg-final { scan-tree-dump-times "Folding predicate j_.*0 to 1" 1
> "vrp1" } } */
> +/* { dg-final { scan-tree-dump-times "Folding predicate i_.*j_.* to 0" 1
> "vrp1" { xfail *-*-* } } } */
>  /* { dg-final { cleanup-tree-dump "vrp1" } } */
>

Re: Minor testsuite improvement

[Jeff Law]

On 01/10/2013 03:13 AM, Richard Biener wrote:
>>
> Hmm, but if the SSA versions are simply i_10 then i_.*0 will still match it
> the same?  I think you want the more elaborate 'i_\[0-9\]* . 0' here?
Per Uros's suggestion I'm using i[0-9]+ to ensure we one or more digits.

>
> Please also open a bug for the added XFAIL.
55936.  Not sure if it's a regression or not -- Diego changed the 
relevant code in tree-ssa-vrp around 2006.  I didn't build a compiler 
from that era to see if VRP caught everything in one pass.

jeff

Re: Minor testsuite improvement

[Jeff Law]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On 01/10/2013 03:13 AM, Richard Biener wrote:
> On Thu, Jan 10, 2013 at 6:15 AM, Jeff Law <law@redhat.com> wrote:
>>
>> Gary Funck noted that vrp06.c has two tests with the same output.  After
>> further investigation it was clear that expected output strings were too
>> lenient and were in fact masking a missed optimization.
>>
>> This patch tightens the expected output from the vrp dump which has the side
>> effect of making each test's string unique.  Obviously the masked failure is
>> xfailed.
>>
>> OK for the trunk?
>
> Hmm, but if the SSA versions are simply i_10 then i_.*0 will still match it
> the same?  I think you want the more elaborate 'i_\[0-9\]* . 0' here?
Applied using Uros's regexp.  The final patch is attached for historical 
purposes.

Jeff

[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 1622 bytes --]

commit 115e3f9b2615bbe7bce75e4d29638e977c10ac86
Author: law <law@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Thu Jan 10 19:16:33 2013 +0000

            * gcc/dg/tree-ssa/vrp06.c: Tighten expected output.  Make each
            pass/fail message unique.
    
    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@195093 138bc75d-0d04-0410-961f-82ee72b054a4

diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 46bae82..6df9b9c 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,9 @@
+2013-01-10  Jeff Law  <law@redhat.com>
+
+	* gcc/dg/tree-ssa/vrp06.c: Tighten expected output.  Make each
+	pass/fail message unique.
+
+
 2013-01-10  Jason Merrill  <jason@redhat.com>
 
 	* ada/.gitignore: New.
diff --git a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
index 1ef6c92..cb6df0d 100644
--- a/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
+++ b/gcc/testsuite/gcc.dg/tree-ssa/vrp06.c
@@ -25,7 +25,7 @@ foo (int i, int j, int a)
   return i + a + j;
 }
 
-/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1" } } */
-/* { dg-final { scan-tree-dump-times "Folding predicate j_.*to 1" 1 "vrp1" } } */
-/* { dg-final { scan-tree-dump-times "Folding predicate i_.*to 0" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate i_\[0-9\]+.*0 to 0" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate j_\[0-9\]+.*0 to 1" 1 "vrp1" } } */
+/* { dg-final { scan-tree-dump-times "Folding predicate i_\[0-9]+.*j_\[0-9\]+.* to 0" 1 "vrp1" { xfail *-*-* } } } */
 /* { dg-final { cleanup-tree-dump "vrp1" } } */

Re: Minor testsuite improvement

[Uros Bizjak]

Hello!

>> This patch tightens the expected output from the vrp dump which has the side
>> effect of making each test's string unique.  Obviously the masked failure is
>> xfailed.
>>
>> OK for the trunk?
>
> Hmm, but if the SSA versions are simply i_10 then i_.*0 will still match it
> the same?  I think you want the more elaborate 'i_\[0-9\]* . 0' here?

IMO 'i_\[0-9\]+' would be better, we need at least one digit here.

Uros.

Re: Minor testsuite improvement

[Jeff Law]

On 01/10/2013 03:31 AM, Uros Bizjak wrote:
> Hello!
>
>>> This patch tightens the expected output from the vrp dump which has the side
>>> effect of making each test's string unique.  Obviously the masked failure is
>>> xfailed.
>>>
>>> OK for the trunk?
>>
>> Hmm, but if the SSA versions are simply i_10 then i_.*0 will still match it
>> the same?  I think you want the more elaborate 'i_\[0-9\]* . 0' here?
>
> IMO 'i_\[0-9\]+' would be better, we need at least one digit here.
Yea, this is a better regexp.  I'll update and verify I get the right 
pass/xfails.

jeff