[PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Tom de Vries]

Hi,

After de-installing gnatmake, I get:
...
Running src/gdb/testsuite/gdb.base/gdb-caching-proc.exp ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
  ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
...

In gdb.sum, we see these FAILs (each paired with an UNSUPPORTED as well)
followed by:
...
PASS: gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
...

Fix this by ignoring PASS/FAIL/UNSUPPORTED during testing, such that we have
just the consistency PASS/FAIL.

Tested on x86_64-linux, with gnatmake installed and de-installed.

OK for trunk?

Thanks,
- Tom

[gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

gdb/testsuite/ChangeLog:

2020-02-13  Tom de Vries  <tdevries@suse.de>

	* gdb.base/gdb-caching-proc.exp (ignore_pass, ignore_fail)
	(ignore_unsupported): New proc.
	(test_proc): Temporarily rename pass/fail/ignore to
	ignore_{pass,fail,ignore} in order to ignore PASS/FAIL/IGNORE messages
	from the tested procs.

---
 gdb/testsuite/gdb.base/gdb-caching-proc.exp | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gdb/testsuite/gdb.base/gdb-caching-proc.exp b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
index b2d71a5e7d..536d7e89c1 100644
--- a/gdb/testsuite/gdb.base/gdb-caching-proc.exp
+++ b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
@@ -22,12 +22,29 @@
 # number of reruns is constant-bounded, and the increase in runtime is bound to
 # this test-case, and could be disabled on slow targets.
 
+proc ignore_pass { args } {
+
+}
+proc ignore_fail { args } {
+
+}
+proc ignore_unsupported { args } {
+
+}
+
 # Test gdb_caching_proc NAME
 proc test_proc { name } {
     set real_name gdb_real__$name
 
     set resultlist [list]
 
+    rename pass save_pass
+    rename fail save_fail
+    rename unsupported save_unsupported
+    rename ignore_pass pass
+    rename ignore_fail fail
+    rename ignore_unsupported unsupported
+
     set first [$real_name]
     lappend resultlist $first
 
@@ -44,6 +61,13 @@ proc test_proc { name } {
 	}
     }
 
+    rename pass ignore_pass
+    rename fail ignore_fail
+    rename unsupported ignore_unsupported
+    rename save_pass pass
+    rename save_fail fail
+    rename save_unsupported unsupported
+
     if { $racy  == 0 } {
 	pass "$name consistency"
     } else {

Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Tom Tromey]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> 2020-02-13  Tom de Vries  <tdevries@suse.de>

Tom> 	* gdb.base/gdb-caching-proc.exp (ignore_pass, ignore_fail)
Tom> 	(ignore_unsupported): New proc.
Tom> 	(test_proc): Temporarily rename pass/fail/ignore to
Tom> 	ignore_{pass,fail,ignore} in order to ignore PASS/FAIL/IGNORE messages
Tom> 	from the tested procs.

Can we just require that caching proc bodies not invoke tests?

Tom> +    rename pass save_pass
Tom> +    rename fail save_fail
Tom> +    rename unsupported save_unsupported
Tom> +    rename ignore_pass pass
Tom> +    rename ignore_fail fail
Tom> +    rename ignore_unsupported unsupported

In this situation perhaps it's best to catch any errors in the body, so
that the code to rename things back will always be run.

Then it could re-throw the error if needed at the end.

Tom

Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Tom de Vries]

On 13-02-2020 15:32, Tom Tromey wrote:
>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
> 
> Tom> 2020-02-13  Tom de Vries  <tdevries@suse.de>
> 
> Tom> 	* gdb.base/gdb-caching-proc.exp (ignore_pass, ignore_fail)
> Tom> 	(ignore_unsupported): New proc.
> Tom> 	(test_proc): Temporarily rename pass/fail/ignore to
> Tom> 	ignore_{pass,fail,ignore} in order to ignore PASS/FAIL/IGNORE messages
> Tom> 	from the tested procs.
> 
> Can we just require that caching proc bodies not invoke tests?
> 

We could, I don't have preference one way or the other.

If I remove all the fail calls from gnat_runtime_has_debug_info, we
still get these PASSes from gdb_compile_ada:
...
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
PASS: gdb.base/gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
...

But we can factor that out, I've already done that once here (
https://sourceware.org/ml/gdb-patches/2019-10/msg01109.html ).

Simon, WDYT?

Thanks,
- Tom

Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Simon Marchi]

On 2020-02-13 10:02 a.m., Tom de Vries wrote:
> On 13-02-2020 15:32, Tom Tromey wrote:
>>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
>>
>> Tom> 2020-02-13  Tom de Vries  <tdevries@suse.de>
>>
>> Tom> 	* gdb.base/gdb-caching-proc.exp (ignore_pass, ignore_fail)
>> Tom> 	(ignore_unsupported): New proc.
>> Tom> 	(test_proc): Temporarily rename pass/fail/ignore to
>> Tom> 	ignore_{pass,fail,ignore} in order to ignore PASS/FAIL/IGNORE messages
>> Tom> 	from the tested procs.
>>
>> Can we just require that caching proc bodies not invoke tests?
>>
> 
> We could, I don't have preference one way or the other.
> 
> If I remove all the fail calls from gnat_runtime_has_debug_info, we
> still get these PASSes from gdb_compile_ada:
> ...
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
> PASS: gdb.base/gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
> ...
> 
> But we can factor that out, I've already done that once here (
> https://sourceware.org/ml/gdb-patches/2019-10/msg01109.html ).
> 
> Simon, WDYT?
> 
> Thanks,
> - Tom
> 

If you can indeed do that, I think it would be best.

Simon

Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 2014 bytes --]

On 17-02-2020 21:51, Simon Marchi wrote:
> On 2020-02-13 10:02 a.m., Tom de Vries wrote:
>> On 13-02-2020 15:32, Tom Tromey wrote:
>>>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
>>>
>>> Tom> 2020-02-13  Tom de Vries  <tdevries@suse.de>
>>>
>>> Tom> 	* gdb.base/gdb-caching-proc.exp (ignore_pass, ignore_fail)
>>> Tom> 	(ignore_unsupported): New proc.
>>> Tom> 	(test_proc): Temporarily rename pass/fail/ignore to
>>> Tom> 	ignore_{pass,fail,ignore} in order to ignore PASS/FAIL/IGNORE messages
>>> Tom> 	from the tested procs.
>>>
>>> Can we just require that caching proc bodies not invoke tests?
>>>
>>
>> We could, I don't have preference one way or the other.
>>
>> If I remove all the fail calls from gnat_runtime_has_debug_info, we
>> still get these PASSes from gdb_compile_ada:
>> ...
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: compilation gnat_debug_info_test.adb
>> PASS: gdb.base/gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
>> ...
>>
>> But we can factor that out, I've already done that once here (
>> https://sourceware.org/ml/gdb-patches/2019-10/msg01109.html ).
>>
>> Simon, WDYT?
>>
>> Thanks,
>> - Tom
>>
> 
> If you can indeed do that, I think it would be best.

Done.  Ok for trunk?

Thanks,
- Tom


[-- Attachment #2: 0001-gdb-testsuite-Don-t-pass-fail-in-gnat_runtime_has_debug_info.patch --]
[-- Type: text/x-patch, Size: 2895 bytes --]

[gdb/testsuite] Don't pass/fail in gnat_runtime_has_debug_info

After de-installing gnatmake, I get:
...
Running src/gdb/testsuite/gdb.base/gdb-caching-proc.exp ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
  ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
...

In gdb.sum, we see these FAILs (each paired with an UNSUPPORTED as well)
followed by:
...
PASS: gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
...

Fix this by removing the fail calls in gnat_runtime_has_debug_info, as well
as using a new variant gdb_compile_ada_1 that doesn't call pass.

Tested on x86_64-linux, with gnatmake installed and de-installed.

gdb/testsuite/ChangeLog:

2020-02-18  Tom de Vries  <tdevries@suse.de>

	* lib/ada.exp (gdb_compile_ada_1): Factor out of ...
	(gdb_compile_ada): ... here.
	(gnat_runtime_has_debug_info): Remove fail calls.  Use
	gdb_compile_ada_1 instead of gdb_compile_ada.

---
 gdb/testsuite/lib/ada.exp | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/gdb/testsuite/lib/ada.exp b/gdb/testsuite/lib/ada.exp
index 9933cc951e..0689eeb239 100644
--- a/gdb/testsuite/lib/ada.exp
+++ b/gdb/testsuite/lib/ada.exp
@@ -52,9 +52,9 @@ proc target_compile_ada_from_dir {builddir source dest type options} {
     return -options $options $result
 }
 
-# Compile some Ada code.
+# Compile some Ada code.  Return "" if the compile was successful.
 
-proc gdb_compile_ada {source dest type options} {
+proc gdb_compile_ada_1 {source dest type options} {
 
     set srcdir [file dirname $source]
     set gprdir [file dirname $srcdir]
@@ -80,6 +80,15 @@ proc gdb_compile_ada {source dest type options} {
     # We therefore simply check whether the dest file has been created
     # or not. Unless not present, the build has succeeded.
     if [file exists $dest] { set result "" }
+    return $result
+}
+
+# Compile some Ada code.  Generate "PASS: foo.exp: compilation SOURCE" if the
+# compile was successful.
+
+proc gdb_compile_ada {source dest type options} {
+    set result [gdb_compile_ada_1 $source $dest $type $options]
+
     gdb_compile_test $source $result
     return $result
 }
@@ -162,15 +171,13 @@ gdb_caching_proc gnat_runtime_has_debug_info {
     set src "$srcdir/lib/gnat_debug_info_test.adb"
     set dst [standard_output_file "gnat_debug_info_test"]
 
-    if { [gdb_compile_ada $src $dst executable {debug}] != "" } {
-	fail "failed to compile gnat-debug-info test binary"
+    if { [gdb_compile_ada_1 $src $dst executable {debug}] != "" } {
 	return 0
     }
 
     clean_restart $dst
 
     if { ! [runto "GNAT_Debug_Info_Test"] } {
-	fail "failed to run to GNAT_Debug_Info_Test"
 	return 0
     }
 
@@ -183,7 +190,6 @@ gdb_caching_proc gnat_runtime_has_debug_info {
 	}
 	default {
 	    # Some other unexpected output...
-	    fail $gdb_test_name
 	}
     }
 

Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in gdb-caching-proc.exp

[Simon Marchi]

On 2020-02-17 6:23 p.m., Tom de Vries wrote:
> diff --git a/gdb/testsuite/lib/ada.exp b/gdb/testsuite/lib/ada.exp
> index 9933cc951e..0689eeb239 100644
> --- a/gdb/testsuite/lib/ada.exp
> +++ b/gdb/testsuite/lib/ada.exp
> @@ -52,9 +52,9 @@ proc target_compile_ada_from_dir {builddir source dest type options} {
>      return -options $options $result
>  }
>  
> -# Compile some Ada code.
> +# Compile some Ada code.  Return "" if the compile was successful.
>  
> -proc gdb_compile_ada {source dest type options} {
> +proc gdb_compile_ada_1 {source dest type options} {
>  
>      set srcdir [file dirname $source]
>      set gprdir [file dirname $srcdir]
> @@ -80,6 +80,15 @@ proc gdb_compile_ada {source dest type options} {
>      # We therefore simply check whether the dest file has been created
>      # or not. Unless not present, the build has succeeded.
>      if [file exists $dest] { set result "" }
> +    return $result
> +}
> +
> +# Compile some Ada code.  Generate "PASS: foo.exp: compilation SOURCE" if the
> +# compile was successful.
> +
> +proc gdb_compile_ada {source dest type options} {
> +    set result [gdb_compile_ada_1 $source $dest $type $options]
> +
>      gdb_compile_test $source $result
>      return $result
>  }
> @@ -162,15 +171,13 @@ gdb_caching_proc gnat_runtime_has_debug_info {
>      set src "$srcdir/lib/gnat_debug_info_test.adb"
>      set dst [standard_output_file "gnat_debug_info_test"]
>  
> -    if { [gdb_compile_ada $src $dst executable {debug}] != "" } {
> -	fail "failed to compile gnat-debug-info test binary"
> +    if { [gdb_compile_ada_1 $src $dst executable {debug}] != "" } {
>  	return 0
>      }
>  
>      clean_restart $dst
>  
>      if { ! [runto "GNAT_Debug_Info_Test"] } {
> -	fail "failed to run to GNAT_Debug_Info_Test"
>  	return 0
>      }
>  
> @@ -183,7 +190,6 @@ gdb_caching_proc gnat_runtime_has_debug_info {
>  	}
>  	default {
>  	    # Some other unexpected output...
> -	    fail $gdb_test_name
>  	}
>      }
>  

I'd be tempted to keep these last two "fail"s.  They should not show up normally,
regardless of whether gnatmake is installed.  But let's say:

- the Ada runtime ever changes (__gnat_debug_raise_exception is renamed), or
- the output of the command "whatis __gnat_debug_raise_exception" changes

then they'll show up, which is desirable because it means we need to come
and change something here.  If the output of
"whatis __gnat_debug_raise_exception" changes in a way that it is not
recognized anymore and there is no FAIL, then gnat_runtime_has_debug_info
will just start returning false all the time and we might not notice it for a
while.

Simon

[committed][gdb/testsuite] Handle missing gnatmake in gnat_runtime_has_debug_info

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 2060 bytes --]

[ was: Re: [PATCH][gdb/testsuite] Ignore pass/fail/unsupported in
gdb-caching-proc.exp ]

On 18-02-2020 00:58, Simon Marchi wrote:
>>  
>>      if { ! [runto "GNAT_Debug_Info_Test"] } {
>> -	fail "failed to run to GNAT_Debug_Info_Test"
>>  	return 0
>>      }
>>  
>> @@ -183,7 +190,6 @@ gdb_caching_proc gnat_runtime_has_debug_info {
>>  	}
>>  	default {
>>  	    # Some other unexpected output...
>> -	    fail $gdb_test_name
>>  	}
>>      }
>>  
> 
> I'd be tempted to keep these last two "fail"s.  They should not show up normally,
> regardless of whether gnatmake is installed.  But let's say:
> 
> - the Ada runtime ever changes (__gnat_debug_raise_exception is renamed), or
> - the output of the command "whatis __gnat_debug_raise_exception" changes
> 
> then they'll show up, which is desirable because it means we need to come
> and change something here.  If the output of
> "whatis __gnat_debug_raise_exception" changes in a way that it is not
> recognized anymore and there is no FAIL, then gnat_runtime_has_debug_info
> will just start returning false all the time and we might not notice it for a
> while.

Ack, committed as attached, keeping the last two fails.

FWIW, I scanned a bit other gdb_caching_procs and there seems to be an
idiom to handle hitting the default case of an expect using a warning,
f.i. in skip_aarch64_sve_tests:
...
        default {
          warning "\n$me: default case taken"
            set skip_sve_tests 1
        }
...
So we could use that at least for the last fail.

Also, here and there verbose seems to be used instead of fail.

The gdb_caching_procs seem to avoid explicitly calling pass and fail,
but usage of gdb_test_multiple does happen a couple of time, which might
result in fails etc (but then again, that might not have been
intentional, I'm not sure).

In any case, we could either:
- forbid calling pass in gdb_caching_proc, or
- more accommodatingly, ignore pass in gdb_caching_proc.
Both by renaming pass in gdb_do_cache.

Likewise, we could perhaps redirect fails to warnings.

Thanks,
- Tom

[-- Attachment #2: 0001-gdb-testsuite-Handle-missing-gnatmake-in-gnat_runtime_has_debug_info.patch --]
[-- Type: text/x-patch, Size: 2874 bytes --]

[gdb/testsuite] Handle missing gnatmake in gnat_runtime_has_debug_info

After de-installing gnatmake, I get on stdout/stderr:
...
Running src/gdb/testsuite/gdb.base/gdb-caching-proc.exp ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
  ...
FAIL: gdb-caching-proc.exp: failed to compile gnat-debug-info test binary
...

In gdb.sum, we see these FAILs (each paired with an UNSUPPORTED as well)
followed by:
...
PASS: gdb-caching-proc.exp: gnat_runtime_has_debug_info consistency
...

Likewise, after re-installing gnatmake, I get a PASS for each of the
UNSUPPORTEDs, and the FAILs disappear.

The FAIL comes from gnat_runtime_has_debug_info, the PASS/UNSUPPORTED comes
from gdb_compile_ada.

Fix this by removing the corresponding fail call in
gnat_runtime_has_debug_info, as well as using a new variant gdb_compile_ada_1
that doesn't call pass/unsupported.

Tested on x86_64-linux, with gnatmake installed and de-installed.

gdb/testsuite/ChangeLog:

2020-02-18  Tom de Vries  <tdevries@suse.de>

	* lib/ada.exp (gdb_compile_ada_1): Factor out of ...
	(gdb_compile_ada): ... here.
	(gnat_runtime_has_debug_info): Remove fail call for gdb_compile_ada
	failure.  Use gdb_compile_ada_1 instead of gdb_compile_ada.

---
 gdb/testsuite/lib/ada.exp | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/gdb/testsuite/lib/ada.exp b/gdb/testsuite/lib/ada.exp
index 9933cc951e..6648566ec3 100644
--- a/gdb/testsuite/lib/ada.exp
+++ b/gdb/testsuite/lib/ada.exp
@@ -52,9 +52,9 @@ proc target_compile_ada_from_dir {builddir source dest type options} {
     return -options $options $result
 }
 
-# Compile some Ada code.
+# Compile some Ada code.  Return "" if the compile was successful.
 
-proc gdb_compile_ada {source dest type options} {
+proc gdb_compile_ada_1 {source dest type options} {
 
     set srcdir [file dirname $source]
     set gprdir [file dirname $srcdir]
@@ -80,6 +80,15 @@ proc gdb_compile_ada {source dest type options} {
     # We therefore simply check whether the dest file has been created
     # or not. Unless not present, the build has succeeded.
     if [file exists $dest] { set result "" }
+    return $result
+}
+
+# Compile some Ada code.  Generate "PASS: foo.exp: compilation SOURCE" if the
+# compile was successful.
+
+proc gdb_compile_ada {source dest type options} {
+    set result [gdb_compile_ada_1 $source $dest $type $options]
+
     gdb_compile_test $source $result
     return $result
 }
@@ -162,8 +171,7 @@ gdb_caching_proc gnat_runtime_has_debug_info {
     set src "$srcdir/lib/gnat_debug_info_test.adb"
     set dst [standard_output_file "gnat_debug_info_test"]
 
-    if { [gdb_compile_ada $src $dst executable {debug}] != "" } {
-	fail "failed to compile gnat-debug-info test binary"
+    if { [gdb_compile_ada_1 $src $dst executable {debug}] != "" } {
 	return 0
     }
 

[RFC][gdb/testsuite] Ignore pass in gdb_caching_proc

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

[ was : Re: [committed][gdb/testsuite] Handle missing gnatmake in
gnat_runtime_has_debug_info ]

On 18-02-2020 10:39, Tom de Vries wrote:
> FWIW, I scanned a bit other gdb_caching_procs and there seems to be an
> idiom to handle hitting the default case of an expect using a warning,
> f.i. in skip_aarch64_sve_tests:
> ...
>         default {
>           warning "\n$me: default case taken"
>             set skip_sve_tests 1
>         }
> ...
> So we could use that at least for the last fail.
> 
> Also, here and there verbose seems to be used instead of fail.
> 
> The gdb_caching_procs seem to avoid explicitly calling pass and fail,
> but usage of gdb_test_multiple does happen a couple of time, which might
> result in fails etc (but then again, that might not have been
> intentional, I'm not sure).
> 
> In any case, we could either:
> - forbid calling pass in gdb_caching_proc, or
> - more accommodatingly, ignore pass in gdb_caching_proc.
> Both by renaming pass in gdb_do_cache.
> 
> Likewise, we could perhaps redirect fails to warnings.

This RFC patch implements "ignore pass in gdb_caching_proc".

Any comments?

Thanks,
- Tom

[-- Attachment #2: 0001-gdb-testsuite-Ignore-pass-in-gdb_caching_proc.patch --]
[-- Type: text/x-patch, Size: 2506 bytes --]

[gdb/testsuite] Ignore pass in gdb_caching_proc

gdb/testsuite/ChangeLog:

2020-02-18  Tom de Vries  <tdevries@suse.de>

	* lib/cache.exp (ignore_pass, gdb_do_cache_wrap): New proc.
	(gdb_do_cache): Use gdb_do_cache_wrap.
	* gdb.base/gdb-caching-proc.exp (test_proc): Use gdb_do_cache_wrap.

---
 gdb/testsuite/gdb.base/gdb-caching-proc.exp |  4 ++--
 gdb/testsuite/lib/cache.exp                 | 29 ++++++++++++++++++++++++++++-
 2 files changed, 30 insertions(+), 3 deletions(-)

diff --git a/gdb/testsuite/gdb.base/gdb-caching-proc.exp b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
index b2d71a5e7d..3810347a65 100644
--- a/gdb/testsuite/gdb.base/gdb-caching-proc.exp
+++ b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
@@ -28,7 +28,7 @@ proc test_proc { name } {
 
     set resultlist [list]
 
-    set first [$real_name]
+    set first [gdb_do_cache_wrap $real_name]
     lappend resultlist $first
 
     # Ten repetitions was enough to trigger target_supports_scheduler_locking,
@@ -37,7 +37,7 @@ proc test_proc { name } {
 
     set racy 0
     for {set i 0} {$i < $repeat} {incr i} {
-	set rerun [$real_name]
+	set rerun [gdb_do_cache_wrap $real_name]
 	lappend resultlist $rerun
 	if { $rerun != $first } {
 	    set racy 1
diff --git a/gdb/testsuite/lib/cache.exp b/gdb/testsuite/lib/cache.exp
index dc57cab964..357d232ae3 100644
--- a/gdb/testsuite/lib/cache.exp
+++ b/gdb/testsuite/lib/cache.exp
@@ -17,6 +17,33 @@
 # The in-memory cache.
 array set gdb_data_cache {}
 
+proc ignore_pass { msg } {
+    verbose -log "gdb_do_cache_wrap ignoring pass: $msg"
+}
+
+proc gdb_do_cache_wrap {real_name} {
+    if { [info procs save_pass] != "" } {
+	return [uplevel 2 $real_name]
+    }
+
+    rename pass save_pass
+    rename ignore_pass pass
+
+    set code [catch {uplevel 2 $real_name} result]
+
+    rename pass ignore_pass
+    rename save_pass pass
+
+    if {$code == 1} {
+	global errorInfo errorCode
+	return -code error -errorinfo $errorInfo -errorcode $errorCode $result
+    } elseif {$code > 1} {
+	return -code $code $result
+    }
+
+    return $result
+}
+
 # A helper for gdb_caching_proc that handles the caching.
 
 proc gdb_do_cache {name} {
@@ -46,7 +73,7 @@ proc gdb_do_cache {name} {
     }
 
     set real_name gdb_real__$name
-    set gdb_data_cache($cache_name) [uplevel 1 $real_name]
+    set gdb_data_cache($cache_name) [gdb_do_cache_wrap $real_name]
 
     if {[info exists GDB_PARALLEL]} {
 	verbose "$name: returning '$gdb_data_cache($cache_name)' and writing file" 2

Re: [RFC][gdb/testsuite] Ignore pass in gdb_caching_proc

[Tom Tromey]

>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:

Tom> +proc ignore_pass { msg } {
Tom> +    verbose -log "gdb_do_cache_wrap ignoring pass: $msg"
Tom> +}
Tom> +
Tom> +proc gdb_do_cache_wrap {real_name} {
Tom> +    if { [info procs save_pass] != "" } {

I think the new procs should have introductory comments, but other than
that, this seems fine to me.

Tom

Re: [RFC][gdb/testsuite] Ignore pass in gdb_caching_proc

[Tom de Vries]

[-- Attachment #1: Type: text/plain, Size: 466 bytes --]

On 18-02-2020 16:29, Tom Tromey wrote:
>>>>>> "Tom" == Tom de Vries <tdevries@suse.de> writes:
> 
> Tom> +proc ignore_pass { msg } {
> Tom> +    verbose -log "gdb_do_cache_wrap ignoring pass: $msg"
> Tom> +}
> Tom> +
> Tom> +proc gdb_do_cache_wrap {real_name} {
> Tom> +    if { [info procs save_pass] != "" } {
> 
> I think the new procs should have introductory comments, but other than
> that, this seems fine to me.

Done, and committed as below.

Thanks,
- Tom

[-- Attachment #2: 0001-gdb-testsuite-Ignore-pass-in-gdb_caching_proc.patch --]
[-- Type: text/x-patch, Size: 3105 bytes --]

[gdb/testsuite] Ignore pass in gdb_caching_proc

Before commit d4295de4f3 "[gdb/testsuite] Handle missing gnatmake in
gnat_runtime_has_debug_info", calling the gdb_caching_proc
gnat_runtime_has_debug_info could generate a pass because of using
gdb_compile_ada.

This has been fixed in that commit by using a factored out variant
gdb_compile_ada_1, which does not call pass.

Additionally, fix cases like this in more generic way: by ignoring pass calls
during execution of a gdb_caching_proc.

Build and reg-tested on x86_64-linux.

gdb/testsuite/ChangeLog:

2020-02-18  Tom de Vries  <tdevries@suse.de>

	* lib/cache.exp (ignore_pass, gdb_do_cache_wrap): New proc.
	(gdb_do_cache): Use gdb_do_cache_wrap.
	* gdb.base/gdb-caching-proc.exp (test_proc): Use gdb_do_cache_wrap.

---
 gdb/testsuite/gdb.base/gdb-caching-proc.exp |  4 ++--
 gdb/testsuite/lib/cache.exp                 | 31 ++++++++++++++++++++++++++++-
 2 files changed, 32 insertions(+), 3 deletions(-)

diff --git a/gdb/testsuite/gdb.base/gdb-caching-proc.exp b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
index b2d71a5e7d..3810347a65 100644
--- a/gdb/testsuite/gdb.base/gdb-caching-proc.exp
+++ b/gdb/testsuite/gdb.base/gdb-caching-proc.exp
@@ -28,7 +28,7 @@ proc test_proc { name } {
 
     set resultlist [list]
 
-    set first [$real_name]
+    set first [gdb_do_cache_wrap $real_name]
     lappend resultlist $first
 
     # Ten repetitions was enough to trigger target_supports_scheduler_locking,
@@ -37,7 +37,7 @@ proc test_proc { name } {
 
     set racy 0
     for {set i 0} {$i < $repeat} {incr i} {
-	set rerun [$real_name]
+	set rerun [gdb_do_cache_wrap $real_name]
 	lappend resultlist $rerun
 	if { $rerun != $first } {
 	    set racy 1
diff --git a/gdb/testsuite/lib/cache.exp b/gdb/testsuite/lib/cache.exp
index dc57cab964..25bfe02510 100644
--- a/gdb/testsuite/lib/cache.exp
+++ b/gdb/testsuite/lib/cache.exp
@@ -17,6 +17,35 @@
 # The in-memory cache.
 array set gdb_data_cache {}
 
+# Print pass message msg into gdb.log
+proc ignore_pass { msg } {
+    verbose -log "gdb_do_cache_wrap ignoring pass: $msg"
+}
+
+# Call proc real_name and return the result, while ignoring calls to pass.
+proc gdb_do_cache_wrap {real_name} {
+    if { [info procs save_pass] != "" } {
+	return [uplevel 2 $real_name]
+    }
+
+    rename pass save_pass
+    rename ignore_pass pass
+
+    set code [catch {uplevel 2 $real_name} result]
+
+    rename pass ignore_pass
+    rename save_pass pass
+
+    if {$code == 1} {
+	global errorInfo errorCode
+	return -code error -errorinfo $errorInfo -errorcode $errorCode $result
+    } elseif {$code > 1} {
+	return -code $code $result
+    }
+
+    return $result
+}
+
 # A helper for gdb_caching_proc that handles the caching.
 
 proc gdb_do_cache {name} {
@@ -46,7 +75,7 @@ proc gdb_do_cache {name} {
     }
 
     set real_name gdb_real__$name
-    set gdb_data_cache($cache_name) [uplevel 1 $real_name]
+    set gdb_data_cache($cache_name) [gdb_do_cache_wrap $real_name]
 
     if {[info exists GDB_PARALLEL]} {
 	verbose "$name: returning '$gdb_data_cache($cache_name)' and writing file" 2