[PATCH] Add check-valgrind

[PATCH] Add check-valgrind

[Tom de Vries]

Hi,

Add a make target check-valgrind, that applies valgrind to dwz when used in
the testsuite.

Any comments?

Thanks,
- Tom

Add check-valgrind

2021-03-25  Tom de Vries  <tdevries@suse.de>

	* Makefile (check-valgrind): New target.

---
 Makefile | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index ff55738..470741b 100644
--- a/Makefile
+++ b/Makefile
@@ -143,9 +143,15 @@ cycle:
 # WARNING: Couldn't find the global config file.
 DEJAGNU ?= /dev/null
 
-check: dwz $(TEST_EXECS)
+check check-valgrind: dwz $(TEST_EXECS)
 	mkdir -p testsuite-bin
-	cd testsuite-bin; ln -sf $(PWD)/dwz .
+	cd testsuite-bin; \
+	  if [ "$@" = "check" ]; then \
+	    ln -sf $(PWD)/dwz .; \
+          else \
+            echo "valgrind -q $(PWD)/dwz \"\$$@\"" > dwz; \
+	    chmod +x dwz; \
+          fi
 	export DEJAGNU=$(DEJAGNU); \
 	export PATH=$(PWD)/testsuite-bin:$$PATH; export LC_ALL=C; \
 	runtest --tool=dwz -srcdir $(srcdir)/testsuite $(RUNTESTFLAGS)

Re: [PATCH] Add check-valgrind

[Mark Wielaard]

Hi,

On Thu, Mar 25, 2021 at 02:57:17PM +0100, Tom de Vries wrote:
> Add a make target check-valgrind, that applies valgrind to dwz when used in
> the testsuite.
> 
> Any comments?

It might be good to simply always run under valgrind when it is
installed. That makes make check slightly slower by default, but it
makes sure things are always also ran under valgrind if available.

To make sure that a valgrind tool error really causes a check failure use
--error-exitcode=99 (or some other non-zero code):

   --error-exitcode=<number> [default: 0]
       Specifies an alternative exit code to return if Valgrind reported
       any errors in the run. When set to the default value (zero), the
       return value from Valgrind will always be the return value of the
       process being simulated. When set to a nonzero value, that value is
       returned instead, if Valgrind detects any errors. This is useful
       for using Valgrind as part of an automated test suite, since it
       makes it easy to detect test cases for which Valgrind has reported
       errors, just by inspecting return codes.

Cheers,

Mark

Re: [PATCH] Add check-valgrind

[Tom de Vries]

On 3/25/21 9:32 PM, Mark Wielaard wrote:
> Hi,
> 
> On Thu, Mar 25, 2021 at 02:57:17PM +0100, Tom de Vries wrote:
>> Add a make target check-valgrind, that applies valgrind to dwz when used in
>> the testsuite.
>>
>> Any comments?
> 
> It might be good to simply always run under valgrind when it is
> installed. That makes make check slightly slower by default, but it
> makes sure things are always also ran under valgrind if available.
> 

Hmm, makes sense.  Make check with dwz-tests.exp goes from 11s to 34s on
my laptop, which is still bearable.

OTOH, I also run the external testsuite once in a while, and that one
goes from 1m19s to 15m17s, which I don't like as default behaviour.

So I'm not sure yet about making this the default.

> To make sure that a valgrind tool error really causes a check failure use
> --error-exitcode=99 (or some other non-zero code):
> 
>    --error-exitcode=<number> [default: 0]
>        Specifies an alternative exit code to return if Valgrind reported
>        any errors in the run. When set to the default value (zero), the
>        return value from Valgrind will always be the return value of the
>        process being simulated. When set to a nonzero value, that value is
>        returned instead, if Valgrind detects any errors. This is useful
>        for using Valgrind as part of an automated test suite, since it
>        makes it easy to detect test cases for which Valgrind has reported
>        errors, just by inspecting return codes.
> 

Thanks, I've included that, and committed.

- Tom