From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (wildebeest.demon.nl [212.238.236.112]) by sourceware.org (Postfix) with ESMTPS id CA8BC3848415 for ; Fri, 3 Sep 2021 23:47:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org CA8BC3848415 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from reform (deer0x19.wildebeest.org [172.31.17.155]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id 306F830002F5; Sat, 4 Sep 2021 01:47:44 +0200 (CEST) Received: by reform (Postfix, from userid 1000) id 746EA2E80B62; Sat, 4 Sep 2021 01:47:43 +0200 (CEST) Date: Sat, 4 Sep 2021 01:47:43 +0200 From: Mark Wielaard To: buildbot@builder.wildebeest.org Cc: elfutils-devel@sourceware.org Subject: Re: Buildbot failure in Wildebeest Builder on whole buildset Message-ID: References: <20210903140840.A291B8026A2@builder.wildebeest.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="im3y/YpaazTM5HXr" Content-Disposition: inline In-Reply-To: <20210903140840.A291B8026A2@builder.wildebeest.org> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_BADIPHTTP, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Sep 2021 23:47:48 -0000 --im3y/YpaazTM5HXr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Sep 03, 2021 at 02:08:40PM +0000, buildbot@builder.wildebeest.org wrote: > The Buildbot has detected a new failure on builder elfutils-debian-armhf while building elfutils. > Full details are available at: > https://builder.wildebeest.org/buildbot/#builders/15/builds/603 > > Buildbot URL: https://builder.wildebeest.org/buildbot/ > > Worker for this Build: debian-armhf > > Build Reason: > Blamelist: Mark Wielaard > > BUILD FAILED: failed test (failure) So some of the builders did pass without any failures. But some had errors, and not all the same ones. I couldn't replicate locally (everything was green) both with make check and make distcheck. But looking at the tests I noticed not all used a clean new cache and some used the default database. So hopefully the attached patch makes things a bit more stable by explicitly setting the DEBUGINFOD_CACHE_PATH and invoke debuginfod with a fresh sqlite database. Cheers, Mark --im3y/YpaazTM5HXr Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-tests-Make-sure-all-debuginfod-tests-use-a-clean-dat.patch" >From 8e9edeab19931f878e16cc1bff6ac286937e997f Mon Sep 17 00:00:00 2001 From: Mark Wielaard Date: Sat, 4 Sep 2021 01:40:53 +0200 Subject: [PATCH] tests: Make sure all debuginfod tests use a clean database and cache. Always set DEBUGINFOD_CACHE_PATH to an unique (new) directory and make sure that each debuginfod invocation uses a new sqlite database. Signed-off-by: Mark Wielaard --- tests/ChangeLog | 15 +++++++++++++++ tests/run-debuginfod-000-permission.sh | 3 ++- tests/run-debuginfod-archive-groom.sh | 3 ++- tests/run-debuginfod-archive-test.sh | 5 ++++- tests/run-debuginfod-dlopen.sh | 7 ++++++- tests/run-debuginfod-duplicate-urls.sh | 4 ++++ tests/run-debuginfod-extraction.sh | 6 +++++- tests/run-debuginfod-fd-prefetch-caches.sh | 6 +++++- tests/run-debuginfod-malformed.sh | 3 ++- tests/run-debuginfod-sizetime.sh | 6 +++++- tests/run-debuginfod-tmp-home.sh | 3 ++- tests/run-debuginfod-writable.sh | 6 +++++- 12 files changed, 57 insertions(+), 10 deletions(-) diff --git a/tests/ChangeLog b/tests/ChangeLog index 9540f738..178697bb 100644 --- a/tests/ChangeLog +++ b/tests/ChangeLog @@ -1,3 +1,18 @@ +2021-09-03 Mark Wielaard + + * run-debuginfod-000-permission.sh: Set DEBUGINFOD_CACHE_PATH + and use an unique sqlite db. + * run-debuginfod-archive-groom.sh: Likewise. + * run-debuginfod-archive-test.sh: Likewise. + * run-debuginfod-dlopen.sh: Likewise. + * run-debuginfod-duplicate-urls.sh: Likewise. + * run-debuginfod-extraction.sh: Likewise. + * run-debuginfod-fd-prefetch-caches.sh: Likewise. + * run-debuginfod-malformed.sh: Likewise. + * run-debuginfod-sizetime.sh: Likewise. + * run-debuginfod-tmp-home.sh: Likewise. + * run-debuginfod-writable.sh: Likewise. + 2021-09-03 Mark Wielaard * debuginfod-subr.sh (EXTRA_DIST): Add debuginfod-subr.sh. diff --git a/tests/run-debuginfod-000-permission.sh b/tests/run-debuginfod-000-permission.sh index 28e54385..1e92bdb8 100755 --- a/tests/run-debuginfod-000-permission.sh +++ b/tests/run-debuginfod-000-permission.sh @@ -22,12 +22,13 @@ set -x unset VALGRIND_CMD DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache # This variable is essential and ensures no time-race for claiming ports occurs # set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test base=8000 get_ports -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 \ +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -d $DB \ -t0 -g0 -v ${PWD} > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 diff --git a/tests/run-debuginfod-archive-groom.sh b/tests/run-debuginfod-archive-groom.sh index 0ee056ff..4b991f53 100755 --- a/tests/run-debuginfod-archive-groom.sh +++ b/tests/run-debuginfod-archive-groom.sh @@ -26,6 +26,7 @@ unset VALGRIND_CMD base=8100 get_ports DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache export DEBUGINFOD_TIMEOUT=10 export DEBUGINFOD_URLS='http://127.0.0.1:'$PORT1 @@ -68,7 +69,7 @@ if [ "$zstd" = "false" ]; then # nuke the zstd fedora 31 ones rm -vrf R/debuginfod-rpms/fedora31 fi -tempfiles vlog3 $DB +tempfiles vlog3 cp -rvp ${abs_srcdir}/debuginfod-tars Z kill -USR1 $PID1 # Wait till both files are in the index and scan/index fully finished diff --git a/tests/run-debuginfod-archive-test.sh b/tests/run-debuginfod-archive-test.sh index bc500540..1ec57bba 100755 --- a/tests/run-debuginfod-archive-test.sh +++ b/tests/run-debuginfod-archive-test.sh @@ -28,8 +28,11 @@ mkdir R # set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test base=8300 get_ports +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache -env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -R -p $PORT1 -t0 -g0 -v R > vlog$PORT1 2>&1 & +env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -R -p $PORT1 -d $DB -t0 -g0 -v R > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 errfiles vlog$PORT1 diff --git a/tests/run-debuginfod-dlopen.sh b/tests/run-debuginfod-dlopen.sh index 6476612c..0f2d13e2 100755 --- a/tests/run-debuginfod-dlopen.sh +++ b/tests/run-debuginfod-dlopen.sh @@ -27,7 +27,12 @@ mkdir F # set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test base=8500 get_ports -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod -F -R $VERBOSE -p $PORT1 \ + +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod -F -R $VERBOSE -p $PORT1 -d $DB \ -t0 -g0 -v F > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 diff --git a/tests/run-debuginfod-duplicate-urls.sh b/tests/run-debuginfod-duplicate-urls.sh index b76b39a3..50e39cb2 100755 --- a/tests/run-debuginfod-duplicate-urls.sh +++ b/tests/run-debuginfod-duplicate-urls.sh @@ -26,6 +26,10 @@ unset VALGRIND_CMD base=8600 get_ports +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod \ $VERBOSE -F -p $PORT1 -t0 -g0 -d ${DB} F > vlog$PORT1 2>&1 & PID1=$! diff --git a/tests/run-debuginfod-extraction.sh b/tests/run-debuginfod-extraction.sh index 7c534d09..e984f32f 100755 --- a/tests/run-debuginfod-extraction.sh +++ b/tests/run-debuginfod-extraction.sh @@ -28,7 +28,11 @@ mkdir R Z base=8700 get_ports -env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -R -Z .tar.xz -Z .tar.bz2=bzcat -p $PORT1 -t0 -g0 -v R Z > vlog$PORT1 2>&1 & +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + +env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d $DB -F -R -Z .tar.xz -Z .tar.bz2=bzcat -p $PORT1 -t0 -g0 -v R Z > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 errfiles vlog$PORT1 diff --git a/tests/run-debuginfod-fd-prefetch-caches.sh b/tests/run-debuginfod-fd-prefetch-caches.sh index 08b32923..61fee9e9 100755 --- a/tests/run-debuginfod-fd-prefetch-caches.sh +++ b/tests/run-debuginfod-fd-prefetch-caches.sh @@ -31,8 +31,12 @@ PREFETCH_MBS=100 base=8800 get_ports +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + echo $PORT1 -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -p $PORT1 \ +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -p $PORT1 -d $DB \ --fdcache-mbs=$FDCACHE_MDS --fdcache-fds=$FDCACHE_FDS --fdcache-prefetch-mbs=$PREFETCH_MBS \ --fdcache-prefetch-fds=$PREFETCH_FDS --fdcache-mintmp 0 -v -F F > vlog$PORT1 2>&1 & PID1=$! diff --git a/tests/run-debuginfod-malformed.sh b/tests/run-debuginfod-malformed.sh index 78b3b7fc..eb504181 100755 --- a/tests/run-debuginfod-malformed.sh +++ b/tests/run-debuginfod-malformed.sh @@ -28,11 +28,12 @@ base=9200 # find an unused port number get_ports DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache mkdir F mkdir -p $DEBUGINFOD_CACHE_PATH -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 \ +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -d $DB \ -t0 -g0 -v F > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 diff --git a/tests/run-debuginfod-sizetime.sh b/tests/run-debuginfod-sizetime.sh index 068b548b..2cf6f252 100755 --- a/tests/run-debuginfod-sizetime.sh +++ b/tests/run-debuginfod-sizetime.sh @@ -25,6 +25,10 @@ unset VALGRIND_CMD base=9600 get_ports +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + echo "int main() { return 0; }" > ${PWD}/prog.c # Create a subdirectory to confound source path names mkdir foobar @@ -34,7 +38,7 @@ tempfiles prog prog.debug prog.c BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \ -a prog | grep 'Build ID' | cut -d ' ' -f 7` -env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -t0 -g0 ${PWD} > vlog$PORT1 2>&1 & +env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -d $DB -t0 -g0 ${PWD} > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 errfiles vlog$PORT1 diff --git a/tests/run-debuginfod-tmp-home.sh b/tests/run-debuginfod-tmp-home.sh index 81986198..25b89be4 100755 --- a/tests/run-debuginfod-tmp-home.sh +++ b/tests/run-debuginfod-tmp-home.sh @@ -23,6 +23,7 @@ set -x unset VALGRIND_CMD DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache mkdir F @@ -32,7 +33,7 @@ mkdir -p $DEBUGINFOD_CACHE_PATH base=9700 get_ports -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 \ +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -d $DB \ -t0 -g0 -v F > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 diff --git a/tests/run-debuginfod-writable.sh b/tests/run-debuginfod-writable.sh index 440a5666..0bc70379 100755 --- a/tests/run-debuginfod-writable.sh +++ b/tests/run-debuginfod-writable.sh @@ -28,7 +28,11 @@ mkdir F base=9800 get_ports -env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 \ +DB=${PWD}/.debuginfod_tmp.sqlite +tempfiles $DB +export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache + +env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -p $PORT1 -d $DB \ -t0 -g0 -v F > vlog$PORT1 2>&1 & PID1=$! tempfiles vlog$PORT1 -- 2.32.0 --im3y/YpaazTM5HXr--