Buildbot failure in Wildebeest Builder on whole buildset

Buildbot failure in Wildebeest Builder on whole buildset

[buildbot]

The Buildbot has detected a failed build on builder whole buildset while building elfutils.
Full details are available at:
    https://builder.wildebeest.org/buildbot/#builders/3/builds/422

Buildbot URL: https://builder.wildebeest.org/buildbot/

Worker for this Build: fedora-x86_64

Build Reason: <unknown>
Blamelist: Frank Ch. Eigler <fche@redhat.com>

BUILD FAILED: failed test (failure)

Sincerely,
 -The BuildbotThe Buildbot has detected a failed build on builder whole buildset while building elfutils.
Full details are available at:
    https://builder.wildebeest.org/buildbot/#builders/10/builds/390

Buildbot URL: https://builder.wildebeest.org/buildbot/

Worker for this Build: fedora-s390x

Build Reason: <unknown>
Blamelist: Frank Ch. Eigler <fche@redhat.com>

BUILD FAILED: failed test (failure)

Sincerely,
 -The Buildbot

Slow valgrind curl hostname lookups (Was: Buildbot failure in Wildebeest Builder on whole buildset)

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1029 bytes --]

Hi,

On Sat, 2019-11-23 at 01:38 +0000, buildbot@builder.wildebeest.org wrote:
> The Buildbot has detected a failed build on builder whole buildset while building elfutils.
> Full details are available at:
>     https://builder.wildebeest.org/buildbot/#builders/3/builds/422
> 
> Buildbot URL: https://builder.wildebeest.org/buildbot/
> 
> Worker for this Build: fedora-x86_64
> 
> Build Reason: <unknown>
> Blamelist: Frank Ch. Eigler <fche@redhat.com>
> 
> BUILD FAILED: failed test (failure)

The problem here was the run-debuginfod-find.sh testcase timeing out
under valgrind. The way curl does hostname lookups in multi_curl mode
seems to take minutes under valgrind (it keeps retrying). But the only
lookups in the testcase are for localhost. By simply replacing
localhost with 127.0.0.1 the time to run this testcase under valgrind
locally goes from 10 minutes down to 3 minutes.

I have committed the patch below and will monitor whether all buildbot
workers will now succeed.

Cheers,

Mark

[-- Attachment #2: 0001-tests-Use-127.0.0.1-in-run-debuginfod-find.sh-instea.patch --]
[-- Type: text/x-patch, Size: 3830 bytes --]

From 1e651ad457f445ff933b4fdd7e2c82bd1fc0f2cd Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sat, 23 Nov 2019 14:09:42 +0100
Subject: [PATCH] tests: Use 127.0.0.1 in run-debuginfod-find.sh instead of
 localhost.

The way curl does hostname lookups in multi_curl mode takes a very long
time under valgrind. Replace all localhosti hostnames with 127.0.0.1 so
no lookups are necessary.

Signed-off-by: Mark Wielaard <mark@klomp.org>
---
 tests/ChangeLog              |  4 ++++
 tests/run-debuginfod-find.sh | 20 ++++++++++----------
 2 files changed, 14 insertions(+), 10 deletions(-)

diff --git a/tests/ChangeLog b/tests/ChangeLog
index fbf1d252..8142a0f6 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,7 @@
+2019-11-23  Mark Wielaard  <mark@klomp.org>
+
+	* run-debuginfod-find.sh: Replace all localhost with 127.0.0.1.
+
 2019-11-07  Frank Ch. Eigler  <fche@redhat.com>
 
 	* run-debuginfod-find.sh: Test debuginfod metrics via curl.
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index 5c66029c..ff68c3de 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -48,7 +48,7 @@ ln -s ${abs_builddir}/dwfllines L/foo   # any program not used elsewhere in this
 env DEBUGINFOD_TEST_WEBAPI_SLEEP=3 LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod -F -R -vvvv -d $DB -p $PORT1 -t0 -g0 R F L &
 PID1=$!
 sleep 3
-export DEBUGINFOD_URLS=http://localhost:$PORT1/   # or without trailing /
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT1/   # or without trailing /
 
 # Be patient when run on a busy machine things might take a bit.
 # And under valgrind debuginfod-find is really, really slow.
@@ -202,7 +202,7 @@ tempfiles ${DB}_2
 sleep 3
 
 # have clients contact the new server
-export DEBUGINFOD_URLS=http://localhost:$PORT2
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT2
 rm -rf $DEBUGINFOD_CACHE_PATH
 testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID
 
@@ -211,17 +211,17 @@ BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
          -a L/foo | grep 'Build ID' | cut -d ' ' -f 7`
 file L/foo
 file -L L/foo
-export DEBUGINFOD_URLS=http://localhost:$PORT1
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT1
 rm -rf $DEBUGINFOD_CACHE_PATH
 testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID && false || true
-export DEBUGINFOD_URLS=http://localhost:$PORT2
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT2
 testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID
 
 
 # test parallel queries in client
 export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache3
 mkdir -p $DEBUGINFOD_CACHE_PATH
-export DEBUGINFOD_URLS="BAD http://localhost:$PORT1 localhost:$PORT1 http://localhost:$PORT2 DNE"
+export DEBUGINFOD_URLS="BAD http://127.0.0.1:$PORT1 127.0.0.1:$PORT1 http://127.0.0.1:$PORT2 DNE"
 
 testrun ${abs_builddir}/debuginfod_build_id_find -e F/prog2 1
 
@@ -229,11 +229,11 @@ testrun ${abs_builddir}/debuginfod_build_id_find -e F/prog2 1
 
 # Fetch some metrics, if curl program is installed
 if type curl 2>/dev/null; then
-    curl http://localhost:$PORT1/badapi
-    curl http://localhost:$PORT1/metrics
-    curl http://localhost:$PORT2/metrics
-    curl http://localhost:$PORT1/metrics | grep -q 'http_responses_total.*result.*error'
-    curl http://localhost:$PORT2/metrics | grep -q 'http_responses_total.*result.*upstream'
+    curl http://127.0.0.1:$PORT1/badapi
+    curl http://127.0.0.1:$PORT1/metrics
+    curl http://127.0.0.1:$PORT2/metrics
+    curl http://127.0.0.1:$PORT1/metrics | grep -q 'http_responses_total.*result.*error'
+    curl http://127.0.0.1:$PORT2/metrics | grep -q 'http_responses_total.*result.*upstream'
 fi
 
 ########################################################################
-- 
2.18.1