Buildbot failure in Wildebeest Builder on whole buildset

Buildbot failure in Wildebeest Builder on whole buildset

[buildbot]

The Buildbot has detected a new failure on builder elfutils-debian-amd64 while building elfutils.
Full details are available at:
    https://builder.wildebeest.org/buildbot/#builders/2/builds/803

Buildbot URL: https://builder.wildebeest.org/buildbot/

Worker for this Build: debian-amd64

Build Reason: <unknown>
Blamelist: Di Chen <dichen@redhat.com>

BUILD FAILED: failed test (failure)

Sincerely,
 -The BuildbotThe Buildbot has detected a new failure on builder elfutils-fedora-x86_64 while building elfutils.
Full details are available at:
    https://builder.wildebeest.org/buildbot/#builders/3/builds/807

Buildbot URL: https://builder.wildebeest.org/buildbot/

Worker for this Build: fedora-x86_64

Build Reason: <unknown>
Blamelist: Di Chen <dichen@redhat.com>

BUILD FAILED: failed test (failure)

Sincerely,
 -The BuildbotThe Buildbot has detected a new failure on builder elfutils-debian-arm64 while building elfutils.
Full details are available at:
    https://builder.wildebeest.org/buildbot/#builders/45/builds/239

Buildbot URL: https://builder.wildebeest.org/buildbot/

Worker for this Build: debian-arm64

Build Reason: <unknown>
Blamelist: Di Chen <dichen@redhat.com>

BUILD FAILED: failed test (failure)

Sincerely,
 -The Buildbot

Re: Buildbot failure in Wildebeest Builder on whole buildset

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]

Hi,

On Fri, Aug 27, 2021 at 11:19:14PM +0000, buildbot@builder.wildebeest.org wrote:
> The Buildbot has detected a new failure on builder elfutils-debian-amd64 while building elfutils.
> Full details are available at:
>     https://builder.wildebeest.org/buildbot/#builders/2/builds/803
> 
> Buildbot URL: https://builder.wildebeest.org/buildbot/
> 
> Worker for this Build: debian-amd64
> 
> Build Reason: <unknown>
> Blamelist: Di Chen <dichen@redhat.com>
> 
> BUILD FAILED: failed test (failure)

This took me a while to figure out because it does not consistently
fail.  It turns out that sharing a database between two debuginfod
instances (especially if they forward queries to each other) is not a
good idea.

Hopefully fixed with the attached patch.

At first I had hoped that using -d :memory: would work. But debuginfod
doesn't really support in memory databases like that. It opens the
database twice in read/write and read-only mode. But :memory: is
special and isn't always a new database, so the read-only copy doesn't
see what is in the read/write copy.

Cheers,

Mark

[-- Attachment #2: 0001-tests-Use-fresh-separate-databases-for-debuginfd-for.patch --]
[-- Type: text/x-diff, Size: 2123 bytes --]

From 17a9b1303e533c13aac6550844bdd68c669091bf Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sat, 28 Aug 2021 15:54:18 +0200
Subject: [PATCH] tests: Use fresh separate databases for debuginfd
 forwarded-ttl-limit

Sharing the database between the two debuginfod instances that forward
queries to each other causes issues. Make both debuginfod instances
use a new fresh database.

Signed-off-by: Mark Wielaard <mark@klomp.org>
---
 tests/ChangeLog              | 5 +++++
 tests/run-debuginfod-find.sh | 7 +++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/tests/ChangeLog b/tests/ChangeLog
index 29c48b97..cbd1c227 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,8 @@
+2021-08-28  Mark Wielaard  <mark@klomp.org>
+
+	* run-debuginfod-find.sh: Use clean, separate databases for
+	forwarded-ttl-limit tests.
+
 2021-08-20  Di Chen  <dichen@redhat.com>
 
 	* run-debuginfod-find.sh: Add test for X-Forwarded-For hops limit.
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index 7e12dd7f..5d38d625 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -824,10 +824,13 @@ done
 tempfiles vlog$PORT4 vlog$PORT5
 errfiles vlog$PORT4 vlog$PORT5
 
-env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT5 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE --forwarded-ttl-limit 0 -p $PORT4 > vlog$PORT4 2>&1 &
+# Give each debuginfd its own clean database.
+tempfiles db.$PORT4.sql db.$PORT5.sql
+
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT5 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d db.$PORT4.sql --forwarded-ttl-limit 0 -p $PORT4 > vlog$PORT4 2>&1 &
 PID5=$!
 
-env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT4 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE --forwarded-ttl-limit 1 -p $PORT5 > vlog$PORT5 2>&1 &
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT4 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d db.$PORT5.sql --forwarded-ttl-limit 1 -p $PORT5 > vlog$PORT5 2>&1 &
 PID6=$!
 
 wait_ready $PORT4 'ready' 1
-- 
2.32.0

Re: Buildbot failure in Wildebeest Builder on whole buildset

[Frank Ch. Eigler]

Hi -

> This took me a while to figure out because it does not consistently
> fail.  It turns out that sharing a database between two debuginfod
> instances (especially if they forward queries to each other) is not a
> good idea.

That's correct.  Each test case invocation of debuginfod should be run
with its own -d $database.

> Hopefully fixed with the attached patch.
> At first I had hoped that using -d :memory: would work. But debuginfod
> doesn't really support in memory databases like that. [...]

Yes, that functionality was broken at the point you identifed.  We could
bring it back by hacking debuginfod's command line parser to map
    -d :memory:
to  -d file::memory:?cache=shared

(https://sqlite.org/inmemorydb.html)

- FChE

Using in-memory database (Was: Buildbot failure in Wildebeest Builder on whole buildset)

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

Hi Frank,

On Sat, Aug 28, 2021 at 10:41:30AM -0400, Frank Ch. Eigler via Elfutils-devel wrote:
> > At first I had hoped that using -d :memory: would work. But debuginfod
> > doesn't really support in memory databases like that. [...]
> 
> Yes, that functionality was broken at the point you identifed.  We could
> bring it back by hacking debuginfod's command line parser to map
>     -d :memory:
> to  -d file::memory:?cache=shared
> 
> (https://sqlite.org/inmemorydb.html)

That is really cool and imho really useful for quick testing of debuginfod.
What do you think of the attached patch to implement this?

Thanks,

Mark

[-- Attachment #2: 0001-debuginfod-Turn-d-memory-into-d-file-memory-cache-sh.patch --]
[-- Type: text/x-diff, Size: 4893 bytes --]

From 61554a292f753032490c349fedf47c4fb461e3ea Mon Sep 17 00:00:00 2001
From: Mark Wielaard <mark@klomp.org>
Date: Sat, 28 Aug 2021 20:25:56 +0200
Subject: [PATCH] debuginfod: Turn -d ":memory:" into -d
 "file::memory:?cache=shared"

debuginfod opens the database twice, once in read/wrote and once in
read-only mode. This means the magic ":memory:" in-memory database
cannot be used as is because the two connections don't really share
the underlying database. Fix this by turning ":memory:" into
":file::memory:?cache=shared" which makes the in-memory database
shared. See https://sqlite.org/inmemorydb.html

Document this in debuginfod.8 and make some tests use -d :memory:

Signed-off-by: Mark Wielaard <mark@klomp.org>
---
 debuginfod/ChangeLog         | 5 +++++
 debuginfod/debuginfod.cxx    | 9 ++++++++-
 doc/ChangeLog                | 4 ++++
 doc/debuginfod.8             | 6 ++++--
 tests/ChangeLog              | 5 +++++
 tests/run-debuginfod-find.sh | 7 ++-----
 6 files changed, 28 insertions(+), 8 deletions(-)

diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 395af94f..c5459823 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -1,3 +1,8 @@
+2021-08-28  Mark Wielaard  <mjw@redhat.com>
+
+	* debuginfod.cxx (parse_opt): Turn the -d arg ":memory:" into
+	"file::memory:?cache=shared" for the db_path.
+
 2021-08-20  Di Chen  <dichen@redhat.com>
 
 	* debuginfod.cxx (options): Add ARGP_KEY_FORWARDED_TTL_LIMIT.
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 6e182a84..3269f657 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -483,7 +483,14 @@ parse_opt (int key, char *arg,
   switch (key)
     {
     case 'v': verbose ++; break;
-    case 'd': db_path = string(arg); break;
+    case 'd':
+      /* When using the in-memory database make sure it is shareable,
+	 so we can open it twice as read/write and read-only.  */
+      if (strcmp (arg, ":memory:") == 0)
+	db_path = "file::memory:?cache=shared";
+      else
+	db_path = string(arg);
+      break;
     case 'p': http_port = (unsigned) atoi(arg);
       if (http_port == 0 || http_port > 65535)
         argp_failure(state, 1, EINVAL, "port number");
diff --git a/doc/ChangeLog b/doc/ChangeLog
index d5f34f0f..ada48383 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,7 @@
+2021-08-28  Di Chen  <dichen@redhat.com>
+
+	* debuginfod.8 (-d): Document ":memory:" as in-memory database.
+
 2021-08-20  Di Chen  <dichen@redhat.com>
 
 	* debuginfod.8: Add --forwarded-ttl-limit=NUM documentation.
diff --git a/doc/debuginfod.8 b/doc/debuginfod.8
index 5b0d793c..f9a418d1 100644
--- a/doc/debuginfod.8
+++ b/doc/debuginfod.8
@@ -117,8 +117,10 @@ file is disposable in the sense that a later rescan will repopulate
 data.  It will contain absolute file path names, so it may not be
 portable across machines.  It may be frequently read/written, so it
 should be on a fast filesystem.  It should not be shared across
-machines or users, to maximize sqlite locking performance.  The
-default database file is \%$HOME/.debuginfod.sqlite.
+machines or users, to maximize sqlite locking performance.  For quick
+testing the magic string ":memory:" can be used to use an one-time
+memory-only database.  The default database file is
+\%$HOME/.debuginfod.sqlite.
 
 .TP
 .B "\-D SQL" "\-\-ddl=SQL"
diff --git a/tests/ChangeLog b/tests/ChangeLog
index cbd1c227..3fa7c2e4 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,8 @@
+2021-08-28  Mark Wielaard  <mark@klomp.org>
+
+	* run-debuginfod-find.sh: Use ":memory:" for the
+	forwarded-ttl-limit tests.
+
 2021-08-28  Mark Wielaard  <mark@klomp.org>
 
 	* run-debuginfod-find.sh: Use clean, separate databases for
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index 5d38d625..7515b7cd 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -824,13 +824,10 @@ done
 tempfiles vlog$PORT4 vlog$PORT5
 errfiles vlog$PORT4 vlog$PORT5
 
-# Give each debuginfd its own clean database.
-tempfiles db.$PORT4.sql db.$PORT5.sql
-
-env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT5 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d db.$PORT4.sql --forwarded-ttl-limit 0 -p $PORT4 > vlog$PORT4 2>&1 &
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT5 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d :memory: --forwarded-ttl-limit 0 -p $PORT4 > vlog$PORT4 2>&1 &
 PID5=$!
 
-env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT4 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d db.$PORT5.sql --forwarded-ttl-limit 1 -p $PORT5 > vlog$PORT5 2>&1 &
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS=http://127.0.0.1:$PORT4 ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -d :memory: --forwarded-ttl-limit 1 -p $PORT5 > vlog$PORT5 2>&1 &
 PID6=$!
 
 wait_ready $PORT4 'ready' 1
-- 
2.32.0

Re: Using in-memory database (Was: Buildbot failure in Wildebeest Builder on whole buildset)

[Frank Ch. Eigler]

Hi -

> That is really cool and imho really useful for quick testing of debuginfod.
> What do you think of the attached patch to implement this?

chef's kiss

- FChE