From: "Frank Ch. Eigler" <fche@redhat.com>
To: elfutils-devel@sourceware.org, amerey@redhat.com
Subject: patch 4 debuginfod: symlink following mode
Date: Thu, 07 Nov 2019 09:07:00 -0000 [thread overview]
Message-ID: <20191107090732.GA19337@redhat.com> (raw)
In-Reply-To: <20191104214823.GA17633@redhat.com>
From: "Frank Ch. Eigler" <fche@redhat.com>
Date: Wed, 6 Nov 2019 18:53:31 -0500
Subject: [PATCH 1/2] debuginfod 4: symbolic link traversal mode
In order to support file/rpm archives that are organized via symlink
trees, add an "-L" option to debuginfod, meaning about the same as for
find(1) or ls(1): to traverse rather than ignore symlinks.
---
debuginfod/ChangeLog | 5 +++++
debuginfod/debuginfod.8 | 15 +++++++++++---
debuginfod/debuginfod.cxx | 15 ++++++++------
tests/ChangeLog | 5 +++++
tests/run-debuginfod-find.sh | 39 ++++++++++++++++++++----------------
5 files changed, 53 insertions(+), 26 deletions(-)
diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 34713746350d..2870b6df9392 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -1,3 +1,8 @@
+2019-11-06 Frank Ch. Eigler <fche@redhat.com>
+
+ * debuginfod.cxx: Add new -L (symlink-following) mode.
+ * debuginfod.8: Document it.
+
2019-11-04 Frank Ch. Eigler <fche@redhat.com>
* debuginfo-client.c (debuginfod_set_progressfn): New function
diff --git a/debuginfod/debuginfod.8 b/debuginfod/debuginfod.8
index 02059a115430..6b45dee68e70 100644
--- a/debuginfod/debuginfod.8
+++ b/debuginfod/debuginfod.8
@@ -35,14 +35,14 @@ debuginfod servers, it queries them for the same information, just as
relays the file content to the original requester.
Each listed PATH creates a thread to scan for matching
-ELF/DWARF/source files under the given directory. Source files are
+ELF/DWARF/source files under the given physical directory. Source files are
matched with DWARF files based on the AT_comp_dir (compilation
directory) attributes inside it. Duplicate directories are ignored.
You may use a file name for a PATH, but source code indexing may be
incomplete; prefer using a directory that contains the binaries.
Each listed PATH also creates a thread to scan for ELF/DWARF/source
-files contained in matching RPMs under the given directory. Duplicate
+files contained in matching RPMs under the given physical directory. Duplicate
directories are ignored. You may use a file name for a PATH, but
source code indexing may be incomplete; prefer using a directory that
contains normal RPMs alongside debuginfo/debugsource RPMs. Because of
@@ -50,7 +50,7 @@ complications such as DWZ-compressed debuginfo, may require \fItwo\fP
scan passes to identify all source code.
If no PATH is listed, then \fBdebuginfod\fP will simply serve content
-that it scanned into its index in previous runs.
+that it scanned into its index in previous runs: the data is cumulative.
File names must match extended regular expressions given by the \-I
option and not the \-X option (if any) in order to be considered.
@@ -133,6 +133,15 @@ concurrently do CPU-intensive operations like parsing an ELF file
or an RPM. The default is the number of processors on the system;
the minimum is 1.
+.TP
+.B "\-L"
+Traverse symbolic links encountered during traversal of the PATHs,
+including across devices - as in \fIfind\ -L\fP. The default is to
+traverse the physical directory structure only, stay on the same
+device, and ignore symlinks - as in \fIfind\ -P\ -xdev\fP. Caution: a
+loops in the symbolic directory tree might lead to \fIinfinite
+traversal\fP.
+
.TP
.B "\-v"
Increase verbosity of logging to the standard error file descriptor.
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index fe8706cf3e07..7c7a0c5d7ef5 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -325,6 +325,7 @@ static const struct argp_option options[] =
// "source-oci-imageregistry" ...
{ NULL, 0, NULL, 0, "Options:", 2 },
+ { "logical", 'L', NULL, 0, "Follow symlinks, default=ignore.", 0 },
{ "rescan-time", 't', "SECONDS", 0, "Number of seconds to wait between rescans, 0=disable.", 0 },
{ "groom-time", 'g', "SECONDS", 0, "Number of seconds to wait between database grooming, 0=disable.", 0 },
{ "maxigroom", 'G', NULL, 0, "Run a complete database groom/shrink pass at startup.", 0 },
@@ -371,6 +372,7 @@ static vector<string> extra_ddl;
static regex_t file_include_regex;
static regex_t file_exclude_regex;
static int test_webapi_sleep; /* testing only */
+static bool traverse_logical;
/* Handle program arguments. */
@@ -388,6 +390,9 @@ parse_opt (int key, char *arg,
break;
case 'F': break; // ignored; blog post compatibility
case 'R': break; // ignored; blog post compatibility
+ case 'L':
+ traverse_logical = true;
+ break;
case 'D': extra_ddl.push_back(string(arg)); break;
case 't':
rescan_s = (unsigned) atoi(arg);
@@ -1459,8 +1464,7 @@ scan_source_file_path (const string& dir)
unsigned fts_scanned=0, fts_regex=0, fts_cached=0, fts_debuginfo=0, fts_executable=0, fts_sourcefiles=0;
FTS *fts = fts_open (dirs,
- FTS_PHYSICAL /* don't follow symlinks */
- | FTS_XDEV /* don't cross devices/mountpoints */
+ (traverse_logical ? FTS_LOGICAL : FTS_PHYSICAL|FTS_XDEV)
| FTS_NOCHDIR /* multithreaded */,
NULL);
if (fts == NULL)
@@ -1655,7 +1659,7 @@ scan_source_file_path (const string& dir)
throw libc_exception(f->fts_errno, string("fts/F traversal ") + string(f->fts_path));
default:
- case FTS_SL: /* NB: don't enter symbolic links into the database */
+ case FTS_SL: /* ignore symlinks; seen in non-L mode only */
break;
}
@@ -1922,8 +1926,7 @@ scan_source_rpm_path (const string& dir)
unsigned fts_executable=0, fts_rpm = 0, fts_sref=0, fts_sdef=0;
FTS *fts = fts_open (dirs,
- FTS_PHYSICAL /* don't follow symlinks */
- | FTS_XDEV /* don't cross devices/mountpoints */
+ (traverse_logical ? FTS_LOGICAL : FTS_PHYSICAL|FTS_XDEV)
| FTS_NOCHDIR /* multithreaded */,
NULL);
if (fts == NULL)
@@ -2056,7 +2059,7 @@ scan_source_rpm_path (const string& dir)
throw libc_exception(f->fts_errno, string("fts/R traversal ") + string(f->fts_path));
default:
- case FTS_SL: /* NB: don't enter symbolic links into the database */
+ case FTS_SL: /* ignore symlinks; seen in non-L mode only */
break;
}
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 7b091158cff2..3d50ee8623ee 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,8 @@
+2019-11-06 Frank Ch. Eigler <fche@redhat.com>
+
+ * run-debuginfod-find.sh: Test debuginfod -L mode. Drop
+ plain debuginfo-find help-output-comparison.
+
2019-11-04 Frank Ch. Eigler <fche@redhat.com>
* run-debuginfod-find.sh: Test debuginfod-find -v progress mode.
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index fe38eaac9c77..644901073d75 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -22,10 +22,11 @@ set -x
testfiles testfile-debuginfod-0.rpm testfile-debuginfod-1.rpm testfile-debuginfod-2.rpm
DB=${PWD}/.debuginfod_tmp.sqlite
+tempfiles $DB
export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache
# clean up trash if we were aborted early
-trap 'set +e; kill $PID1 $PID2; rm -rf F R ${PWD}/.client_cache*; exit_cleanup; exit 0' 0 1 2 3 5 9 15
+trap 'kill $PID1 $PID2 || true; rm -rf F R L ${PWD}/.client_cache*; exit_cleanup' 0 1 2 3 5 9 15
# find an unused port number
while true; do
@@ -42,10 +43,11 @@ done
# So we gather the LD_LIBRARY_PATH with this cunning trick:
ldpath=`testrun sh -c 'echo $LD_LIBRARY_PATH'`
-mkdir F R
-tempfiles F R
-env DEBUGINFOD_TEST_WEBAPI_SLEEP=3 LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod -vvvv -d $DB \
--p $PORT1 -t0 -g0 R F &
+mkdir F R L
+tempfiles F R L
+ln -s ${abs_builddir}/dwfllines L/foo # any program not used elsewhere in this test
+
+env DEBUGINFOD_TEST_WEBAPI_SLEEP=3 LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod -vvvv -d $DB -p $PORT1 -t0 -g0 R F L &
PID1=$!
sleep 3
export DEBUGINFOD_URLS=http://localhost:$PORT1/ # or without trailing /
@@ -171,14 +173,28 @@ done
export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache2
mkdir -p $DEBUGINFOD_CACHE_PATH
# NB: inherits the DEBUGINFOD_URLS to the first server
-env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod -d ${DB}_2 -p $PORT2 &
+# NB: run in -L symlink-following mode for the L subdir
+env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod -vvvv -d ${DB}_2 -p $PORT2 -L L &
PID2=$!
+tempfiles ${DB}_2
sleep 3
# have clients contact the new server
export DEBUGINFOD_URLS=http://localhost:$PORT2
testrun ${abs_builddir}/debuginfod_build_id_find -e F/prog 1
+# confirm that first server can't resolve symlinked info in L/ but second can
+BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
+ -a L/foo | grep 'Build ID' | cut -d ' ' -f 7`
+file L/foo
+file -L L/foo
+export DEBUGINFOD_URLS=http://localhost:$PORT1
+rm -rf $DEBUGINFOD_CACHE_PATH
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID && false || true
+export DEBUGINFOD_URLS=http://localhost:$PORT2
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID
+
+
# test parallel queries in client
export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache3
mkdir -p $DEBUGINFOD_CACHE_PATH
@@ -209,15 +225,4 @@ testrun ${abs_builddir}/debuginfod_build_id_find -e F/prog 1
testrun ${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $BUILDID2 && false || true
-########################################################################
-
-# Ensure debuginfod-find can be safely called with no arguments.
-# Use a relative path to prevent automatic line breaks in the output
-# due to excessive characters.
-testrun_compare ../../debuginfod/debuginfod-find <<EOF
-Usage: ../../debuginfod/debuginfod-find debuginfo BUILDID
- or: ../../debuginfod/debuginfod-find executable BUILDID
- or: ../../debuginfod/debuginfod-find source BUILDID /FILENAME
-EOF
-
exit 0
--
2.21.0
next prev parent reply other threads:[~2019-11-07 9:07 UTC|newest]
Thread overview: 78+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-28 19:04 patch 0/2 debuginfod submission Frank Ch. Eigler
2019-10-28 19:06 ` patch 1/2 debuginfod client Frank Ch. Eigler
2019-10-28 19:09 ` patch 2/2 debuginfod server etc Frank Ch. Eigler
2019-11-04 21:48 ` patch 3/3 debuginfod client interruptability Frank Ch. Eigler
2019-11-07 9:07 ` Frank Ch. Eigler [this message]
2019-11-07 9:08 ` patch 5 debuginfod: prometheus metrics Frank Ch. Eigler
2019-11-15 17:26 ` Mark Wielaard
2019-11-15 17:58 ` Frank Ch. Eigler
2019-11-18 16:20 ` Mark Wielaard
2019-11-18 16:48 ` Frank Ch. Eigler
2019-11-19 16:13 ` Mark Wielaard
2019-11-15 16:49 ` patch 4 debuginfod: symlink following mode Mark Wielaard
2019-11-15 18:31 ` Frank Ch. Eigler
2019-11-18 16:27 ` Mark Wielaard
2019-11-15 16:16 ` patch 3/3 debuginfod client interruptability Mark Wielaard
2019-11-15 17:03 ` Aaron Merey
2019-11-15 17:35 ` Mark Wielaard
2019-11-15 18:14 ` Pedro Alves
2019-11-17 23:44 ` Mark Wielaard
2019-11-18 2:50 ` Frank Ch. Eigler
2019-11-18 9:24 ` Pedro Alves
2019-11-19 12:58 ` Mark Wielaard
2019-11-13 17:22 ` patch 2/2 debuginfod server etc Mark Wielaard
2019-11-14 11:54 ` Frank Ch. Eigler
2019-11-16 1:31 ` Mark Wielaard
2019-11-13 23:19 ` Mark Wielaard
2019-11-14 12:30 ` Frank Ch. Eigler
2019-11-18 14:17 ` Mark Wielaard
2019-11-18 18:41 ` Frank Ch. Eigler
2019-11-19 15:41 ` Mark Wielaard
2019-11-19 16:13 ` Frank Ch. Eigler
2019-11-19 20:11 ` Mark Wielaard
2019-11-19 21:15 ` Frank Ch. Eigler
2019-11-20 11:53 ` Mark Wielaard
2019-11-20 12:29 ` Frank Ch. Eigler
2019-11-21 14:16 ` Mark Wielaard
2019-11-21 15:40 ` Mark Wielaard
2019-11-21 16:01 ` Frank Ch. Eigler
2019-11-21 15:58 ` Frank Ch. Eigler
2019-11-21 16:37 ` Mark Wielaard
2019-11-21 17:18 ` Frank Ch. Eigler
2019-11-21 20:42 ` Mark Wielaard
2019-11-22 12:08 ` Mark Wielaard
2019-11-14 20:45 ` Mark Wielaard
2019-11-15 11:03 ` Mark Wielaard
2019-11-15 21:00 ` Frank Ch. Eigler
2019-11-18 15:01 ` Mark Wielaard
2019-11-15 14:40 ` Mark Wielaard
2019-11-15 19:54 ` Frank Ch. Eigler
2019-11-18 15:31 ` Mark Wielaard
2019-11-18 15:49 ` Frank Ch. Eigler
2019-11-12 11:12 ` patch 1/2 debuginfod client Mark Wielaard
2019-11-12 15:14 ` Frank Ch. Eigler
2019-11-12 21:59 ` Mark Wielaard
2019-11-14 0:33 ` Frank Ch. Eigler
2019-11-15 21:33 ` Mark Wielaard
2019-11-12 21:25 ` Mark Wielaard
2019-11-13 23:25 ` Frank Ch. Eigler
2019-11-16 0:46 ` Mark Wielaard
2019-11-16 18:53 ` Frank Ch. Eigler
2019-11-18 17:17 ` Mark Wielaard
2019-11-18 20:33 ` Frank Ch. Eigler
2019-11-19 15:57 ` Mark Wielaard
2019-11-19 16:20 ` Frank Ch. Eigler
2019-11-19 20:16 ` Mark Wielaard
2019-11-19 21:22 ` Frank Ch. Eigler
2019-11-20 12:50 ` Mark Wielaard
2019-11-20 13:30 ` Frank Ch. Eigler
2019-11-21 14:02 ` Mark Wielaard
2019-11-13 13:57 ` Mark Wielaard
2019-11-14 11:24 ` Frank Ch. Eigler
2019-11-16 0:52 ` Mark Wielaard
2019-11-16 2:28 ` Frank Ch. Eigler
2019-10-30 11:04 ` patch 0/2 debuginfod submission Mark Wielaard
2019-10-30 13:40 ` Frank Ch. Eigler
2019-10-30 14:12 ` Mark Wielaard
2019-10-30 18:11 ` Frank Ch. Eigler
2019-10-31 11:18 ` Mark Wielaard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191107090732.GA19337@redhat.com \
--to=fche@redhat.com \
--cc=amerey@redhat.com \
--cc=elfutils-devel@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).