From: Aaron Merey <amerey@redhat.com>
To: elfutils-devel@sourceware.org
Cc: Aaron Merey <amerey@redhat.com>
Subject: [PATCH] debuginfod: Support queries for ELF/DWARF sections.
Date: Tue, 27 Sep 2022 22:10:52 -0400 [thread overview]
Message-ID: <20220928021052.315981-1-amerey@redhat.com> (raw)
This patch adds a new function debuginfod_find_section which queries
debuginfod servers for the binary contents of the specified ELF/DWARF
section in a file matching the given build-id.
In order to distinguish between debuginfo and executable files with the
same build-id, this function includes a bool parameter use_debuginfo.
If true, attempt to retrieve the section from the debuginfo file with
the given build-id. If false, use the executable instead.
If the server can find the section in the parent file, the section's
binary contents are written to a temporary file. This file is interned
by the server's fdcache as if it was extracted from an archive.
Modify the webapi to include "debuginfo-section" and
"executable-section" artifacttypes. This helps prevent debuginfod
servers built without _find_section() support from responding with the
entire debuginfo/executable when a section was requested. The section
name is included in the url like source paths are for _find_source()
queries.
One of the primary use cases for this feature is so that tools can
aquire indecies such as .gdb_index without having to download entire
files. Although this patch does not implement it, we could generate
.gdb_index on-the-fly if the target file does not contain it. However
for large debuginfo files generating the index can take a non-trivial
amount of time (ex. about 25 seconds for a 2.5GB debuginfo file).
---
ChangeLog | 4 +
NEWS | 2 +
debuginfod/ChangeLog | 19 ++
debuginfod/debuginfod-client.c | 41 +++-
debuginfod/debuginfod-find.c | 39 +++-
debuginfod/debuginfod.cxx | 388 ++++++++++++++++++++++++++------
debuginfod/debuginfod.h.in | 9 +
debuginfod/libdebuginfod.map | 1 +
doc/ChangeLog | 6 +
doc/Makefile.am | 1 +
doc/debuginfod_find_debuginfo.3 | 30 ++-
doc/debuginfod_find_section.3 | 1 +
tests/ChangeLog | 5 +
tests/Makefile.am | 4 +-
tests/run-debuginfod-section.sh | 124 ++++++++++
15 files changed, 590 insertions(+), 84 deletions(-)
create mode 100644 doc/debuginfod_find_section.3
create mode 100755 tests/run-debuginfod-section.sh
diff --git a/ChangeLog b/ChangeLog
index 1f449d60..a1e37f88 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2022-09-27 Aaron Merey <amerey@redhat.com>
+
+ * NEWS: Add debuginfod_find_section.
+
2022-09-27 Taketo Kabe <kabe@sra-tohoku.co.jp>
* debuginfod/debuginfod-client.c: Correctly get timestamp when
diff --git a/NEWS b/NEWS
index 156f78df..974ce920 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,8 @@ readelf: Add -D, --use-dynamic option.
debuginfod: Add --disable-source-scan option.
+debuginfod-client: Add new function debuginfod_find_section.
+
libdwfl: Add new function dwfl_get_debuginfod_client.
Add new function dwfl_frame_reg.
diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 680720ff..49bdf250 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -1,3 +1,22 @@
+2022-09-27 Aaron Merey <amerey@redhat.com>
+
+ * debuginfod-client.c (debuginfod_find_section): New function.
+ (debuginfod_query_server): Add support for section queries.
+ * debuginfod-find.c (main): Add support for section queries.
+ * debuginfod.cxx (extract_section_to_fd): New function.
+ (handle_buildid_f_match): Add section parameter. When non-empty,
+ try to create response from section contents.
+ (handle_buildid_r_match): Add section parameter. When non-empty,
+ try to create response from section contents.
+ (handle_buildid_match): Add section parameter. Pass to
+ handle_buildid_{f,r}_match.
+ (handle_buildid): Handle section name when artifacttype is set to
+ "debuginfo-section" or "executable-section". Query upstream servers
+ via debuginfod_find_section when necessary.
+ (debuginfod.h.in): Include stdbool.h. Add declaration for
+ debuginfod_find_section.
+ (libdebuginfod.map): Add debuginfod_find_section.
+
2022-09-08 Frank Ch. Eigler <fche@redhat.com>
* debuginfod-client.c (debuginfod_query_server): Clear
diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c
index 28ad04c0..9795f6b1 100644
--- a/debuginfod/debuginfod-client.c
+++ b/debuginfod/debuginfod-client.c
@@ -55,6 +55,9 @@ int debuginfod_find_executable (debuginfod_client *c, const unsigned char *b,
int s, char **p) { return -ENOSYS; }
int debuginfod_find_source (debuginfod_client *c, const unsigned char *b,
int s, const char *f, char **p) { return -ENOSYS; }
+int debuginfod_find_section (debuginfod_client *c, const unsigned char *b,
+ int s, const char *scn, bool u,
+ char **p) { return -ENOSYS; }
void debuginfod_set_progressfn(debuginfod_client *c,
debuginfod_progressfn_t fn) { }
void debuginfod_set_verbose_fd(debuginfod_client *c, int fd) { }
@@ -546,11 +549,13 @@ debuginfod_query_server (debuginfod_client *c,
const unsigned char *build_id,
int build_id_len,
const char *type,
- const char *filename,
+ const char *type_arg,
char **path)
{
char *server_urls;
char *urls_envvar;
+ const char *section = NULL;
+ const char *filename = NULL;
char *cache_path = NULL;
char *maxage_path = NULL;
char *interval_path = NULL;
@@ -563,9 +568,21 @@ debuginfod_query_server (debuginfod_client *c,
int vfd = c->verbose_fd;
int rc;
+ if (strcmp (type, "source") == 0)
+ filename = type_arg;
+ else
+ {
+ section = type_arg;
+ if (section != NULL && section[0] != '.')
+ return -EINVAL;
+ }
+
if (vfd >= 0)
{
- dprintf (vfd, "debuginfod_find_%s ", type);
+ if (section != NULL)
+ dprintf (vfd, "debuginfod_find_section");
+ else
+ dprintf (vfd, "debuginfod_find_%s ", type);
if (build_id_len == 0) /* expect clean hexadecimal */
dprintf (vfd, "%s", (const char *) build_id);
else
@@ -681,6 +698,8 @@ debuginfod_query_server (debuginfod_client *c,
PATH_MAX and truncate/collide. Oh well, that'll teach
them! */
}
+ else if (section != NULL)
+ strncpy (suffix, section, PATH_MAX + 1);
else
suffix[0] = '\0';
@@ -1008,6 +1027,9 @@ debuginfod_query_server (debuginfod_client *c,
snprintf(data[i].url, PATH_MAX, "%s/%s/%s/%s", server_url,
build_id_bytes, type, escaped_string);
}
+ else if (section)
+ snprintf(data[i].url, PATH_MAX, "%s/%s/%s-section/%s", server_url,
+ build_id_bytes, type, section);
else
snprintf(data[i].url, PATH_MAX, "%s/%s/%s", server_url, build_id_bytes, type);
if (vfd >= 0)
@@ -1604,6 +1626,21 @@ int debuginfod_find_source(debuginfod_client *client,
"source", filename, path);
}
+int
+debuginfod_find_section (debuginfod_client *client,
+ const unsigned char *build_id, int build_id_len,
+ const char *section, bool from_debuginfo, char **path)
+{
+ int rc;
+
+ if (from_debuginfo)
+ rc = debuginfod_query_server(client, build_id, build_id_len,
+ "debuginfo", section, path);
+ else
+ rc = debuginfod_query_server(client, build_id, build_id_len,
+ "executable", section, path);
+ return rc;
+}
/* Add an outgoing HTTP header. */
int debuginfod_add_http_header (debuginfod_client *client, const char* header)
diff --git a/debuginfod/debuginfod-find.c b/debuginfod/debuginfod-find.c
index 778fb09b..74e5f001 100644
--- a/debuginfod/debuginfod-find.c
+++ b/debuginfod/debuginfod-find.c
@@ -190,13 +190,40 @@ main(int argc, char** argv)
debuginfod_find_* function. If FILETYPE is "source"
then ensure a FILENAME was also supplied as an argument. */
if (strcmp(argv[remaining], "debuginfo") == 0)
- rc = debuginfod_find_debuginfo(client,
- build_id, build_id_len,
- &cache_name);
+ {
+ if (remaining+2 < argc)
+ {
+ if (argv[remaining+2][0] != '.')
+ {
+ fprintf(stderr, "Section names must start with '.'\n");
+ return 1;
+ }
+ rc = debuginfod_find_section(client, build_id, build_id_len,
+ argv[remaining+2], true,
+ &cache_name);
+ }
+ else
+ rc = debuginfod_find_debuginfo(client, build_id,
+ build_id_len, &cache_name);
+ }
else if (strcmp(argv[remaining], "executable") == 0)
- rc = debuginfod_find_executable(client,
- build_id, build_id_len,
- &cache_name);
+ {
+ if (remaining+2 < argc)
+ {
+ if (argv[remaining+2][0] != '.')
+ {
+ fprintf(stderr, "Section names must start with '.'\n");
+ return 1;
+ }
+
+ rc = debuginfod_find_section(client, build_id, build_id_len,
+ argv[remaining+2], false,
+ &cache_name);
+ }
+ else
+ rc = debuginfod_find_executable(client, build_id, build_id_len,
+ &cache_name);
+ }
else if (strcmp(argv[remaining], "source") == 0)
{
if (remaining+2 == argc || argv[remaining+2][0] != '/')
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 000a41c4..512e015a 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -1137,66 +1137,6 @@ add_mhd_last_modified (struct MHD_Response *resp, time_t mtime)
add_mhd_response_header (resp, "Cache-Control", "public");
}
-
-
-static struct MHD_Response*
-handle_buildid_f_match (bool internal_req_t,
- int64_t b_mtime,
- const string& b_source0,
- int *result_fd)
-{
- (void) internal_req_t; // ignored
- int fd = open(b_source0.c_str(), O_RDONLY);
- if (fd < 0)
- throw libc_exception (errno, string("open ") + b_source0);
-
- // NB: use manual close(2) in error case instead of defer_dtor, because
- // in the normal case, we want to hand the fd over to libmicrohttpd for
- // file transfer.
-
- struct stat s;
- int rc = fstat(fd, &s);
- if (rc < 0)
- {
- close(fd);
- throw libc_exception (errno, string("fstat ") + b_source0);
- }
-
- if ((int64_t) s.st_mtime != b_mtime)
- {
- if (verbose)
- obatched(clog) << "mtime mismatch for " << b_source0 << endl;
- close(fd);
- return 0;
- }
-
- inc_metric ("http_responses_total","result","file");
- struct MHD_Response* r = MHD_create_response_from_fd ((uint64_t) s.st_size, fd);
- if (r == 0)
- {
- if (verbose)
- obatched(clog) << "cannot create fd-response for " << b_source0 << endl;
- close(fd);
- }
- else
- {
- std::string file = b_source0.substr(b_source0.find_last_of("/")+1, b_source0.length());
- add_mhd_response_header (r, "Content-Type", "application/octet-stream");
- add_mhd_response_header (r, "X-DEBUGINFOD-SIZE",
- to_string(s.st_size).c_str());
- add_mhd_response_header (r, "X-DEBUGINFOD-FILE", file.c_str());
- add_mhd_last_modified (r, s.st_mtime);
- if (verbose > 1)
- obatched(clog) << "serving file " << b_source0 << endl;
- /* libmicrohttpd will close it. */
- if (result_fd)
- *result_fd = fd;
- }
-
- return r;
-}
-
-
// quote all questionable characters of str for safe passage through a sh -c expansion.
static string
shell_escape(const string& str)
@@ -1590,6 +1530,209 @@ public:
};
static libarchive_fdcache fdcache;
+static int
+extract_section_to_fd (int elf_fd, int64_t parent_mtime,
+ const string& b_source, const string& section)
+{
+ /* Search the fdcache. */
+ struct stat fs;
+ int fd = fdcache.lookup (b_source, section);
+ if (fd >= 0)
+ {
+ if (fstat (fd, &fs) != 0)
+ {
+ if (verbose)
+ obatched (clog) << "cannot fstate fdcache "
+ << b_source << " " << section << endl;
+ close (fd);
+ return -1;
+ }
+ if ((int64_t) fs.st_mtime != parent_mtime)
+ {
+ if (verbose)
+ obatched(clog) << "mtime mismatch for "
+ << b_source << " " << section << endl;
+ close (fd);
+ return -1;
+ }
+ /* Success. */
+ return fd;
+ }
+
+ Elf *elf = elf_begin (elf_fd, ELF_C_READ_MMAP_PRIVATE, NULL);
+ if (elf == NULL)
+ return -1;
+
+ /* Try to find the section and copy the contents into a separate file. */
+ try
+ {
+ size_t shstrndx;
+ int rc = elf_getshdrstrndx (elf, &shstrndx);
+ if (rc < 0)
+ throw elfutils_exception (rc, "getshdrstrndx");
+
+ Elf_Scn *scn = NULL;
+ while (true)
+ {
+ scn = elf_nextscn (elf, scn);
+ if (scn == NULL)
+ break;
+ GElf_Shdr shdr_storage;
+ GElf_Shdr *shdr = gelf_getshdr (scn, &shdr_storage);
+ if (shdr == NULL)
+ break;
+
+ const char *scn_name = elf_strptr (elf, shstrndx, shdr->sh_name);
+ if (scn_name == NULL)
+ break;
+ if (scn_name == section)
+ {
+ Elf_Data *data = NULL;
+
+ /* We found the desired section. */
+ data = elf_getdata (scn, NULL);
+ if (data == NULL)
+ throw elfutils_exception (elf_errno (), "getdata");
+ if (data->d_buf == NULL)
+ {
+ obatched(clog) << "section " << section
+ << " is empty" << endl;
+ break;
+ }
+
+ /* Create temporary file containing the section. */
+ char *tmppath = NULL;
+ rc = asprintf (&tmppath, "%s/debuginfod.XXXXXX", tmpdir.c_str());
+ if (rc < 0)
+ throw libc_exception (ENOMEM, "cannot allocate tmppath");
+ defer_dtor<void*,void> tmmpath_freer (tmppath, free);
+ fd = mkstemp (tmppath);
+ if (fd < 0)
+ throw libc_exception (errno, "cannot create temporary file");
+ ssize_t res = write (fd, data->d_buf, data->d_size);
+ if (res < 0 || (size_t) res != data->d_size)
+ throw libc_exception (errno, "cannot write to temporary file");
+
+ /* Set mtime to be the same as the parent file's mtime. */
+ struct timeval tvs[2];
+ if (fstat (elf_fd, &fs) != 0)
+ throw libc_exception (errno, "cannot fstat file");
+
+ tvs[0].tv_sec = tvs[1].tv_sec = fs.st_mtime;
+ tvs[0].tv_usec = tvs[1].tv_usec = 0;
+ (void) futimes (fd, tvs);
+
+ /* Add to fdcache. */
+ fdcache.intern (b_source, section, tmppath, data->d_size, true);
+ break;
+ }
+ }
+ }
+ catch (const reportable_exception &e)
+ {
+ e.report (clog);
+ close (fd);
+ }
+
+ elf_end (elf);
+ return fd;
+}
+
+static struct MHD_Response*
+handle_buildid_f_match (bool internal_req_t,
+ int64_t b_mtime,
+ const string& b_source0,
+ const string& section,
+ int *result_fd)
+{
+ (void) internal_req_t; // ignored
+ int fd = open(b_source0.c_str(), O_RDONLY);
+ if (fd < 0)
+ throw libc_exception (errno, string("open ") + b_source0);
+
+ // NB: use manual close(2) in error case instead of defer_dtor, because
+ // in the normal case, we want to hand the fd over to libmicrohttpd for
+ // file transfer.
+
+ struct stat s;
+ int rc = fstat(fd, &s);
+ if (rc < 0)
+ {
+ close(fd);
+ throw libc_exception (errno, string("fstat ") + b_source0);
+ }
+
+ if ((int64_t) s.st_mtime != b_mtime)
+ {
+ if (verbose)
+ obatched(clog) << "mtime mismatch for " << b_source0 << endl;
+ close(fd);
+ return 0;
+ }
+
+ if (!section.empty ())
+ {
+ int scn_fd = extract_section_to_fd (fd, s.st_mtime, b_source0, section);
+ close (fd);
+
+ if (scn_fd >= 0)
+ fd = scn_fd;
+ else
+ {
+ if (verbose)
+ obatched (clog) << "cannot find section " << section
+ << " for " << b_source0 << endl;
+ return 0;
+ }
+
+ rc = fstat(fd, &s);
+ if (rc < 0)
+ {
+ close (fd);
+ throw libc_exception (errno, string ("fstat ") + b_source0
+ + string (" ") + section);
+ }
+ }
+
+ struct MHD_Response* r = MHD_create_response_from_fd ((uint64_t) s.st_size, fd);
+ inc_metric ("http_responses_total","result","file");
+ if (r == 0)
+ {
+ if (verbose)
+ {
+ if (!section.empty ())
+ obatched(clog) << "cannot create fd-response for " << b_source0
+ << " section " << section << endl;
+ else
+ obatched(clog) << "cannot create fd-response for " << b_source0 << endl;
+ }
+ close(fd);
+ }
+ else
+ {
+ std::string file = b_source0.substr(b_source0.find_last_of("/")+1, b_source0.length());
+ add_mhd_response_header (r, "Content-Type", "application/octet-stream");
+ add_mhd_response_header (r, "X-DEBUGINFOD-SIZE",
+ to_string(s.st_size).c_str());
+ add_mhd_response_header (r, "X-DEBUGINFOD-FILE", file.c_str());
+ add_mhd_last_modified (r, s.st_mtime);
+ if (!section.empty ())
+ add_mhd_response_header (r, "X-DEBUGINFOD-SECTION", section.c_str ());
+ if (verbose > 1)
+ {
+ if (!section.empty ())
+ obatched(clog) << "serving file " << b_source0
+ << " section " << section << endl;
+ else
+ obatched(clog) << "serving file " << b_source0 << endl;
+ }
+ /* libmicrohttpd will close it. */
+ if (result_fd)
+ *result_fd = fd;
+ }
+
+ return r;
+}
// For security/portability reasons, many distro-package archives have
// a "./" in front of path names; others have nothing, others have
@@ -1615,6 +1758,7 @@ handle_buildid_r_match (bool internal_req_p,
int64_t b_mtime,
const string& b_source0,
const string& b_source1,
+ const string& section,
int *result_fd)
{
struct stat fs;
@@ -1643,6 +1787,33 @@ handle_buildid_r_match (bool internal_req_p,
break; // branch out of if "loop", to try new libarchive fetch attempt
}
+ if (!section.empty ())
+ {
+ int scn_fd = extract_section_to_fd (fd, fs.st_mtime,
+ b_source0 + ":" + b_source1,
+ section);
+ close (fd);
+ if (scn_fd >= 0)
+ fd = scn_fd;
+ else
+ {
+ if (verbose)
+ obatched (clog) << "cannot find section " << section
+ << " for archive " << b_source0
+ << " file " << b_source1 << endl;
+ return 0;
+ }
+
+ rc = fstat(fd, &fs);
+ if (rc < 0)
+ {
+ close (fd);
+ throw libc_exception (errno,
+ string ("fstat archive ") + b_source0 + string (" file ") + b_source1
+ + string (" section ") + section);
+ }
+ }
+
struct MHD_Response* r = MHD_create_response_from_fd (fs.st_size, fd);
if (r == 0)
{
@@ -1659,9 +1830,19 @@ handle_buildid_r_match (bool internal_req_p,
to_string(fs.st_size).c_str());
add_mhd_response_header (r, "X-DEBUGINFOD-ARCHIVE", b_source0.c_str());
add_mhd_response_header (r, "X-DEBUGINFOD-FILE", b_source1.c_str());
+ if (!section.empty ())
+ add_mhd_response_header (r, "X-DEBUGINFOD-SECTION", section.c_str ());
add_mhd_last_modified (r, fs.st_mtime);
if (verbose > 1)
- obatched(clog) << "serving fdcache archive " << b_source0 << " file " << b_source1 << endl;
+ {
+ if (!section.empty ())
+ obatched(clog) << "serving fdcache archive " << b_source0
+ << " file " << b_source1
+ << " section " << section << endl;
+ else
+ obatched(clog) << "serving fdcache archive " << b_source0
+ << " file " << b_source1 << endl;
+ }
/* libmicrohttpd will close it. */
if (result_fd)
*result_fd = fd;
@@ -1792,8 +1973,36 @@ handle_buildid_r_match (bool internal_req_p,
tmppath, archive_entry_size(e),
true); // requested ones go to the front of lru
+ if (!section.empty ())
+ {
+ int scn_fd = extract_section_to_fd (fd, b_mtime,
+ b_source0 + ":" + b_source1,
+ section);
+ close (fd);
+ if (scn_fd >= 0)
+ fd = scn_fd;
+ else
+ {
+ if (verbose)
+ obatched (clog) << "cannot find section " << section
+ << " for archive " << b_source0
+ << " file " << b_source1 << endl;
+ return 0;
+ }
+
+ rc = fstat(fd, &fs);
+ if (rc < 0)
+ {
+ close (fd);
+ throw libc_exception (errno,
+ string ("fstat ") + b_source0 + string (" ") + section);
+ }
+ r = MHD_create_response_from_fd (fs.st_size, fd);
+ }
+ else
+ r = MHD_create_response_from_fd (archive_entry_size(e), fd);
+
inc_metric ("http_responses_total","result",archive_extension + " archive");
- r = MHD_create_response_from_fd (archive_entry_size(e), fd);
if (r == 0)
{
if (verbose)
@@ -1811,9 +2020,19 @@ handle_buildid_r_match (bool internal_req_p,
add_mhd_response_header (r, "X-DEBUGINFOD-ARCHIVE",
b_source0.c_str());
add_mhd_response_header (r, "X-DEBUGINFOD-FILE", file.c_str());
+ if (!section.empty ())
+ add_mhd_response_header (r, "X-DEBUGINFOD-SECTION", section.c_str ());
add_mhd_last_modified (r, archive_entry_mtime(e));
if (verbose > 1)
- obatched(clog) << "serving archive " << b_source0 << " file " << b_source1 << endl;
+ {
+ if (!section.empty ())
+ obatched(clog) << "serving archive " << b_source0
+ << " file " << b_source1
+ << " section " << section << endl;
+ else
+ obatched(clog) << "serving archive " << b_source0
+ << " file " << b_source1 << endl;
+ }
/* libmicrohttpd will close it. */
if (result_fd)
*result_fd = fd;
@@ -1832,14 +2051,17 @@ handle_buildid_match (bool internal_req_p,
const string& b_stype,
const string& b_source0,
const string& b_source1,
+ const string& section,
int *result_fd)
{
try
{
if (b_stype == "F")
- return handle_buildid_f_match(internal_req_p, b_mtime, b_source0, result_fd);
+ return handle_buildid_f_match(internal_req_p, b_mtime, b_source0,
+ section, result_fd);
else if (b_stype == "R")
- return handle_buildid_r_match(internal_req_p, b_mtime, b_source0, b_source1, result_fd);
+ return handle_buildid_r_match(internal_req_p, b_mtime, b_source0,
+ b_source1, section, result_fd);
}
catch (const reportable_exception &e)
{
@@ -1911,8 +2133,10 @@ handle_buildid (MHD_Connection* conn,
{
// validate artifacttype
string atype_code;
- if (artifacttype == "debuginfo") atype_code = "D";
- else if (artifacttype == "executable") atype_code = "E";
+ if (artifacttype == "debuginfo" || artifacttype == "debuginfo-section")
+ atype_code = "D";
+ else if (artifacttype == "executable" || artifacttype == "executable-section")
+ atype_code = "E";
else if (artifacttype == "source") atype_code = "S";
else {
artifacttype = "invalid"; // PR28242 ensure http_resposes metrics don't propagate unclean user data
@@ -1921,7 +2145,18 @@ handle_buildid (MHD_Connection* conn,
if (conn != 0)
inc_metric("http_requests_total", "type", artifacttype);
-
+
+ string section;
+ if (artifacttype == "debuginfo-section" || artifacttype == "executable-section")
+ {
+ // Ensure suffix contains a section name starting with "."
+ if (suffix.size () < 3 || suffix.substr (1, 1) != ".")
+ throw reportable_exception ("invalid section suffix");
+
+ // Remove leading '/'
+ section = suffix.substr(1);
+ }
+
if (atype_code == "S" && suffix == "")
throw reportable_exception("invalid source suffix");
@@ -1995,9 +2230,16 @@ handle_buildid (MHD_Connection* conn,
// Try accessing the located match.
// XXX: in case of multiple matches, attempt them in parallel?
auto r = handle_buildid_match (conn ? false : true,
- b_mtime, b_stype, b_source0, b_source1, result_fd);
+ b_mtime, b_stype, b_source0, b_source1,
+ section, result_fd);
if (r)
return r;
+
+ // If the file was found but it didn't contain the given section
+ // or the section was empty, then don't bother querying upstream servers.
+ if (!section.empty ()
+ && fdcache.probe (b_source0, b_source1))
+ throw reportable_exception(MHD_HTTP_NOT_FOUND, "not found");
}
pp->reset();
@@ -2067,10 +2309,18 @@ and will not query the upstream servers");
fd = debuginfod_find_debuginfo (client,
(const unsigned char*) buildid.c_str(),
0, NULL);
+ else if (artifacttype == "debuginfo-section")
+ fd = debuginfod_find_section (client,
+ (const unsigned char*) buildid.c_str(),
+ 0, section.c_str(), true, NULL);
else if (artifacttype == "executable")
fd = debuginfod_find_executable (client,
(const unsigned char*) buildid.c_str(),
0, NULL);
+ else if (artifacttype == "executable-section")
+ fd = debuginfod_find_section (client,
+ (const unsigned char*) buildid.c_str(),
+ 0, section.c_str(), false, NULL);
else if (artifacttype == "source")
fd = debuginfod_find_source (client,
(const unsigned char*) buildid.c_str(),
diff --git a/debuginfod/debuginfod.h.in b/debuginfod/debuginfod.h.in
index 40b1ea00..2e5ce5ab 100644
--- a/debuginfod/debuginfod.h.in
+++ b/debuginfod/debuginfod.h.in
@@ -42,6 +42,8 @@
/* The libdebuginfod soname. */
#define DEBUGINFOD_SONAME "@LIBDEBUGINFOD_SONAME@"
+#include <stdbool.h>
+
/* Handle for debuginfod-client connection. */
typedef struct debuginfod_client debuginfod_client;
@@ -78,6 +80,13 @@ int debuginfod_find_source (debuginfod_client *client,
const char *filename,
char **path);
+int debuginfod_find_section (debuginfod_client *client,
+ const unsigned char *build_id,
+ int build_id_len,
+ const char *section,
+ bool from_debuginfo,
+ char **path);
+
typedef int (*debuginfod_progressfn_t)(debuginfod_client *c, long a, long b);
void debuginfod_set_progressfn(debuginfod_client *c,
debuginfod_progressfn_t fn);
diff --git a/debuginfod/libdebuginfod.map b/debuginfod/libdebuginfod.map
index 93964167..6334373f 100644
--- a/debuginfod/libdebuginfod.map
+++ b/debuginfod/libdebuginfod.map
@@ -20,4 +20,5 @@ ELFUTILS_0.183 {
} ELFUTILS_0.179;
ELFUTILS_0.188 {
debuginfod_get_headers;
+ debuginfod_find_section;
} ELFUTILS_0.183;
diff --git a/doc/ChangeLog b/doc/ChangeLog
index b2bb4890..15279470 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,9 @@
+2022-09-27 Aaron Merey <amerey@redhat.com>
+
+ * Makefile.am (notrans_dist_man3_MANS): Add debuginfod_find_section.3.
+ * debuginfod_find_section.3: New file.
+ * debuginfod_find_debuginfo.3: Document debuginfod_find_section.
+
2022-09-02 Frank Ch. Eigler <fche@redhat.com>
* debuginfod_find_debuginfo.3: Tweaked debuginfod_get_headers docs.
diff --git a/doc/Makefile.am b/doc/Makefile.am
index db2506fd..db5a842e 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -38,6 +38,7 @@ notrans_dist_man3_MANS += debuginfod_end.3
notrans_dist_man3_MANS += debuginfod_find_debuginfo.3
notrans_dist_man3_MANS += debuginfod_find_executable.3
notrans_dist_man3_MANS += debuginfod_find_source.3
+notrans_dist_man3_MANS += debuginfod_find_section.3
notrans_dist_man3_MANS += debuginfod_get_user_data.3
notrans_dist_man3_MANS += debuginfod_get_url.3
notrans_dist_man3_MANS += debuginfod_set_progressfn.3
diff --git a/doc/debuginfod_find_debuginfo.3 b/doc/debuginfod_find_debuginfo.3
index aebbec3f..994b30dd 100644
--- a/doc/debuginfod_find_debuginfo.3
+++ b/doc/debuginfod_find_debuginfo.3
@@ -43,6 +43,13 @@ LOOKUP FUNCTIONS
.BI " int " build_id_len ","
.BI " const char *" filename ","
.BI " char ** " path ");"
+.BI "int debuginfod_find_section(debuginfod_client *" client ","
+.BI " const unsigned char *" build_id ","
+.BI " int " build_id_len ","
+.BI " const char * " section ","
+.BI " bool" from_debuginfo ","
+.BI " char ** " path ");"
+
OPTIONAL FUNCTIONS
@@ -98,6 +105,16 @@ accepts both forms. Specifically, debuginfod canonicalizes path names
according to RFC3986 section 5.2.4 (Remove Dot Segments), plus reducing
any \fB//\fP to \fB/\fP in the path.
+.BR debuginfod_find_section ()
+queries the debuginfod server URLs contained in
+.BR $DEBUGINFOD_URLS
+for the binary contents of an ELF/DWARF section contained in a debuginfo
+or executable file with the given \fIbuild_id\fP. \fIsection\fP should
+be the name of the desired ELF/DWARF section and must begin with ".".
+To fetch the section from the debuginfo file matching \fIbuild_id\fP,
+set \fIfrom_debuginfo\fP to true. To fetch the section from the executable
+file matching \fIbuild_id\fP set \fIfrom_debuginfo\fP to false.
+
If \fIpath\fP is not NULL and the query is successful, \fIpath\fP is set
to the path of the file in the cache. The caller must \fBfree\fP() this value.
@@ -228,11 +245,8 @@ void *debuginfod_so = dlopen(DEBUGINFOD_SONAME, RTLD_LAZY);
.in
.SH "SECURITY"
-.BR debuginfod_find_debuginfo (),
-.BR debuginfod_find_executable (),
-and
-.BR debuginfod_find_source ()
-\fBdo not\fP include any particular security
+.BR debuginfod_find_* ()
+functions \fBdo not\fP include any particular security
features. They trust that the binaries returned by the debuginfod(s)
are accurate. Therefore, the list of servers should include only
trustworthy ones. If accessed across HTTP rather than HTTPS, the
@@ -267,7 +281,11 @@ Unable to resolve remote host.
.BR EINVAL
One or more arguments are incorrectly formatted. \fIbuild_id\fP may
be too long (greater than 256 characters), \fIfilename\fP may not
-be an absolute path or a debuginfod URL is malformed.
+be an absolute path or a debuginfod URL is malformed. \fIsection\fP
+may not begin with ".". This error may also indicate that the section
+requested by
+.BR debuginfod_find_section ()
+had section header type SHT_NOBITS.
.TP
.BR EIO
diff --git a/doc/debuginfod_find_section.3 b/doc/debuginfod_find_section.3
new file mode 100644
index 00000000..16279936
--- /dev/null
+++ b/doc/debuginfod_find_section.3
@@ -0,0 +1 @@
+.so man3/debuginfod_find_debuginfo.3
diff --git a/tests/ChangeLog b/tests/ChangeLog
index f2cc7875..e44c2c27 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,8 @@
+2022-09-27 Aaron Merey <amerey@redhat.com>
+
+ * Makefile.am (TESTS): Add run-debuginfod-section.sh.
+ * run-debuginfod-section.sh: New test.
+
2022-08-26 Mark Wielaard <mark@klomp.org>
* run-ar-N.sh: New test.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 85514898..3d510f43 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -245,7 +245,8 @@ TESTS += run-debuginfod-dlopen.sh \
run-debuginfod-x-forwarded-for.sh \
run-debuginfod-response-headers.sh \
run-debuginfod-extraction-passive.sh \
- run-debuginfod-webapi-concurrency.sh
+ run-debuginfod-webapi-concurrency.sh \
+ run-debuginfod-section.sh
endif
if !OLD_LIBMICROHTTPD
# Will crash on too old libmicrohttpd
@@ -551,6 +552,7 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
run-debuginfod-response-headers.sh \
run-debuginfod-extraction-passive.sh \
run-debuginfod-webapi-concurrency.sh \
+ run-debuginfod-section.sh \
debuginfod-rpms/fedora30/hello2-1.0-2.src.rpm \
debuginfod-rpms/fedora30/hello2-1.0-2.x86_64.rpm \
debuginfod-rpms/fedora30/hello2-debuginfo-1.0-2.x86_64.rpm \
diff --git a/tests/run-debuginfod-section.sh b/tests/run-debuginfod-section.sh
new file mode 100755
index 00000000..bf8fa890
--- /dev/null
+++ b/tests/run-debuginfod-section.sh
@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+#
+# Copyright (C) 2019-2021 Red Hat, Inc.
+# This file is part of elfutils.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# elfutils is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+. $srcdir/debuginfod-subr.sh
+
+# for test case debugging, uncomment:
+set -x
+unset VALGRIND_CMD
+
+DB=${PWD}/.debuginfod_tmp.sqlite
+tempfiles $DB
+export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache
+
+# Set up directories for scanning
+mkdir F
+mkdir R
+cp -rvp ${abs_srcdir}/debuginfod-rpms R
+
+# This variable is essential and ensures no time-race for claiming ports occurs
+# set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test
+base=13000
+get_ports
+
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE \
+ -R R -F F -p $PORT1 -d $DB -t0 -g0 -v F > vlog$PORT1 2>&1 &
+PID1=$!
+tempfiles vlog$PORT1
+errfiles vlog$PORT1
+# Server must become ready
+wait_ready $PORT1 'ready' 1
+# And initial scan should be done
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 1
+
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT1
+
+# Check thread comm names
+ps -q $PID1 -e -L -o '%p %c %a' | grep groom
+ps -q $PID1 -e -L -o '%p %c %a' | grep scan
+ps -q $PID1 -e -L -o '%p %c %a' | grep traverse
+
+# We use -t0 and -g0 here to turn off time-based scanning & grooming.
+# For testing purposes, we just sic SIGUSR1 / SIGUSR2 at the process.
+
+########################################################################
+
+# Compile a simple program, strip its debuginfo and save the build-id.
+# Also move the debuginfo into another directory so that elfutils
+# cannot find it without debuginfod.
+echo "int main() { return 0; }" > ${PWD}/prog.c
+tempfiles prog.c
+
+gcc -Wl,--build-id -g -o F/prog ${PWD}/prog.c
+
+testrun ${abs_top_builddir}/src/strip -g -f F/prog.debug ${PWD}/F/prog
+BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
+ -a F/prog | grep 'Build ID' | cut -d ' ' -f 7`
+
+kill -USR1 $PID1
+# Wait till both files are in the index.
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 2
+wait_ready $PORT1 'thread_work_pending{role="scan"}' 0
+wait_ready $PORT1 'thread_busy{role="scan"}' 0
+
+########################################################################
+
+# Build-id for a file in the one of the testsuite's F31 rpms
+RPM_BUILDID=420e9e3308971f4b817cc5bf83928b41a6909d88
+
+# Download sections from files indexed with -F
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find -vvv debuginfo $BUILDID .debug_info > vlog-find.1 2>&1
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find -vvv executable $BUILDID .text > vlog-find.2 2>&1
+tempfiles vlog-find.1 vlog-find.2
+
+# Check that the X-DEBUGINFOD-SECTION response header contains the correct value
+testrun grep -i 'X-DEBUGINFOD-SECTION: .debug_info' vlog-find.1
+testrun grep -i 'X-DEBUGINFOD-SECTION: .text' vlog-find.2
+
+# Download sections from files indexed with -R
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find -vvv debuginfo $RPM_BUILDID .debug_info > vlog-find.1 2>&1
+testrun ${abs_top_builddir}/debuginfod/debuginfod-find -vvv executable $RPM_BUILDID .text > vlog-find.2 2>&1
+
+testrun grep -i 'X-DEBUGINFOD-SECTION: .debug_info' vlog-find.1
+testrun grep -i 'X-DEBUGINFOD-SECTION: .text' vlog-find.2
+
+# Verify that the downloaded files match the contents of the original sections
+objcopy F/prog.debug -O binary --only-section=.debug_info --set-section-flags .debug_info=alloc $BUILDID.debug_info
+tempfiles ${BUILDID}.debug_info
+testrun diff -u ${BUILDID}.debug_info ${DEBUGINFOD_CACHE_PATH}/${BUILDID}/debuginfo.debug_info
+
+objcopy F/prog -O binary --only-section=.text ${BUILDID}.text
+tempfiles ${BUILDID}.text
+testrun diff -u ${BUILDID}.text ${DEBUGINFOD_CACHE_PATH}/${BUILDID}/executable.text
+
+# Download the files contained in the rpm in order to objcopy the original sections
+DEBUGFILE=`${abs_top_builddir}/debuginfod/debuginfod-find debuginfo $RPM_BUILDID`
+EXECFILE=`${abs_top_builddir}/debuginfod/debuginfod-find executable $RPM_BUILDID`
+
+objcopy $DEBUGFILE -O binary --only-section=.debug_info --set-section-flags .debug_info=alloc DEBUGFILE.debug_info
+tempfiles DEBUGFILE.debug_info
+testrun diff -u DEBUGFILE.debug_info ${DEBUGINFOD_CACHE_PATH}/${RPM_BUILDID}/debuginfo.debug_info
+
+objcopy $EXECFILE -O binary --only-section=.text EXECFILE.text
+tempfiles EXECFILE.text
+testrun diff -u EXECFILE.text ${DEBUGINFOD_CACHE_PATH}/${RPM_BUILDID}/executable.text
+
+kill $PID1
+wait $PID1
+PID1=0
+exit 0
--
2.37.3
next reply other threads:[~2022-09-28 2:11 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-28 2:10 Aaron Merey [this message]
2022-09-28 14:28 ` Frank Ch. Eigler
2022-09-28 18:20 ` Aaron Merey
2022-10-21 0:06 [PATCH v2] " Aaron Merey
2022-10-22 0:09 ` [PATCH] " Aaron Merey
2022-10-24 18:38 ` Frank Ch. Eigler
2022-10-26 14:57 ` Mark Wielaard
2022-10-26 15:28 ` Frank Ch. Eigler
2022-10-26 17:57 ` Aaron Merey
2022-10-26 15:25 ` Mark Wielaard
2022-10-26 17:55 ` Aaron Merey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220928021052.315981-1-amerey@redhat.com \
--to=amerey@redhat.com \
--cc=elfutils-devel@sourceware.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).