public inbox for elfutils@sourceware.org
 help / color / mirror / Atom feed
* [Bug debuginfod/27277] Describe retrieved files when verbose
@ 2021-08-04 18:54 Noah Sanci
  2021-08-05 15:13 ` Mark Wielaard
  0 siblings, 1 reply; 23+ messages in thread
From: Noah Sanci @ 2021-08-04 18:54 UTC (permalink / raw)
  To: elfutils-devel

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]

Hello,

The attached patch dumps the http headers retrieved from the
debuginfod server which a file was downloaded from. Some custom
headers are now returned such as X-FILE, X-FILE-SIZE, and X-ARCHIVE to
give some context about the downloaded files.

Noah Sanci

[-- Attachment #2: 0001-debuginfod-PR27277-Describe-retrieved-files-when-ver.patch --]
[-- Type: text/x-patch, Size: 13772 bytes --]

From dcbe8672d6be30f92ad8baa2fa157b9b7a551b48 Mon Sep 17 00:00:00 2001
From: Noah Sanci <nsanci@redhat.com>
Date: Wed, 28 Jul 2021 14:46:05 -0400
Subject: [PATCH] debuginfod: PR27277 - Describe retrieved files when verbose

There appear to exist use cases that intend to simply check for the
existence of content in a debuginfod server, without actually
downloading it.  In HTTP land, the HEAD operation is the natural
expression of this.  We could support this in the webapi, and give
options to debuginfod-find and the client API to use it.
Instead of implementing a describe option, allow users, with enough
verbosity, to get a file description printed to the verbose output
stream upon retrieving a file. The HEAD operation is not supported in
this patch.

E.g output:

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 2428240
Cache-Control: public
Last-Modified: Sat, 15 May 2021 20:49:51 GMT
X-FILE-SIZE: 656906321
X-FILE: /usr/lib/debug/lib/modules/5.11.21-200.fc33.x86_64/kernel/drivers/scsi/bnx2fc/bnx2fc.ko.debug
X-ARCHIVE: kernel-debuginfo-5.11.21-200.fc33.x86_64.rpm
Content-Type: application/octet-stream
Date: Tue, 03 Aug 2021 18:50:36 GMT

https://sourceware.org/bugzilla/show_bug.cgi?id=27277

Signed-off-by: Noah Sanci <nsanci@redhat.com>
---
 debuginfod/ChangeLog           | 16 ++++++++++
 debuginfod/debuginfod-client.c | 58 ++++++++++++++++++++++++++++++++--
 debuginfod/debuginfod-find.c   |  3 ++
 debuginfod/debuginfod.cxx      | 11 +++++++
 debuginfod/debuginfod.h.in     |  3 ++
 debuginfod/libdebuginfod.map   |  3 ++
 doc/ChangeLog                  |  6 ++++
 doc/debuginfod-find.1          |  3 +-
 tests/ChangeLog                |  6 ++++
 tests/run-debuginfod-find.sh   | 19 ++++++++++-
 10 files changed, 124 insertions(+), 4 deletions(-)

diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 9e82d78d..4297b088 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -1,3 +1,19 @@
+2021-08-02  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* debuginfod-client.c (debuginfod_client): New field winning_headers.
+	(handle_data): New field response_data.
+	(header_callback): Store received headers in response_data.
+	(debuginfod_query_server): Activate CURLOPT_HEADERFUNCTION.
+	Save winning response_data.
+	(debuginfod_get_response_headers): Return the winning headers.
+	* debuginfod.h.in: Declare new API function.
+	* libdebuginfod.map: Export it.
+	* debuginfod-find.c: Call it in verbose mode.
+	* debuginfod.cxx (handle_buildid_f_match): Add X-FILE and X-FILE-SIZE
+	http headers to response.
+	(handle_buildid_r_match): Add X-FILE, X-FILE-SIZE and X-ARCHIVE headers.
+
 2021-07-26  Noah Sanci  <nsanci@redhat.com>
 
 	PR27982
diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c
index 7d4b220f..5c2a646c 100644
--- a/debuginfod/debuginfod-client.c
+++ b/debuginfod/debuginfod-client.c
@@ -127,6 +127,7 @@ struct debuginfod_client
      timeout or other info gotten from environment variables, the
      handle data, etc. So those don't have to be reparsed and
      recreated on each request.  */
+  char * winning_headers;
 };
 
 /* The cache_clean_interval_s file within the debuginfod cache specifies
@@ -183,6 +184,8 @@ struct handle_data
      to the cache. Used to ensure that a file is not downloaded from
      multiple servers unnecessarily.  */
   CURL **target_handle;
+  /* Response http headers for this client handle, sent from the server */
+  char *response_data;
 };
 
 static size_t
@@ -499,6 +502,33 @@ default_progressfn (debuginfod_client *c, long a, long b)
 }
 
 
+static size_t
+header_callback (char * buffer, size_t size, size_t numitems, void * userdata)
+{
+  if (size != 1)
+    return 0;
+  /* Temporary buffer for realloc */
+  char *temp = NULL;
+  size_t userlen = 0;
+  if (*(char**)userdata == NULL)
+    {
+      temp = malloc(numitems+1);
+      if (temp == NULL)
+        return 0;
+      memset(temp, '\0', numitems+1);
+    }
+  else
+    {
+      userlen = strlen(*(char**)userdata);
+      temp = realloc(*(char**)userdata, userlen + numitems + 1);
+      if (temp == NULL)
+       return 0;
+    }
+  strncat(temp, buffer, numitems);
+  *(char**)userdata = temp;
+  return numitems;
+}
+
 /* Query each of the server URLs found in $DEBUGINFOD_URLS for the file
    with the specified build-id, type (debuginfo, executable or source)
    and filename. filename may be NULL. If found, return a file
@@ -936,10 +966,13 @@ debuginfod_query_server (debuginfod_client *c,
 	  curl_easy_setopt (data[i].handle, CURLOPT_LOW_SPEED_LIMIT,
 			    100 * 1024L);
 	}
+      data[i].response_data = NULL;
       curl_easy_setopt(data[i].handle, CURLOPT_FILETIME, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_FOLLOWLOCATION, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_FAILONERROR, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_NOSIGNAL, (long) 1);
+      curl_easy_setopt(data[i].handle, CURLOPT_HEADERFUNCTION, header_callback);
+      curl_easy_setopt(data[i].handle, CURLOPT_HEADERDATA, (void *) &(data[i].response_data));
 #if LIBCURL_VERSION_NUM >= 0x072a00 /* 7.42.0 */
       curl_easy_setopt(data[i].handle, CURLOPT_PATH_AS_IS, (long) 1);
 #else
@@ -1161,10 +1194,10 @@ debuginfod_query_server (debuginfod_client *c,
                 {
                   char *effective_url = NULL;
                   long resp_code = 500;
-                  CURLcode ok1 = curl_easy_getinfo (target_handle,
+                  CURLcode ok1 = curl_easy_getinfo (msg->easy_handle,
 						    CURLINFO_EFFECTIVE_URL,
 						    &effective_url);
-                  CURLcode ok2 = curl_easy_getinfo (target_handle,
+                  CURLcode ok2 = curl_easy_getinfo (msg->easy_handle,
 						    CURLINFO_RESPONSE_CODE,
 						    &resp_code);
                   if(ok1 == CURLE_OK && ok2 == CURLE_OK && effective_url)
@@ -1238,6 +1271,7 @@ debuginfod_query_server (debuginfod_client *c,
             {
               curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
               curl_easy_cleanup (data[i].handle);
+              free(data[i].response_data);
             }
 	    goto query_in_parallel;
 	}
@@ -1264,6 +1298,16 @@ debuginfod_query_server (debuginfod_client *c,
   tvs[0].tv_usec = tvs[1].tv_usec = 0;
   (void) futimes (fd, tvs);  /* best effort */
 
+  c->winning_headers = NULL;
+  for (int i = 0; i < num_urls; ++i)
+      if (data[i].handle == verified_handle && data[i].response_data != NULL)
+        {
+          c->winning_headers = data[i].response_data;
+          if (vfd >= 0)
+            dprintf(vfd, "\n%s", c->winning_headers);
+          data[i].response_data = NULL;
+        }
+
   /* PR27571: make cache files casually unwriteable; dirs are already 0700 */
   (void) fchmod(fd, 0400);
                 
@@ -1281,6 +1325,7 @@ debuginfod_query_server (debuginfod_client *c,
     {
       curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
       curl_easy_cleanup (data[i].handle);
+      free (data[i].response_data);
     }
 
   for (int i = 0; i < num_urls; ++i)
@@ -1304,6 +1349,7 @@ debuginfod_query_server (debuginfod_client *c,
     {
       curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
       curl_easy_cleanup (data[i].handle);
+      free (data[i].response_data);
     }
 
   unlink (target_cache_tmppath);
@@ -1415,6 +1461,7 @@ debuginfod_end (debuginfod_client *client)
 
   curl_multi_cleanup (client->server_mhandle);
   curl_slist_free_all (client->headers);
+  free (client->winning_headers);
   free (client->url);
   free (client);
 }
@@ -1483,6 +1530,13 @@ debuginfod_set_progressfn(debuginfod_client *client,
   client->progressfn = fn;
 }
 
+/* Return the http response headers that came from the winning server */
+const char *
+debuginfod_get_response_headers(debuginfod_client *c)
+{
+  return c->winning_headers;
+}
+
 void
 debuginfod_set_verbose_fd(debuginfod_client *client, int fd)
 {
diff --git a/debuginfod/debuginfod-find.c b/debuginfod/debuginfod-find.c
index 3e8ab203..1ed76ffc 100644
--- a/debuginfod/debuginfod-find.c
+++ b/debuginfod/debuginfod-find.c
@@ -218,6 +218,9 @@ main(int argc, char** argv)
       const char* url = debuginfod_get_url (client);
       if (url != NULL)
         fprintf(stderr, "Downloaded from %s\n", url);
+      const char *headers = debuginfod_get_response_headers (client);
+      if (headers != NULL)
+        fprintf(stderr, "Headers:\n%s\n", headers);
     }
 
   debuginfod_end (client);
diff --git a/debuginfod/debuginfod.cxx b/debuginfod/debuginfod.cxx
index 4ddd9255..cc653a3a 100644
--- a/debuginfod/debuginfod.cxx
+++ b/debuginfod/debuginfod.cxx
@@ -1063,6 +1063,8 @@ handle_buildid_f_match (bool internal_req_t,
   else
     {
       MHD_add_response_header (r, "Content-Type", "application/octet-stream");
+      (void) MHD_add_response_header(r, "X-FILE-SIZE", std::to_string(s.st_size).c_str());
+      (void) MHD_add_response_header(r, "X-FILE", b_source0.c_str());
       add_mhd_last_modified (r, s.st_mtime);
       if (verbose > 1)
         obatched(clog) << "serving file " << b_source0 << endl;
@@ -1673,6 +1675,15 @@ handle_buildid_r_match (bool internal_req_p,
       else
         {
           MHD_add_response_header (r, "Content-Type", "application/octet-stream");
+          size_t place = b_source0.find_last_of("/");
+
+          if (place == string::npos)
+            (void) MHD_add_response_header(r, "X-ARCHIVE", b_source0.c_str());
+          else
+            (void) MHD_add_response_header(r, "X-ARCHIVE", (char *) ((unsigned long)b_source0.c_str() + place + 1) );
+
+          (void) MHD_add_response_header(r, "X-FILE", b_source1.c_str());
+          (void) MHD_add_response_header(r, "X-FILE-SIZE", std::to_string(fs.st_size).c_str());
           add_mhd_last_modified (r, archive_entry_mtime(e));
           if (verbose > 1)
             obatched(clog) << "serving archive " << b_source0 << " file " << b_source1 << endl;
diff --git a/debuginfod/debuginfod.h.in b/debuginfod/debuginfod.h.in
index c358df4d..21a1bab8 100644
--- a/debuginfod/debuginfod.h.in
+++ b/debuginfod/debuginfod.h.in
@@ -93,6 +93,9 @@ void* debuginfod_get_user_data (debuginfod_client *client);
 /* Get the current or last active URL, if known.  */
 const char* debuginfod_get_url (debuginfod_client *client);
 
+/* Get last set of HTTP response headers, if known.  */
+const char* debuginfod_get_response_headers (debuginfod_client *client);
+  
 /* Add an outgoing HTTP request  "Header: Value".  Copies string.  */
 int debuginfod_add_http_header (debuginfod_client *client, const char* header);
 
diff --git a/debuginfod/libdebuginfod.map b/debuginfod/libdebuginfod.map
index 7d2f5882..1798ff63 100644
--- a/debuginfod/libdebuginfod.map
+++ b/debuginfod/libdebuginfod.map
@@ -18,3 +18,6 @@ ELFUTILS_0.179 {
 ELFUTILS_0.183 {
   debuginfod_set_verbose_fd;
 } ELFUTILS_0.179;
+ELFUTILS_0.186 {
+  debuginfod_get_response_headers;
+} ELFUTILS_0.183;
diff --git a/doc/ChangeLog b/doc/ChangeLog
index 1822fc6b..855cc4a2 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -1,3 +1,9 @@
+2021-08-04  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* debuginfod-find.1: Increasing verbosity describes the downloaded
+	file.
+
 2021-07-26  Noah Sanci <nsanci@redhat.com>
 
 	PR27982
diff --git a/doc/debuginfod-find.1 b/doc/debuginfod-find.1
index 482a8ae7..9010aa7b 100644
--- a/doc/debuginfod-find.1
+++ b/doc/debuginfod-find.1
@@ -110,7 +110,8 @@ l l.
 
 .TP
 .B "\-v"
-Increase verbosity, including printing frequent download-progress messages.
+Increase verbosity, including printing frequent download-progress messages
+and printing a brief description of the downloaded file.
 
 
 .SH "SECURITY"
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 34666609..3ca81418 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,9 @@
+2021-08-02  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* run-debuginfod-find.sh: Add a test to ensure that file descriptions
+	are accurate for files outside or within archives
+
 2021-07-26  Noah Sanci  <nsanci@redhat.com>
 
 	PR27982
diff --git a/tests/run-debuginfod-find.sh b/tests/run-debuginfod-find.sh
index 991d1dc5..710ac03f 100755
--- a/tests/run-debuginfod-find.sh
+++ b/tests/run-debuginfod-find.sh
@@ -526,7 +526,24 @@ wait_ready $PORT1 'fdcache_op_count{op="evict"}' $( grep -c 'evicted a=.*' vlog$
 wait_ready $PORT1 'fdcache_op_count{op="prefetch_enqueue"}' $( grep -c 'interned.*front=0' vlog$PORT1 )
 wait_ready $PORT1 'fdcache_op_count{op="prefetch_evict"}' $( grep -c 'evicted from prefetch a=.*front=0' vlog$PORT1 || true )
 ########################################################################
-
+## PR27277
+# Check to see if source file not located in an archive prints the file's description
+env DEBUGINFOD_URLS="http://127.0.0.1:$PORT1" LD_LIBRARY_PATH=$ldpath DEBUGINFOD_MASIZE=1\
+ ${abs_top_builddir}/debuginfod/debuginfod-find -v source $BUILDID2 ${PWD}/prog2.c > vlog-find$PORT1.1 2>&1
+
+tempfiles vlog-find$PORT1.1
+grep 'X-FILE: '$(realpath $PWD)'/prog2.c' vlog-find$PORT1.1
+grep 'X-FILE-SIZE: '$(wc -c ${PWD}'/prog2.c' | awk '{print $1}') vlog-find$PORT1.1
+
+# Check to see if an executable file located in an archive prints the file's description and archive
+env DEBUGINFOD_URLS="http://127.0.0.1:$PORT1" LD_LIBRARY_PATH=$ldpath \
+ ${abs_top_builddir}/debuginfod/debuginfod-find $VERBOSE executable c36708a78618d597dee15d0dc989f093ca5f9120 > vlog-find$PORT1.2 2>&1
+
+tempfiles vlog-find$PORT1.2
+grep 'X-FILE: ' vlog-find$PORT1.2
+grep 'X-FILE-SIZE: ' vlog-find$PORT1.2
+grep 'X-ARCHIVE: hello2-1.0-2.x86_64.rpm' vlog-find$PORT1.2
+########################################################################
 # Federation mode
 
 # find another unused port
-- 
2.31.1


^ permalink raw reply	[flat|nested] 23+ messages in thread
* [Bug debuginfod/27277] Describe retrieved files when verbose
@ 2021-08-25 18:08 Noah Sanci
  2021-09-08 15:01 ` Mark Wielaard
  0 siblings, 1 reply; 23+ messages in thread
From: Noah Sanci @ 2021-08-25 18:08 UTC (permalink / raw)
  To: elfutils-devel

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

Hello,

There appear to exist use cases that intend to simply check for the
existence of content in a debuginfod server, without actually
downloading it.  In HTTP land, the HEAD operation is the natural
expression of this.
Instead of implementing a HEAD/describe option, allow users, with enough
verbosity, to print the HTTP response headers upon retrieving a file.
E.g output:

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 2428240
Cache-Control: public
Last-Modified: Sat, 15 May 2021 20:49:51 GMT
Content-Type: application/octet-stream
Date: Tue, 03 Aug 2021 18:50:36 GMT

Note: the new test is mostly compatible with the nsanci/test-fix
commit. When nsanci/test-fix merges with master I can resubmit the
patch with the test properly rebased and adjusted for maximum
efficiency.

The HEAD functionality will (likely) be put into a new PR for existing checking.

-Noah Sanci

[-- Attachment #2: 0001-debuginfod-PR27277-Describe-retrieved-files-when-ver.patch --]
[-- Type: text/x-patch, Size: 16446 bytes --]

From 6351258d707337b69563d4be8effbb30fc42f784 Mon Sep 17 00:00:00 2001
From: Noah Sanci <nsanci@redhat.com>
Date: Wed, 28 Jul 2021 14:46:05 -0400
Subject: [PATCH] debuginfod: PR27277 - Describe retrieved files when verbose

There appear to exist use cases that intend to simply check for the
existence of content in a debuginfod server, without actually
downloading it.  In HTTP land, the HEAD operation is the natural
expression of this.
Instead of implementing a HEAD/describe option, allow users, with enough
verbosity, to print the HTTP response headers upon retrieving a file.
E.g output:

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Length: 2428240
Cache-Control: public
Last-Modified: Sat, 15 May 2021 20:49:51 GMT
Content-Type: application/octet-stream
Date: Tue, 03 Aug 2021 18:50:36 GMT

https://sourceware.org/bugzilla/show_bug.cgi?id=27277

Signed-off-by: Noah Sanci <nsanci@redhat.com>
---
 debuginfod/ChangeLog                     |  11 ++
 debuginfod/debuginfod-client.c           |  58 ++++++-
 doc/ChangeLog                            |   6 +
 doc/debuginfod-find.1                    |   3 +-
 tests/ChangeLog                          |   7 +
 tests/Makefile.am                        |   3 +-
 tests/run-debuginfod-response-headers.sh | 186 +++++++++++++++++++++++
 7 files changed, 268 insertions(+), 6 deletions(-)
 create mode 100755 tests/run-debuginfod-response-headers.sh

diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 530f7dc7..cb9e50c7 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -4,6 +4,17 @@
 	* debuginfod.cxx (handler_cb): Fix after_you unique_set key
 	to the entire incoming URL.
 
+2021-08-02  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* debuginfod-client.c (struct debuginfod_client): New field 
+	winning_headers.
+	(struct handle_data): New field response_data.
+	(header_callback): Store received headers in response_data.
+	(debuginfod_query_server): Activate CURLOPT_HEADERFUNCTION.
+	Save winning response_data.
+	(debuginfod_end): free client winning headers.
+
 2021-07-26  Noah Sanci  <nsanci@redhat.com>
 
 	PR27982
diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c
index 7d4b220f..985a17c5 100644
--- a/debuginfod/debuginfod-client.c
+++ b/debuginfod/debuginfod-client.c
@@ -127,6 +127,7 @@ struct debuginfod_client
      timeout or other info gotten from environment variables, the
      handle data, etc. So those don't have to be reparsed and
      recreated on each request.  */
+  char * winning_headers;
 };
 
 /* The cache_clean_interval_s file within the debuginfod cache specifies
@@ -183,6 +184,8 @@ struct handle_data
      to the cache. Used to ensure that a file is not downloaded from
      multiple servers unnecessarily.  */
   CURL **target_handle;
+  /* Response http headers for this client handle, sent from the server */
+  char *response_data;
 };
 
 static size_t
@@ -499,6 +502,33 @@ default_progressfn (debuginfod_client *c, long a, long b)
 }
 
 
+static size_t
+header_callback (char * buffer, size_t size, size_t numitems, void * userdata)
+{
+  if (size != 1)
+    return 0;
+  /* Temporary buffer for realloc */
+  char *temp = NULL;
+  size_t userlen = 0;
+  if (*(char**)userdata == NULL)
+    {
+      temp = malloc(numitems+1);
+      if (temp == NULL)
+        return 0;
+      memset(temp, '\0', numitems+1);
+    }
+  else
+    {
+      userlen = strlen(*(char**)userdata);
+      temp = realloc(*(char**)userdata, userlen + numitems + 1);
+      if (temp == NULL)
+       return 0;
+    }
+  strncat(temp, buffer, numitems);
+  *(char**)userdata = temp;
+  return numitems;
+}
+
 /* Query each of the server URLs found in $DEBUGINFOD_URLS for the file
    with the specified build-id, type (debuginfo, executable or source)
    and filename. filename may be NULL. If found, return a file
@@ -936,10 +966,13 @@ debuginfod_query_server (debuginfod_client *c,
 	  curl_easy_setopt (data[i].handle, CURLOPT_LOW_SPEED_LIMIT,
 			    100 * 1024L);
 	}
+      data[i].response_data = NULL;
       curl_easy_setopt(data[i].handle, CURLOPT_FILETIME, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_FOLLOWLOCATION, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_FAILONERROR, (long) 1);
       curl_easy_setopt(data[i].handle, CURLOPT_NOSIGNAL, (long) 1);
+      curl_easy_setopt(data[i].handle, CURLOPT_HEADERFUNCTION, header_callback);
+      curl_easy_setopt(data[i].handle, CURLOPT_HEADERDATA, (void *) &(data[i].response_data));
 #if LIBCURL_VERSION_NUM >= 0x072a00 /* 7.42.0 */
       curl_easy_setopt(data[i].handle, CURLOPT_PATH_AS_IS, (long) 1);
 #else
@@ -961,6 +994,7 @@ debuginfod_query_server (debuginfod_client *c,
   int committed_to = -1;
   bool verbose_reported = false;
   struct timespec start_time, cur_time;
+  c->winning_headers = NULL;
   if ( maxtime > 0 && clock_gettime(CLOCK_MONOTONIC_RAW, &start_time) == -1)
     {
       rc = errno;
@@ -995,8 +1029,18 @@ debuginfod_query_server (debuginfod_client *c,
 	    if (data[i].handle != target_handle)
 	      curl_multi_remove_handle(curlm, data[i].handle);
 	    else
-	      committed_to = i;
-	}
+              {
+	        committed_to = i;
+                if (c->winning_headers == NULL)
+                  {
+                    c->winning_headers = data[committed_to].response_data;
+                    if (vfd >= 0)
+                      dprintf(vfd, "\n%s", c->winning_headers);
+                    data[committed_to].response_data = NULL;
+                  }
+
+              }
+        }
 
       if (vfd >= 0 && !verbose_reported && committed_to >= 0)
 	{
@@ -1161,10 +1205,10 @@ debuginfod_query_server (debuginfod_client *c,
                 {
                   char *effective_url = NULL;
                   long resp_code = 500;
-                  CURLcode ok1 = curl_easy_getinfo (target_handle,
+                  CURLcode ok1 = curl_easy_getinfo (msg->easy_handle,
 						    CURLINFO_EFFECTIVE_URL,
 						    &effective_url);
-                  CURLcode ok2 = curl_easy_getinfo (target_handle,
+                  CURLcode ok2 = curl_easy_getinfo (msg->easy_handle,
 						    CURLINFO_RESPONSE_CODE,
 						    &resp_code);
                   if(ok1 == CURLE_OK && ok2 == CURLE_OK && effective_url)
@@ -1238,7 +1282,10 @@ debuginfod_query_server (debuginfod_client *c,
             {
               curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
               curl_easy_cleanup (data[i].handle);
+              free(data[i].response_data);
             }
+            free(c->winning_headers);
+            c->winning_headers = NULL;
 	    goto query_in_parallel;
 	}
       else
@@ -1281,6 +1328,7 @@ debuginfod_query_server (debuginfod_client *c,
     {
       curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
       curl_easy_cleanup (data[i].handle);
+      free (data[i].response_data);
     }
 
   for (int i = 0; i < num_urls; ++i)
@@ -1304,6 +1352,7 @@ debuginfod_query_server (debuginfod_client *c,
     {
       curl_multi_remove_handle(curlm, data[i].handle); /* ok to repeat */
       curl_easy_cleanup (data[i].handle);
+      free (data[i].response_data);
     }
 
   unlink (target_cache_tmppath);
@@ -1415,6 +1464,7 @@ debuginfod_end (debuginfod_client *client)
 
   curl_multi_cleanup (client->server_mhandle);
   curl_slist_free_all (client->headers);
+  free (client->winning_headers);
   free (client->url);
   free (client);
 }
diff --git a/doc/ChangeLog b/doc/ChangeLog
index 77d1d705..3f542e08 100644
--- a/doc/ChangeLog
+++ b/doc/ChangeLog
@@ -10,6 +10,12 @@
 	* Makefile.am: Updated to include debuginfod-client-config.7
 	* man3, man7: Symlinks for source tree man page testing.
 
+2021-08-04  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* debuginfod-find.1: Increasing verbosity describes the downloaded
+	file.
+
 2021-07-26  Noah Sanci <nsanci@redhat.com>
 
 	PR27982
diff --git a/doc/debuginfod-find.1 b/doc/debuginfod-find.1
index a61673f5..957ec7e7 100644
--- a/doc/debuginfod-find.1
+++ b/doc/debuginfod-find.1
@@ -110,7 +110,8 @@ l l.
 
 .TP
 .B "\-v"
-Increase verbosity, including printing frequent download-progress messages.
+Increase verbosity, including printing frequent download-progress messages
+and printing the http response headers from the server.
 
 
 .SH "SECURITY"
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 3bfd1ca2..f4aaf3ee 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -4,6 +4,13 @@
 	* backtrace.c (callback_verify): Check for pthread_kill as first
 	frame. Change asserts to fprintf plus abort.
 
+2021-08-02  Noah Sanci  <nsanci@redhat.com>
+
+	PR27277
+	* run-debuginfod-response-headers.sh: Add a test to ensure that file descriptions
+	are accurate for files outside or within archives.
+	* Makefile.am: Add run-debuginfod-response-headers.sh to TESTS.
+
 2021-07-26  Noah Sanci  <nsanci@redhat.com>
 
 	PR27982
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 429649f4..99fa61ba 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -211,7 +211,7 @@ if DEBUGINFOD
 check_PROGRAMS += debuginfod_build_id_find
 # With the dummy delegation doesn't work
 if !DUMMY_LIBDEBUGINFOD
-TESTS += run-debuginfod-find.sh
+TESTS += run-debuginfod-find.sh run-debuginfod-response-headers.sh
 endif
 endif
 
@@ -475,6 +475,7 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
 	     run-disasm-riscv64.sh \
 	     testfile-riscv64-dis1.o.bz2 testfile-riscv64-dis1.expect.bz2 \
              run-debuginfod-find.sh \
+             run-debuginfod-response-headers.sh \
 	     debuginfod-rpms/fedora30/hello2-1.0-2.src.rpm \
 	     debuginfod-rpms/fedora30/hello2-1.0-2.x86_64.rpm \
 	     debuginfod-rpms/fedora30/hello2-debuginfo-1.0-2.x86_64.rpm \
diff --git a/tests/run-debuginfod-response-headers.sh b/tests/run-debuginfod-response-headers.sh
new file mode 100755
index 00000000..be1345fc
--- /dev/null
+++ b/tests/run-debuginfod-response-headers.sh
@@ -0,0 +1,186 @@
+#!/usr/bin/env bash
+#
+# Copyright (C) 2019-2021 Red Hat, Inc.
+# This file is part of elfutils.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# elfutils is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+. $srcdir/test-subr.sh  # includes set -e
+
+# for test case debugging, uncomment:
+set -x
+
+DB=${PWD}/.debuginfod_tmp.sqlite
+tempfiles $DB
+export DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache
+
+while true; do
+    PORT1=`expr '(' $RANDOM % 1000 ')' + 9000`
+    ss -atn | fgrep ":$PORT1" || break
+done
+PID1=0
+PID2=0
+PID3=0
+PID4=0
+
+cleanup()
+{
+  if [ $PID1 -ne 0 ]; then kill $PID1; wait $PID1; fi
+  if [ $PID2 -ne 0 ]; then kill $PID2; wait $PID2; fi
+  if [ $PID3 -ne 0 ]; then kill $PID3; wait $PID3; fi
+  if [ $PID4 -ne 0 ]; then kill $PID4; wait $PID4; fi
+  rm -rf F R D L Z ${PWD}/foobar ${PWD}/mocktree ${PWD}/.client_cache* ${PWD}/tmp*
+  exit_cleanup
+}
+
+# clean up trash if we were aborted early
+trap cleanup 0 1 2 3 5 9 15
+
+errfiles_list=
+err() {
+    echo ERROR REPORTS
+    for ports in $PORT1 $PORT2 $PORT3
+    do
+        echo ERROR REPORT $port metrics
+        curl -s http://127.0.0.1:$port/metrics
+        echo
+    done
+    for x in $errfiles_list
+    do
+        echo ERROR REPORT "$x"
+        cat $x
+        echo
+    done
+    false # trigger set -e
+}
+trap err ERR
+
+errfiles() {
+    errfiles_list="$errfiles_list $*"
+}
+
+wait_ready()
+{
+  port=$1;
+  what=$2;
+  value=$3;
+  timeout=20;
+
+  echo "Wait $timeout seconds on $port for metric $what to change to $value"
+  while [ $timeout -gt 0 ]; do
+    mvalue="$(curl -s http://127.0.0.1:$port/metrics \
+              | grep "$what" | awk '{print $NF}')"
+    if [ -z "$mvalue" ]; then mvalue=0; fi
+      echo "metric $what: $mvalue"
+      if [ "$mvalue" -eq "$value" ]; then
+        break;
+    fi
+    sleep 0.5;
+    ((timeout--));
+  done;
+
+  if [ $timeout -eq 0 ]; then
+    echo "metric $what never changed to $value on port $port"
+    err
+  fi
+}
+
+# We want to run debuginfod in the background.  We also want to start
+# it with the same check/installcheck-sensitive LD_LIBRARY_PATH stuff
+# that the testrun alias sets.  But: we if we just use
+#    testrun .../debuginfod
+# it runs in a subshell, with different pid, so not helpful.
+#
+# So we gather the LD_LIBRARY_PATH with this cunning trick:
+ldpath=`testrun sh -c 'echo $LD_LIBRARY_PATH'`
+
+mkdir F R
+
+env LD_LIBRARY_PATH=$ldpath DEBUGINFOD_URLS= ${abs_builddir}/../debuginfod/debuginfod $VERBOSE -F -R -d $DB -p $PORT1 -t0 -g0 -v R F > vlog$PORT1 2>&1 &
+PID1=$!
+tempfiles vlog$PORT1
+errfiles vlog$PORT1
+# Server must become ready
+wait_ready $PORT1 'ready' 1
+export DEBUGINFOD_URLS=http://127.0.0.1:$PORT1/   # or without trailing /
+# Check thread comm names
+ps -q $PID1 -e -L -o '%p %c %a' | grep groom
+ps -q $PID1 -e -L -o '%p %c %a' | grep scan
+ps -q $PID1 -e -L -o '%p %c %a' | grep traverse
+
+# We use -t0 and -g0 here to turn off time-based scanning & grooming.
+# For testing purposes, we just sic SIGUSR1 / SIGUSR2 at the process.
+
+########################################################################
+
+# Compile a simple program, strip its debuginfo and save the build-id.
+# Also move the debuginfo into another directory so that elfutils
+# cannot find it without debuginfod.
+echo "int main() { return 0; }" > ${PWD}/prog.c
+tempfiles prog.c
+# Create a subdirectory to confound source path names
+mkdir foobar
+gcc -Wl,--build-id -g -o prog ${PWD}/foobar///./../prog.c
+testrun ${abs_top_builddir}/src/strip -g -f prog.debug ${PWD}/prog
+BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
+          -a prog | grep 'Build ID' | cut -d ' ' -f 7`
+
+mv prog F
+mv prog.debug F
+kill -USR1 $PID1
+# Wait till both files are in the index.
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 1
+wait_ready $PORT1 'thread_work_pending{role="scan"}' 0
+wait_ready $PORT1 'thread_busy{role="scan"}' 0
+
+cp -rvp ${abs_srcdir}/debuginfod-rpms R
+if [ "$zstd" = "false" ]; then  # nuke the zstd fedora 31 ones
+    rm -vrf R/debuginfod-rpms/fedora31
+fi
+
+kill -USR1 $PID1
+# Wait till both files are in the index and scan/index fully finished
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 2
+wait_ready $PORT1 'thread_work_pending{role="scan"}' 0
+wait_ready $PORT1 'thread_busy{role="scan"}' 0
+# All rpms need to be in the index, except the dummy permission-000 one
+rpms=$(find R -name \*rpm | grep -v nothing | wc -l)
+wait_ready $PORT1 'scanned_files_total{source=".rpm archive"}' $rpms
+kill -USR1 $PID1  # two hits of SIGUSR1 may be needed to resolve .debug->dwz->srefs
+# Wait till both files are in the index and scan/index fully finished
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 3
+wait_ready $PORT1 'thread_work_pending{role="scan"}' 0
+wait_ready $PORT1 'thread_busy{role="scan"}' 0
+
+########################################################################
+## PR27277
+# Make a simple request to the debuginfod server and check debuginfod-find's vlog to see if
+# the custom HTTP headers are received.
+rm -rf $DEBUGINFOD_CACHE_PATH
+env DEBUGINFOD_URLS="http://127.0.0.1:"$PORT1 LD_LIBRARY_PATH=$ldpath ${abs_top_builddir}/debuginfod/debuginfod-find\
+    -vvv executable F/prog > vlog-find$PORT1.1 2>&1
+tempfiles vlog-find$PORT1.1
+cat vlog-find$PORT1.1
+#grep 'Content-Length: ' vlog-find$PORT1.1
+#grep 'Connection: ' vlog-find$PORT1.1
+#grep 'Cache-Control: ' vlog-find$PORT1.1
+
+# Check to see if an executable file located in an archive prints the file's description and archive
+env DEBUGINFOD_URLS="http://127.0.0.1:"$PORT1 LD_LIBRARY_PATH=$ldpath ${abs_top_builddir}/debuginfod/debuginfod-find\
+    -vvv executable c36708a78618d597dee15d0dc989f093ca5f9120 > vlog-find$PORT1.2 2>&1
+tempfiles vlog-find$PORT1.2
+grep 'Content-Length: ' vlog-find$PORT1.2
+grep 'Connection: ' vlog-find$PORT1.2
+grep 'Cache-Control: ' vlog-find$PORT1.2
+
-- 
2.31.1


^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2022-08-08 14:35 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-04 18:54 [Bug debuginfod/27277] Describe retrieved files when verbose Noah Sanci
2021-08-05 15:13 ` Mark Wielaard
2021-08-05 16:54   ` Frank Ch. Eigler
2021-08-06 10:04     ` Mark Wielaard
2021-08-06 18:54       ` Frank Ch. Eigler
2021-08-09  9:25         ` Mark Wielaard
2021-08-23 15:11           ` Noah Sanci
2021-08-24  8:18             ` Mark Wielaard
2021-08-27 18:38               ` Noah Sanci
2021-09-08 20:56                 ` Mark Wielaard
2021-09-10 18:22                   ` Noah Sanci
2021-09-12 19:08                     ` Mark Wielaard
2021-09-13 20:07                       ` Noah Sanci
2021-09-16 10:50                         ` Mark Wielaard
2021-09-22 20:33           ` Frank Ch. Eigler
2021-09-29 14:55             ` Mark Wielaard
2021-09-29 21:28               ` Frank Ch. Eigler
2021-10-05 14:28                 ` Mark Wielaard
2022-07-14 15:32                   ` Noah Sanci
2022-08-04 13:12                     ` Mark Wielaard
     [not found]                       ` <CAJXA7qg09YkxK-NRQ31Hem0+54Us=jYC5+1siPSbHangx=SCow@mail.gmail.com>
2022-08-08 14:35                         ` Mark Wielaard
2021-08-25 18:08 Noah Sanci
2021-09-08 15:01 ` Mark Wielaard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).