From: Noah Sanci <nsanci@redhat.com>
To: Mark Wielaard <mark@klomp.org>, elfutils-devel@sourceware.org
Subject: Re: [Bug debuginfod/28034] client-side %-escape url characters
Date: Mon, 13 Sep 2021 12:20:29 -0400 [thread overview]
Message-ID: <CAJXA7qgb1Z9BoKVbE-Tc8BaGm8wEDd9_Ot6tCF7aU_JQQev4uQ@mail.gmail.com> (raw)
In-Reply-To: <YT433jt593F8OUjx@wildebeest.org>
[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]
Hello,
On Sun, Sep 12, 2021 at 1:24 PM Mark Wielaard <mark@klomp.org> wrote:
> The escaped_string is created outside the loop and reused each time
> (good). But...
> > +
> > /* Initialize each handle. */
> > for (int i = 0; i < num_urls; i++)
> > {
> > @@ -904,16 +908,23 @@ debuginfod_query_server (debuginfod_client *c,
> > if (filename) /* must start with / */
> > {
> > /* PR28034 escape characters in completed url to %hh format. */
> > - char *escaped_string;
> > - escaped_string = curl_easy_escape(data[i].handle, filename, 0);
> > + char *loc = escaped_string;
> > if (!escaped_string)
> > {
> > rc = -ENOMEM;
> > goto out2;
> > }
>
> This check, and...
>
> > +
> > + size_t escaped_strlen = strlen(escaped_string);
> > + while ((loc = strstr(loc, "%2F")))
> > + {
> > + loc[0] = '/';
> > + // pull the string back after replacement
> > + memmove(loc+1, loc+3,escaped_strlen - (loc - escaped_string + 2) );
> > + escaped_strlen -=2;
> > + }
>
> the manipulation of the escaped_string, could both also be done
> outside the loop, since they always do the same thing.
Fixed.
>
> I think you should simply add a new ChangeLog entry at the top instead
> of changing an old existing one. And please do mention the Makefile.am
> (TESTS) and (EXTRA_DIST) addition.
Should be good.
> OK, that is certainly a file name with lots of unexpected characters :)
Gotta be sure :)
If there are any more efficiency changes, ChangeLog changes, or other
changes, don't be afraid to reach out again.
Thanks,
Noah Sanci
[-- Attachment #2: 0001-debuginfod-PR28034-Percent-escape-debuginfod-urls.patch --]
[-- Type: text/x-patch, Size: 8522 bytes --]
From fa9104c7435503d08308e18d4e1a82792f83fc56 Mon Sep 17 00:00:00 2001
From: Noah Sanci <nsanci@redhat.com>
Date: Thu, 9 Sep 2021 13:10:33 -0400
Subject: [PATCH] debuginfod: PR28034 - Percent escape debuginfod urls
When requesting some source files, some URL-inconvenient chars
sometimes pop up, including but not limited to '+', '^', and '&'.
Example from f33 libstdc++:
/buildid/44d8485cb75512c2ca5c8f70afbd475cae30af4f/source/usr/src/debug
/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/
libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/
condition_variable.cc
As this URL is passed into debuginfod's handler_cb, it appears that the
+ signs are helpfully unescaped to spaces by libmicrohttpd, which
'course breaks everything.
To combat this, before querying the debuginfod daemon, clients now %
escape the source filename. This converts many alphanumeric characters
into their %-code format, including '/' to %2F. We want to preserve the
'/' in the url, so after conversion replace %2Fs with a '/'.
https://sourceware.org/bugzilla/show_bug.cgi?id=28034
Signed-off-by: Noah Sanci <nsanci@redhat.com>
---
debuginfod/ChangeLog | 4 ++
debuginfod/debuginfod-client.c | 29 +++++++++----
tests/ChangeLog | 12 +++---
tests/Makefile.am | 2 +
tests/run-debuginfod-percent-escape.sh | 60 ++++++++++++++++++++++++++
5 files changed, 94 insertions(+), 13 deletions(-)
create mode 100755 tests/run-debuginfod-percent-escape.sh
diff --git a/debuginfod/ChangeLog b/debuginfod/ChangeLog
index 1173f9cd..b8eaa88c 100644
--- a/debuginfod/ChangeLog
+++ b/debuginfod/ChangeLog
@@ -2,6 +2,10 @@
* debuginfod.cxx (libarchive_fdcache::lookup): Add endl after
obatched(clog) line.
+2021-09-13 Noah Sanci <nsanci@redhat.com>
+
+ * debuginfod-client.c (debuginfod_query_server): Removed constant
+ operations from a loop. curl_free memory.
2021-09-06 Dmitry V. Levin <ldv@altlinux.org>
diff --git a/debuginfod/debuginfod-client.c b/debuginfod/debuginfod-client.c
index d41723ce..f55e1489 100644
--- a/debuginfod/debuginfod-client.c
+++ b/debuginfod/debuginfod-client.c
@@ -883,6 +883,26 @@ debuginfod_query_server (debuginfod_client *c,
data[i].errbuf[0] = '\0';
}
+ char *escaped_string = NULL;
+ size_t escaped_strlen = 0;
+ if (filename)
+ {
+ escaped_string = curl_easy_escape(&target_handle, filename+1, 0);
+ if (!escaped_string)
+ {
+ rc = -ENOMEM;
+ goto out2;
+ }
+ char *loc = escaped_string;
+ escaped_strlen = strlen(escaped_string) - 2 + (size_t)escaped_string;
+ while ((loc = strstr(loc, "%2F")))
+ {
+ loc[0] = '/';
+ // pull the string back after replacement
+ memmove(loc+1, loc+3,escaped_strlen - (size_t)loc );
+ escaped_strlen -=2;
+ }
+ }
/* Initialize each handle. */
for (int i = 0; i < num_urls; i++)
{
@@ -904,16 +924,8 @@ debuginfod_query_server (debuginfod_client *c,
if (filename) /* must start with / */
{
/* PR28034 escape characters in completed url to %hh format. */
- char *escaped_string;
- escaped_string = curl_easy_escape(data[i].handle, filename, 0);
- if (!escaped_string)
- {
- rc = -ENOMEM;
- goto out2;
- }
snprintf(data[i].url, PATH_MAX, "%s/%s/%s/%s", server_url,
build_id_bytes, type, escaped_string);
- curl_free(escaped_string);
}
else
snprintf(data[i].url, PATH_MAX, "%s/%s/%s", server_url, build_id_bytes, type);
@@ -953,6 +965,7 @@ debuginfod_query_server (debuginfod_client *c,
curl_multi_add_handle(curlm, data[i].handle);
}
+ if (filename) curl_free(escaped_string);
/* Query servers in parallel. */
if (vfd >= 0)
dprintf (vfd, "query %d urls in parallel\n", num_urls);
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 1154686a..3f219320 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -30,6 +30,11 @@
* run-debuginfod-federation-metrics.sh: Likewise.
* run-debuginfod-federation-sqlite.sh: Likewise.
+2021-09-13 Noah Sanci <nsanci@redhat.com>
+
+ * Makefile.am: added run-debuginfod-percent-escape.sh to TESTS and
+ EXTRA_DIST.
+
2021-09-06 Dmitry V. Levin <ldv@altlinux.org>
* elfcopy.c (copy_elf): Remove cast of malloc return value.
@@ -164,11 +169,8 @@
2021-07-16 Noah Sanci <nsanci@redhat.com>
PR28034
- * run-debuginfod-find.sh: Added a test ensuring files with %
- escapable characters in their paths are accessible. The test
- itself is changing the name of a binary known previously as prog to
- p+r%o$g. General operations such as accessing p+r%o$g acts as the
- test for %-escape checking.
+ * run-debuginfod-percent-escape.sh: Added a test ensuring files with %
+ escapable characters in their paths are accessible.
2021-07-21 Noah Sanci <nsanci@redhat.com>
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 22942733..43c34ce6 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -231,6 +231,7 @@ TESTS += run-debuginfod-dlopen.sh \
run-debuginfod-federation-sqlite.sh \
run-debuginfod-federation-link.sh \
run-debuginfod-federation-metrics.sh \
+ run-debuginfod-percent-escape.sh \
run-debuginfod-x-forwarded-for.sh
endif
endif
@@ -524,6 +525,7 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
run-debuginfod-archive-groom.sh \
run-debuginfod-archive-rename.sh \
run-debuginfod-archive-test.sh \
+ run-debuginfod-percent-escape.sh \
debuginfod-rpms/fedora30/hello2-1.0-2.src.rpm \
debuginfod-rpms/fedora30/hello2-1.0-2.x86_64.rpm \
debuginfod-rpms/fedora30/hello2-debuginfo-1.0-2.x86_64.rpm \
diff --git a/tests/run-debuginfod-percent-escape.sh b/tests/run-debuginfod-percent-escape.sh
new file mode 100755
index 00000000..f7d8dc66
--- /dev/null
+++ b/tests/run-debuginfod-percent-escape.sh
@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+#
+# Copyright (C) 2019-2021 Red Hat, Inc.
+# This file is part of elfutils.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# elfutils is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+. $srcdir/debuginfod-subr.sh # includes set -e
+# for test case debugging, uncomment:
+set -x
+unset VALGRIND_CMD
+# This variable is essential and ensures no time-race for claiming ports occurs
+# set base to a unique multiple of 100 not used in any other 'run-debuginfod-*' test
+base=10000
+get_ports
+DB=${PWD}/.debuginfod_tmp.sqlite
+tempfiles $DB
+mkdir F
+env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../debuginfod/debuginfod $VERBOSE \
+ -F -R -d $DB -p $PORT1 -t0 -g0 -v R ${PWD}/F > vlog$PORT1 2>&1 &
+PID1=$!
+tempfiles vlog$PORT1
+errfiles vlog$PORT1
+# Server must become ready
+wait_ready $PORT1 'ready' 1
+# Be patient when run on a busy machine things might take a bit.
+
+# Build a non-stripped binary
+echo "int main() { return 0; }" > ${PWD}/F/p++r\$\#o^^g.c
+gcc -Wl,--build-id -g -o ${PWD}/F/p++r\$\#o^^g ${PWD}/F/p++r\$\#o^^g.c
+BUILDID=`env LD_LIBRARY_PATH=$ldpath ${abs_builddir}/../src/readelf \
+ -a ${PWD}/F/p++r\\$\#o^^g | grep 'Build ID' | cut -d ' ' -f 7`
+tempfiles ${PWD}/F/p++r\$\#o^^g.c ${PWD}/F/p++r\$\#o^^g
+kill -USR1 $PID1
+# Now there should be 1 files in the index
+wait_ready $PORT1 'thread_work_total{role="traverse"}' 1
+wait_ready $PORT1 'thread_work_pending{role="scan"}' 0
+wait_ready $PORT1 'thread_busy{role="scan"}' 0
+rm -rf $DEBUGINFOD_CACHE_PATH # clean it from previous tests
+ls F
+env DEBUGINFOD_CACHE_PATH=${PWD}/.client_cache DEBUGINFOD_URLS="http://127.0.0.1:$PORT1" \
+ LD_LIBRARY_PATH=$ldpath ${abs_top_builddir}/debuginfod/debuginfod-find -vvv source F/p++r\$\#o^^g ${abs_builddir}/F/p++r\$\#o^^g.c > vlog1 2>&1 || true
+tempfiles vlog1
+grep 'F/p%2B%2Br%24%23o%5E%5Eg.c' vlog1
+
+kill $PID1
+wait $PID1
+PID1=0
+exit 0
--
2.31.1
next prev parent reply other threads:[~2021-09-13 16:20 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-26 19:27 Noah Sanci
2021-08-26 21:02 ` Frank Ch. Eigler
2021-08-27 14:44 ` Noah Sanci
2021-08-27 15:07 ` Noah Sanci
2021-08-27 15:30 ` Noah Sanci
2021-09-08 13:38 ` Mark Wielaard
2021-09-09 17:28 ` Noah Sanci
2021-09-12 17:24 ` Mark Wielaard
2021-09-13 16:20 ` Noah Sanci [this message]
2021-09-13 18:11 ` Noah Sanci
2021-09-16 10:35 ` Mark Wielaard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJXA7qgb1Z9BoKVbE-Tc8BaGm8wEDd9_Ot6tCF7aU_JQQev4uQ@mail.gmail.com \
--to=nsanci@redhat.com \
--cc=elfutils-devel@sourceware.org \
--cc=mark@klomp.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).