[PATCH 2/2] new example: capture_ssl_master_secrets.stp

[PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Timo Juhani Lindfors]

Tested to work with Debian 9 openssl and gnutls.
---
 .../io/capture_ssl_master_secrets.meta             | 11 ++++++++
 .../io/capture_ssl_master_secrets.stp              | 33 ++++++++++++++++++++++
 .../io/capture_ssl_master_secrets.txt              | 22 +++++++++++++++
 3 files changed, 66 insertions(+)
 create mode 100644 testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
 create mode 100755 testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
 create mode 100644 testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt

diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
new file mode 100644
index 000000000..2ac462734
--- /dev/null
+++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
@@ -0,0 +1,11 @@
+title: Capture SSL/TLS master secrets from gnutls and openssl users
+name: capture_ssl_master_secrets.stp
+version: 1.0
+author: Timo Juhani Lindfors <timo.lindfors@iki.fi>
+keywords: io monitoring
+application: openssl gnutls
+status: alpha
+exit: user-controlled
+output: trace
+scope: system-wide
+description: Capture SSL/TLS master secrets from gnutls and openssl users for easy traffic decryption
diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
new file mode 100755
index 000000000..e021fd9bd
--- /dev/null
+++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
@@ -0,0 +1,33 @@
+#!/usr/bin/env stap
+
+function cast_char_to_unsigned_char (val:long) {
+  if (val < 0) {
+    return val + 256;
+  } else {
+    return val;
+  }
+}
+
+function print_buffer (buf:long, len:long) {
+  for (i = 0; i < len; i++) {
+    printf("%02x", cast_char_to_unsigned_char(user_char(buf + i)));
+  }
+}
+
+probe process("/usr/lib/*/libssl.so.*").function("tls1_generate_master_secret").return {
+  printf("# %d %s %s (%d)\n", gettimeofday_us(), pp(), execname(), pid());
+  printf("CLIENT_RANDOM ");
+  print_buffer(@entry($s->s3->client_random), 32);
+  printf(" ");
+  print_buffer(@entry($out), $return);
+  printf("\n");
+}
+
+probe process("/usr/lib/*/libgnutls.so.*").function("generate_normal_master").return {
+  printf("# %d %s %s (%d)\n", gettimeofday_us(), pp(), execname(), pid());
+  printf("CLIENT_RANDOM ");
+  print_buffer(@entry($session) + 72, 32);
+  printf(" ");
+  print_buffer(@entry($session) + 24, 48);
+  printf("\n");
+}
diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt
new file mode 100644
index 000000000..d9c20d45f
--- /dev/null
+++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt
@@ -0,0 +1,22 @@
+Capture SSL/TLS master secrets from gnutls and openssl users
+
+The following usage example has been tested to work on Debian 9:
+
+$ sudo stap-prep
+$ sudo apt-get install libgnutls30-dbgsym libssl1.0.2-dbgsym libssl1.1-dbgsym libssl-dev
+$ ./capture_ssl_master_secrets.stp | tee keylog.txt &
+$ sudo tcpdump -s0 -w traffic.pcap -U port 443 &
+$ curl https://www.ssllabs.com/curl_secret
+$ wget https://www.ssllabs.com/wget_secret
+$ echo "GET /sclient_secret HTTP/1.1\nHost: www.ssllabs.com\n\n" | openssl s_client -connect www.ssllabs.com:443 -servername www.ssllabs.com
+$ cat keylog.txt
+# 1509378583063892 process("/usr/lib/x86_64-linux-gnu/libssl.so.1.0.2").function("tls1_generate_master_secret@./ssl/t1_enc.c:1134").return curl (24745)
+CLIENT_RANDOM 924207933a2eda5d90ccd2552a620924c6cd12bf72036ced2227bfc0016152ad 9bffacb095403182e9a2f515851d3fa49838b93599de6507230bac0c0666c29d140588739635d4ad19bdfd4fced69000
+# 1509378587558501 process("/usr/lib/x86_64-linux-gnu/libgnutls.so.30.13.1").function("generate_normal_master@./lib/kx.c:131").return wget (24755)
+CLIENT_RANDOM 59f74aa0d72f90753e989d049953deb9fc6479a2c7091936520d280a4b1be28a 5604af95f156eaa21a93f6982c1de24289b86dac9331e0080bfc4b1a67ab13535f03c7d50530e5b3f8cd572b5d8967c8
+# 1509378592611222 process("/usr/lib/x86_64-linux-gnu/libssl.so.1.1").function("tls1_generate_master_secret@../ssl/t1_enc.c:463").return openssl (24757)
+CLIENT_RANDOM aa211423644611d7b52f254e44e55c3919a48d81cc0a7f0c6af604190720fc93 74150d7854157f7e6b01e40238641d065c37d7f931bac6a14aa9fac6a44b1ea7da0943f15714039acc3f71077c21127a
+$ tshark -o ssl.keylog_file:keylog.txt -d tcp.port==443,ssl -x -r traffic.pcap -V | grep -A1 'Decrypted SSL data' |grep "GET "
+0000  47 45 54 20 2f 63 75 72 6c 5f 73 65 63 72 65 74   GET /curl_secret
+0000  47 45 54 20 2f 77 67 65 74 5f 73 65 63 72 65 74   GET /wget_secret
+0000  47 45 54 20 2f 73 63 6c 69 65 6e 74 5f 73 65 63   GET /sclient_sec
-- 
2.11.0

[PATCH 1/2] Match non-executable shared libraries in glob patterns

[Timo Juhani Lindfors]

Shared libraries in Debian do not have executable permissions. This
patch makes glob patterns work for those. Debian policy 8.1:

"Shared libraries should not be installed executable, since the dynamic
linker does not require this and trying to execute a shared library
usually results in a core dump."
---
 tapsets.cxx | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tapsets.cxx b/tapsets.cxx
index 36eb73c08..f571b8825 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -8054,8 +8054,7 @@ glob_executable(const string& pattern)
       const char* globbed = the_blob.gl_pathv[i];
       struct stat st;
 
-      if (access (globbed, X_OK) == 0
-          && stat (globbed, &st) == 0
+      if (stat (globbed, &st) == 0
           && S_ISREG (st.st_mode)) // see find_executable()
         {
           // Need to call resolve_path here, in order to path-expand
-- 
2.11.0

Re: [PATCH 1/2] Match non-executable shared libraries in glob patterns

[David Smith]

Checked in as commit c539cb3d7.

Thanks!

On Mon, Oct 30, 2017 at 11:09 AM, Timo Juhani Lindfors
<timo.lindfors@iki.fi> wrote:
> Shared libraries in Debian do not have executable permissions. This
> patch makes glob patterns work for those. Debian policy 8.1:
>
> "Shared libraries should not be installed executable, since the dynamic
> linker does not require this and trying to execute a shared library
> usually results in a core dump."
> ---
>  tapsets.cxx | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/tapsets.cxx b/tapsets.cxx
> index 36eb73c08..f571b8825 100644
> --- a/tapsets.cxx
> +++ b/tapsets.cxx
> @@ -8054,8 +8054,7 @@ glob_executable(const string& pattern)
>        const char* globbed = the_blob.gl_pathv[i];
>        struct stat st;
>
> -      if (access (globbed, X_OK) == 0
> -          && stat (globbed, &st) == 0
> +      if (stat (globbed, &st) == 0
>            && S_ISREG (st.st_mode)) // see find_executable()
>          {
>            // Need to call resolve_path here, in order to path-expand
> --
> 2.11.0
>



-- 
David Smith
Principal Software Engineer
Red Hat

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[David Smith]

Thanks! I worked on the meta file a bit and checked it in as commit 327716850.

On Mon, Oct 30, 2017 at 11:09 AM, Timo Juhani Lindfors
<timo.lindfors@iki.fi> wrote:
> Tested to work with Debian 9 openssl and gnutls.
> ---
>  .../io/capture_ssl_master_secrets.meta             | 11 ++++++++
>  .../io/capture_ssl_master_secrets.stp              | 33 ++++++++++++++++++++++
>  .../io/capture_ssl_master_secrets.txt              | 22 +++++++++++++++
>  3 files changed, 66 insertions(+)
>  create mode 100644 testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
>  create mode 100755 testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
>  create mode 100644 testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt
>
> diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
> new file mode 100644
> index 000000000..2ac462734
> --- /dev/null
> +++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.meta
> @@ -0,0 +1,11 @@
> +title: Capture SSL/TLS master secrets from gnutls and openssl users
> +name: capture_ssl_master_secrets.stp
> +version: 1.0
> +author: Timo Juhani Lindfors <timo.lindfors@iki.fi>
> +keywords: io monitoring
> +application: openssl gnutls
> +status: alpha
> +exit: user-controlled
> +output: trace
> +scope: system-wide
> +description: Capture SSL/TLS master secrets from gnutls and openssl users for easy traffic decryption
> diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
> new file mode 100755
> index 000000000..e021fd9bd
> --- /dev/null
> +++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.stp
> @@ -0,0 +1,33 @@
> +#!/usr/bin/env stap
> +
> +function cast_char_to_unsigned_char (val:long) {
> +  if (val < 0) {
> +    return val + 256;
> +  } else {
> +    return val;
> +  }
> +}
> +
> +function print_buffer (buf:long, len:long) {
> +  for (i = 0; i < len; i++) {
> +    printf("%02x", cast_char_to_unsigned_char(user_char(buf + i)));
> +  }
> +}
> +
> +probe process("/usr/lib/*/libssl.so.*").function("tls1_generate_master_secret").return {
> +  printf("# %d %s %s (%d)\n", gettimeofday_us(), pp(), execname(), pid());
> +  printf("CLIENT_RANDOM ");
> +  print_buffer(@entry($s->s3->client_random), 32);
> +  printf(" ");
> +  print_buffer(@entry($out), $return);
> +  printf("\n");
> +}
> +
> +probe process("/usr/lib/*/libgnutls.so.*").function("generate_normal_master").return {
> +  printf("# %d %s %s (%d)\n", gettimeofday_us(), pp(), execname(), pid());
> +  printf("CLIENT_RANDOM ");
> +  print_buffer(@entry($session) + 72, 32);
> +  printf(" ");
> +  print_buffer(@entry($session) + 24, 48);
> +  printf("\n");
> +}
> diff --git a/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt
> new file mode 100644
> index 000000000..d9c20d45f
> --- /dev/null
> +++ b/testsuite/systemtap.examples/io/capture_ssl_master_secrets.txt
> @@ -0,0 +1,22 @@
> +Capture SSL/TLS master secrets from gnutls and openssl users
> +
> +The following usage example has been tested to work on Debian 9:
> +
> +$ sudo stap-prep
> +$ sudo apt-get install libgnutls30-dbgsym libssl1.0.2-dbgsym libssl1.1-dbgsym libssl-dev
> +$ ./capture_ssl_master_secrets.stp | tee keylog.txt &
> +$ sudo tcpdump -s0 -w traffic.pcap -U port 443 &
> +$ curl https://www.ssllabs.com/curl_secret
> +$ wget https://www.ssllabs.com/wget_secret
> +$ echo "GET /sclient_secret HTTP/1.1\nHost: www.ssllabs.com\n\n" | openssl s_client -connect www.ssllabs.com:443 -servername www.ssllabs.com
> +$ cat keylog.txt
> +# 1509378583063892 process("/usr/lib/x86_64-linux-gnu/libssl.so.1.0.2").function("tls1_generate_master_secret@./ssl/t1_enc.c:1134").return curl (24745)
> +CLIENT_RANDOM 924207933a2eda5d90ccd2552a620924c6cd12bf72036ced2227bfc0016152ad 9bffacb095403182e9a2f515851d3fa49838b93599de6507230bac0c0666c29d140588739635d4ad19bdfd4fced69000
> +# 1509378587558501 process("/usr/lib/x86_64-linux-gnu/libgnutls.so.30.13.1").function("generate_normal_master@./lib/kx.c:131").return wget (24755)
> +CLIENT_RANDOM 59f74aa0d72f90753e989d049953deb9fc6479a2c7091936520d280a4b1be28a 5604af95f156eaa21a93f6982c1de24289b86dac9331e0080bfc4b1a67ab13535f03c7d50530e5b3f8cd572b5d8967c8
> +# 1509378592611222 process("/usr/lib/x86_64-linux-gnu/libssl.so.1.1").function("tls1_generate_master_secret@../ssl/t1_enc.c:463").return openssl (24757)
> +CLIENT_RANDOM aa211423644611d7b52f254e44e55c3919a48d81cc0a7f0c6af604190720fc93 74150d7854157f7e6b01e40238641d065c37d7f931bac6a14aa9fac6a44b1ea7da0943f15714039acc3f71077c21127a
> +$ tshark -o ssl.keylog_file:keylog.txt -d tcp.port==443,ssl -x -r traffic.pcap -V | grep -A1 'Decrypted SSL data' |grep "GET "
> +0000  47 45 54 20 2f 63 75 72 6c 5f 73 65 63 72 65 74   GET /curl_secret
> +0000  47 45 54 20 2f 77 67 65 74 5f 73 65 63 72 65 74   GET /wget_secret
> +0000  47 45 54 20 2f 73 63 6c 69 65 6e 74 5f 73 65 63   GET /sclient_sec
> --
> 2.11.0
>



-- 
David Smith
Principal Software Engineer
Red Hat

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Timo Lindfors]

Thanks for the improvements! Do you perhaps have any other comments about 
the example? Could we have something like print_buffer in a standard tapset? I 
need to print binary buffers quite often.

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Frank Ch. Eigler]

timo.lindfors wrote:

> [...] Could we have something like print_buffer in a standard tapset?
> I need to print binary buffers quite often.

See the {user,kernel}_buffer_quoted tapset functions.

- FChE

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Timo Lindfors]

Hmm. That seems to call _stp_text_str which calls _stp_decode_utf8 which is completely unnecessary for printing binary data?

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Frank Ch. Eigler]

timo.lindfors wrote:

> Hmm. That seems to call _stp_text_str which calls _stp_decode_utf8
> which is completely unnecessary for printing binary data?

I wouldn't read too much into the utf8 angle here.  Those functions make
an effort to quote the buffer into readable text.  If you want a
hexdump, there's printf("%*.M", buffer, size).  Maybe we could have a
function that prints an old-school hex/ascii side-by-side dump?

- FChE

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Timo Lindfors]

On Tue, 31 Oct 2017, Frank Ch. Eigler wrote:
> I wouldn't read too much into the utf8 angle here.  Those functions make
> an effort to quote the buffer into readable text.  If you want a
> hexdump, there's printf("%*.M", buffer, size).  Maybe we could have a
> function that prints an old-school hex/ascii side-by-side dump?

Interesting, I didn't know about that format specifier. Can I use it for 
accessing userland buffers? Could it be more optimal than user_char()? I 
mainly need this data for easy post processing in e.g. python, I don't 
really care about human-readability.

-Timo

Re: [PATCH 2/2] new example: capture_ssl_master_secrets.stp

[Frank Ch. Eigler]

Hi -

> >[...] there's printf("%*.M", buffer, size).  Maybe we could have a
> >function that prints an old-school hex/ascii side-by-side dump?
> 
> Interesting, I didn't know about that format specifier. Can I use it
> for accessing userland buffers? [...]

Certainly.  As listed in [man stap], printf("%#*.M", user_buffer, size)

- FChE