Re: problems with sysroot variable when cross compiling uprobes

Re: problems with sysroot variable when cross compiling uprobes

[Holger Brunck]

Hi Torsten,

On 03/01/2016 08:59 PM, Torsten Polle wrote:
>  > I am working on a powerpc board with 32 bit architecture and try to
>  > compile userspace probes. This means I have my own cross compiled
>  > root filesystem and I am using stap then with the --sysroot option.
> 
>  > I had a setup up and running with systemtap version 2.5 for a longer
>  > time. Now I tried to upgrade my sytemtap version to 2.9. But already
>  > with version 2.6 I see the following problem: When I try to compile
>  > my stap probe it complains with an error message that the executable
>  > can not be found. And what I see that the path I specified for my
>  > sysroot variable is duplicated.
> 
> [...]
> 
>  > As I said it works fine with an own build systemtap 2.5 but does not
>  > work with an own build systemtap 2.9. Also the official installed
>  > stap version 2.8 on my host system shows the same error.
> 
>  > Am I am doing something wrong here or has anybody seen the same
>  > problem? Any help is appreciated.
> 
> I'm using a cross compile environment for ARM. For me everything works
> fine. But I'm using a number of patches to make everything in my

great, the third patch definitely solves the problem I have on my side and I can
use systemtap 2.9 in my powerpc environment. Thanks a lot.

> environment. I never dared to upstream those because my knowledge of the
> SystemTap code itself is very poor and might be wrong for the general
> case. If they also work fine for you, the patches might be a starting

I am also not familar with the code. But it might be worth to try to upstream
it. Even if it might be not a perfect solutions for the general case, others
around may then propose clean solutions which are ready to be merged upstream. I
mean the sysroot option seems to be broken. Or is anyone around here who can
work with the upstream version in a cross compiled environment and uses this
option successfully?

Best regards
Holger Brunck

Re: problems with sysroot variable when cross compiling uprobes

[David Smith]

On 03/02/2016 03:38 AM, Holger Brunck wrote:
> Hi Torsten,

... stuff deleted ...

>> environment. I never dared to upstream those because my knowledge of the
>> SystemTap code itself is very poor and might be wrong for the general
>> case. If they also work fine for you, the patches might be a starting
>
> I am also not familar with the code. But it might be worth to try to upstream
> it. Even if it might be not a perfect solutions for the general case, others
> around may then propose clean solutions which are ready to be merged upstream. I
> mean the sysroot option seems to be broken. Or is anyone around here who can
> work with the upstream version in a cross compiled environment and uses this
> option successfully?

Always feel free to send a patch, even a rough patch, to the list. We'll
try to work with you to polish it up.

The problem with the sysroot code is that the main systemtap developers
don't tend to use it very often. So, when we do find a need for it
(typically with arm 32-bit), we'll fix it up and get it working again.
Time passes, something changes and sysroot gets broken again. We find a
need for it again, fix it up, ... the cycle goes on and on.

Another problem with the sysroot code is that we could certainly use
better tests for it. That way we'd know when it is broken and try to
keep it working and up to date. Even if you don't feel like you could
write a test case, having repeatable steps to follow would be helpful.
Setting up the proper environment might be the trickiest part.

Basically the sysroot code *should* work, but tends to slowly break over
time from lack of use. We'll happily look at patches and testing ideas
to get things working again here (and keep them working).

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: problems with sysroot variable when cross compiling uprobes

[Torsten Polle]

[-- Attachment #1: Type: text/plain, Size: 2491 bytes --]

Dear all,

my original mail was classified as spam. As a different mail got accepted, I hope the problems are fixed now.

> Am 02.03.2016 um 10:38 schrieb Holger Brunck <holger.brunck@keymile.com>:
> 
> Hi Torsten,
> 
> On 03/01/2016 08:59 PM, Torsten Polle wrote:
>>> I am working on a powerpc board with 32 bit architecture and try to
>>> compile userspace probes. This means I have my own cross compiled
>>> root filesystem and I am using stap then with the --sysroot option.
>> 
>>> I had a setup up and running with systemtap version 2.5 for a longer
>>> time. Now I tried to upgrade my sytemtap version to 2.9. But already
>>> with version 2.6 I see the following problem: When I try to compile
>>> my stap probe it complains with an error message that the executable
>>> can not be found. And what I see that the path I specified for my
>>> sysroot variable is duplicated.
>> 
>> [...]
>> 
>>> As I said it works fine with an own build systemtap 2.5 but does not
>>> work with an own build systemtap 2.9. Also the official installed
>>> stap version 2.8 on my host system shows the same error.
>> 
>>> Am I am doing something wrong here or has anybody seen the same
>>> problem? Any help is appreciated.
>> 
>> I'm using a cross compile environment for ARM. For me everything works
>> fine. But I'm using a number of patches to make everything in my
> 
> great, the third patch definitely solves the problem I have on my side and I can
> use systemtap 2.9 in my powerpc environment. Thanks a lot.
> 
>> environment. I never dared to upstream those because my knowledge of the
>> SystemTap code itself is very poor and might be wrong for the general
>> case. If they also work fine for you, the patches might be a starting
> 
> I am also not familar with the code. But it might be worth to try to upstream
> it. Even if it might be not a perfect solutions for the general case, others
> around may then propose clean solutions which are ready to be merged upstream. I
> mean the sysroot option seems to be broken. Or is anyone around here who can
> work with the upstream version in a cross compiled environment and uses this
> option successfully?
> 
> Best regards
> Holger Brunck

Enclosed are four patches I use to be able to use SystemTap in a cross compile environment. As my knowledge about the internals of SystemTap are limited, they might be a bit rough. Any comments to improve them are welcome.

Kind Regards,
Torsten


[-- Attachment #2: 0001-Support-the-sysroot-option-for-modules-added-for-sym.patch --]
[-- Type: application/octet-stream, Size: 2318 bytes --]

From 1b535eda5c320271d25b1b511cd5a4eb2ad9d575 Mon Sep 17 00:00:00 2001
Message-Id: <1b535eda5c320271d25b1b511cd5a4eb2ad9d575.1456861753.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Fri, 13 Dec 2013 23:26:07 +0100
Subject: [PATCH] Support the sysroot option for modules added for symbol/unwind information

The sysroot option failed for modules added for their symbol/unwind
information. Those user module names are prepended by sysroot.

Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 session.cxx |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/session.cxx b/session.cxx
index 10b36d0..dd99284 100644
--- a/session.cxx
+++ b/session.cxx
@@ -15,6 +15,7 @@
 #include "buildrun.h"
 #include "coveragedb.h"
 #include "hash.h"
+#include "setupdwfl.h"
 #include "task_finder.h"
 #include "csclient.h"
 #include "rpm_finder.h"
@@ -689,6 +690,7 @@ int
 systemtap_session::parse_cmdline (int argc, char * const argv [])
 {
   client_options_disallowed_for_unprivileged = "";
+  std::set<std::string> additional_unwindsym_modules;
   struct rlimit our_rlimit;
   while (true)
     {
@@ -768,7 +770,8 @@ systemtap_session::parse_cmdline (int argc, char * const argv [])
             }
             // At runtime user module names are resolved through their
             // canonical (absolute) path, or else it's a kernel module name.
-            unwindsym_modules.insert (resolve_path (optarg));
+	    // The sysroot option might change the path to a user module.
+            additional_unwindsym_modules.insert (resolve_path (optarg));
             // NB: we used to enable_vma_tracker() here for PR10228, but now
             // we'll leave that to pragma:vma functions which actually use it.
             break;
@@ -1504,6 +1507,20 @@ systemtap_session::parse_cmdline (int argc, char * const argv [])
         }
     }
 
+  for (std::set<std::string>::iterator it = additional_unwindsym_modules.begin();
+       it != additional_unwindsym_modules.end();
+       it++)
+    {
+      if (is_user_module(*it))
+	{
+	  unwindsym_modules.insert (resolve_path(sysroot + *it));
+	}
+      else
+	{
+	  unwindsym_modules.insert (*it);
+	}
+    }
+
   return 0;
 }
 
-- 
1.7.4.1


[-- Attachment #3: 0002-Fix-Debug-links-are-not-found-if-the-sysroot-is-used.patch --]
[-- Type: application/octet-stream, Size: 1215 bytes --]

From b97f491dfbf857028acd7b1f89ef146f07a9f7be Mon Sep 17 00:00:00 2001
Message-Id: <b97f491dfbf857028acd7b1f89ef146f07a9f7be.1456861785.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Thu, 6 Mar 2014 21:38:49 +0100
Subject: [PATCH] Fix: Debug links are not found if the sysroot is used.


Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 setupdwfl.cxx |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/setupdwfl.cxx b/setupdwfl.cxx
index 33239fe..c60d608 100644
--- a/setupdwfl.cxx
+++ b/setupdwfl.cxx
@@ -627,6 +627,18 @@ internal_find_debuginfo (Dwfl_Module *mod,
 
   call_dwfl_standard_find_debuginfo:
 
+  if (current_session_for_find_debuginfo)
+    {
+      string sysroot = current_session_for_find_debuginfo->sysroot + "/*";
+      int    found   = fnmatch(sysroot.c_str(), file_name, 0);
+
+      if (found)
+	{
+	  file_name = file_name
+	    + current_session_for_find_debuginfo->sysroot.length() - 1;
+	}
+    }
+
   /* Call the original dwfl_standard_find_debuginfo */
   return dwfl_standard_find_debuginfo(mod, userdata, modname, base,
               file_name, debuglink_file,
-- 
1.7.4.1


[-- Attachment #4: 0003-Fix-Processes-are-not-found.patch --]
[-- Type: application/octet-stream, Size: 1324 bytes --]

From c62883fe56b7ea39d6be17d4663fefb0ff02099b Mon Sep 17 00:00:00 2001
Message-Id: <c62883fe56b7ea39d6be17d4663fefb0ff02099b.1456861792.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Fri, 13 Jun 2014 22:36:44 +0200
Subject: [PATCH] Fix: Processes are not found.

If sysroot is set, module_val will get sysroot as prefix. But
is_fully_resolved() is appending sysroot once more.

Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 tapsets.cxx |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tapsets.cxx b/tapsets.cxx
index 1579deb..347a32d 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -728,8 +728,8 @@ base_query::base_query(dwflpp & dw, literal_map_t const & params):
               pid_val = 0;
               get_string_param(params, TOK_PROCESS, module_val);
             }
-          module_val = find_executable (module_val, sess.sysroot, sess.sysenv);
-          if (!is_fully_resolved(module_val, sess.sysroot, sess.sysenv))
+          module_val = find_executable (module_val, "", sess.sysenv);
+          if (!is_fully_resolved(module_val, "", sess.sysenv))
             throw SEMANTIC_ERROR(_F("cannot find executable '%s'",
                                     module_val.to_string().c_str()));
         }
-- 
1.7.4.1


[-- Attachment #5: 0004-Fix-process.library.function-failed-with-sysroot.patch --]
[-- Type: application/octet-stream, Size: 2365 bytes --]

From 38bab065e8228508ad3e62c03c92f14f64ca529e Mon Sep 17 00:00:00 2001
Message-Id: <38bab065e8228508ad3e62c03c92f14f64ca529e.1456861797.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Wed, 3 Dec 2014 22:17:14 +0100
Subject: [PATCH] Fix: process.library.function failed with sysroot.


Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 tapsets.cxx |   26 ++++++++++++++++++--------
 1 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/tapsets.cxx b/tapsets.cxx
index 347a32d..5883ecc 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -737,11 +737,15 @@ base_query::base_query(dwflpp & dw, literal_map_t const & params):
       // Library probe? Let's target that instead if it is fully resolved (such
       // as what query_one_library() would have done for us). Otherwise, we
       // resort to iterate_over_libraries().
-      if (has_library && is_fully_resolved(library_name, sess.sysroot, sess.sysenv,
-                                           "LD_LIBRARY_PATH"))
-        {
-          path = path_remove_sysroot(sess, module_val);
-          module_val = library_name;
+      if (has_library)
+	{
+	  string library = find_executable (library_name, sess.sysroot,
+					    sess.sysenv, "LD_LIBRARY_PATH");
+	  if (is_fully_resolved(library, "", sess.sysenv, "LD_LIBRARY_PATH"))
+	    {
+	      path = path_remove_sysroot(sess, module_val);
+	      module_val = library;
+	    }
         }
     }
 
@@ -8162,9 +8166,15 @@ dwarf_builder::build(systemtap_session & sess,
       // do this only if the library path is already fully resolved (such as
       // what query_one_library() would have done for us). Otherwise, we resort
       // to iterate_over_libraries.
-      if (get_param (parameters, TOK_LIBRARY, user_lib) && !user_lib.empty()
-          && is_fully_resolved(user_lib, sess.sysroot, sess.sysenv, "LD_LIBRARY_PATH"))
-        module_name = user_lib;
+      if (get_param (parameters, TOK_LIBRARY, user_lib) && !user_lib.empty())
+	{
+	  string library = find_executable (user_lib, sess.sysroot,
+					    sess.sysenv, "LD_LIBRARY_PATH");
+	  if (is_fully_resolved(library, "", sess.sysenv, "LD_LIBRARY_PATH"))
+	    {
+	      module_name = library;
+	    }
+	}
       else
         module_name = user_path; // canonicalize it
 
-- 
1.7.4.1


[-- Attachment #6: Type: text/plain, Size: 2 bytes --]

problems with sysroot variable when cross compiling uprobes

[Holger Brunck]

Hi all,
I am working on a powerpc board with 32 bit architecture and try to compile
userspace probes. This means I have my own cross compiled root filesystem and I
am using stap then with the --sysroot option.

I had a setup up and running with systemtap version 2.5 for a longer time. Now I
tried to upgrade my sytemtap version to 2.9. But already with version 2.6 I see
the following problem: When I try to compile my stap probe it complains with an
error message that the executable can not be found. And what I see that the path
I specified for my sysroot variable is duplicated.

My probe is e.g.:

#!/usr/bin/env stap

probe process("/bin/my_process.out").function("my_function")
{
  printf ("Hello World!\n");
}

My stap command is:
stap -a powerpc --sysroot=/opt/root -B CROSS_COMPILE=powerpc-linux- -m hello -r
<kernel_dir> hello.stp

And the error message is:
semantic error: while resolving probe point: identifier 'process' at hello.stp:3:7
        source: probe process("/bin/my_process.out").function("my_function")
                      ^

semantic error: cannot find executable '/opt/root//opt/root//bin/my_process.out'

Pass 2: analysis failed.  [man error::pass2]


And as you can see  the path I specified with my --sysroot parameter is
duplicated and of course this path (/opt/root//opt/root//usr/...) is not present.

As I said it works fine with an own build systemtap 2.5 but does not work with
an own build systemtap 2.9. Also the official installed stap version 2.8 on my
host system shows the same error.

Am I am doing something wrong here or has anybody seen the same problem? Any
help is appreciated.

Regards
Holger Brunck