Re: problems with sysroot variable when cross compiling uprobes

[Torsten Polle <Torsten.Polle@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 2491 bytes --]

Dear all,

my original mail was classified as spam. As a different mail got accepted, I hope the problems are fixed now.

> Am 02.03.2016 um 10:38 schrieb Holger Brunck <holger.brunck@keymile.com>:
> 
> Hi Torsten,
> 
> On 03/01/2016 08:59 PM, Torsten Polle wrote:
>>> I am working on a powerpc board with 32 bit architecture and try to
>>> compile userspace probes. This means I have my own cross compiled
>>> root filesystem and I am using stap then with the --sysroot option.
>> 
>>> I had a setup up and running with systemtap version 2.5 for a longer
>>> time. Now I tried to upgrade my sytemtap version to 2.9. But already
>>> with version 2.6 I see the following problem: When I try to compile
>>> my stap probe it complains with an error message that the executable
>>> can not be found. And what I see that the path I specified for my
>>> sysroot variable is duplicated.
>> 
>> [...]
>> 
>>> As I said it works fine with an own build systemtap 2.5 but does not
>>> work with an own build systemtap 2.9. Also the official installed
>>> stap version 2.8 on my host system shows the same error.
>> 
>>> Am I am doing something wrong here or has anybody seen the same
>>> problem? Any help is appreciated.
>> 
>> I'm using a cross compile environment for ARM. For me everything works
>> fine. But I'm using a number of patches to make everything in my
> 
> great, the third patch definitely solves the problem I have on my side and I can
> use systemtap 2.9 in my powerpc environment. Thanks a lot.
> 
>> environment. I never dared to upstream those because my knowledge of the
>> SystemTap code itself is very poor and might be wrong for the general
>> case. If they also work fine for you, the patches might be a starting
> 
> I am also not familar with the code. But it might be worth to try to upstream
> it. Even if it might be not a perfect solutions for the general case, others
> around may then propose clean solutions which are ready to be merged upstream. I
> mean the sysroot option seems to be broken. Or is anyone around here who can
> work with the upstream version in a cross compiled environment and uses this
> option successfully?
> 
> Best regards
> Holger Brunck

Enclosed are four patches I use to be able to use SystemTap in a cross compile environment. As my knowledge about the internals of SystemTap are limited, they might be a bit rough. Any comments to improve them are welcome.

Kind Regards,
Torsten


[-- Attachment #2: 0001-Support-the-sysroot-option-for-modules-added-for-sym.patch --]
[-- Type: application/octet-stream, Size: 2318 bytes --]

From 1b535eda5c320271d25b1b511cd5a4eb2ad9d575 Mon Sep 17 00:00:00 2001
Message-Id: <1b535eda5c320271d25b1b511cd5a4eb2ad9d575.1456861753.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Fri, 13 Dec 2013 23:26:07 +0100
Subject: [PATCH] Support the sysroot option for modules added for symbol/unwind information

The sysroot option failed for modules added for their symbol/unwind
information. Those user module names are prepended by sysroot.

Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 session.cxx |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/session.cxx b/session.cxx
index 10b36d0..dd99284 100644
--- a/session.cxx
+++ b/session.cxx
@@ -15,6 +15,7 @@
 #include "buildrun.h"
 #include "coveragedb.h"
 #include "hash.h"
+#include "setupdwfl.h"
 #include "task_finder.h"
 #include "csclient.h"
 #include "rpm_finder.h"
@@ -689,6 +690,7 @@ int
 systemtap_session::parse_cmdline (int argc, char * const argv [])
 {
   client_options_disallowed_for_unprivileged = "";
+  std::set<std::string> additional_unwindsym_modules;
   struct rlimit our_rlimit;
   while (true)
     {
@@ -768,7 +770,8 @@ systemtap_session::parse_cmdline (int argc, char * const argv [])
             }
             // At runtime user module names are resolved through their
             // canonical (absolute) path, or else it's a kernel module name.
-            unwindsym_modules.insert (resolve_path (optarg));
+	    // The sysroot option might change the path to a user module.
+            additional_unwindsym_modules.insert (resolve_path (optarg));
             // NB: we used to enable_vma_tracker() here for PR10228, but now
             // we'll leave that to pragma:vma functions which actually use it.
             break;
@@ -1504,6 +1507,20 @@ systemtap_session::parse_cmdline (int argc, char * const argv [])
         }
     }
 
+  for (std::set<std::string>::iterator it = additional_unwindsym_modules.begin();
+       it != additional_unwindsym_modules.end();
+       it++)
+    {
+      if (is_user_module(*it))
+	{
+	  unwindsym_modules.insert (resolve_path(sysroot + *it));
+	}
+      else
+	{
+	  unwindsym_modules.insert (*it);
+	}
+    }
+
   return 0;
 }
 
-- 
1.7.4.1


[-- Attachment #3: 0002-Fix-Debug-links-are-not-found-if-the-sysroot-is-used.patch --]
[-- Type: application/octet-stream, Size: 1215 bytes --]

From b97f491dfbf857028acd7b1f89ef146f07a9f7be Mon Sep 17 00:00:00 2001
Message-Id: <b97f491dfbf857028acd7b1f89ef146f07a9f7be.1456861785.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Thu, 6 Mar 2014 21:38:49 +0100
Subject: [PATCH] Fix: Debug links are not found if the sysroot is used.


Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 setupdwfl.cxx |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/setupdwfl.cxx b/setupdwfl.cxx
index 33239fe..c60d608 100644
--- a/setupdwfl.cxx
+++ b/setupdwfl.cxx
@@ -627,6 +627,18 @@ internal_find_debuginfo (Dwfl_Module *mod,
 
   call_dwfl_standard_find_debuginfo:
 
+  if (current_session_for_find_debuginfo)
+    {
+      string sysroot = current_session_for_find_debuginfo->sysroot + "/*";
+      int    found   = fnmatch(sysroot.c_str(), file_name, 0);
+
+      if (found)
+	{
+	  file_name = file_name
+	    + current_session_for_find_debuginfo->sysroot.length() - 1;
+	}
+    }
+
   /* Call the original dwfl_standard_find_debuginfo */
   return dwfl_standard_find_debuginfo(mod, userdata, modname, base,
               file_name, debuglink_file,
-- 
1.7.4.1


[-- Attachment #4: 0003-Fix-Processes-are-not-found.patch --]
[-- Type: application/octet-stream, Size: 1324 bytes --]

From c62883fe56b7ea39d6be17d4663fefb0ff02099b Mon Sep 17 00:00:00 2001
Message-Id: <c62883fe56b7ea39d6be17d4663fefb0ff02099b.1456861792.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Fri, 13 Jun 2014 22:36:44 +0200
Subject: [PATCH] Fix: Processes are not found.

If sysroot is set, module_val will get sysroot as prefix. But
is_fully_resolved() is appending sysroot once more.

Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 tapsets.cxx |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tapsets.cxx b/tapsets.cxx
index 1579deb..347a32d 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -728,8 +728,8 @@ base_query::base_query(dwflpp & dw, literal_map_t const & params):
               pid_val = 0;
               get_string_param(params, TOK_PROCESS, module_val);
             }
-          module_val = find_executable (module_val, sess.sysroot, sess.sysenv);
-          if (!is_fully_resolved(module_val, sess.sysroot, sess.sysenv))
+          module_val = find_executable (module_val, "", sess.sysenv);
+          if (!is_fully_resolved(module_val, "", sess.sysenv))
             throw SEMANTIC_ERROR(_F("cannot find executable '%s'",
                                     module_val.to_string().c_str()));
         }
-- 
1.7.4.1


[-- Attachment #5: 0004-Fix-process.library.function-failed-with-sysroot.patch --]
[-- Type: application/octet-stream, Size: 2365 bytes --]

From 38bab065e8228508ad3e62c03c92f14f64ca529e Mon Sep 17 00:00:00 2001
Message-Id: <38bab065e8228508ad3e62c03c92f14f64ca529e.1456861797.git.Torsten.Polle@gmx.de>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Wed, 3 Dec 2014 22:17:14 +0100
Subject: [PATCH] Fix: process.library.function failed with sysroot.


Signed-off-by: Torsten Polle <Torsten.Polle@gmx.de>
---
 tapsets.cxx |   26 ++++++++++++++++++--------
 1 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/tapsets.cxx b/tapsets.cxx
index 347a32d..5883ecc 100644
--- a/tapsets.cxx
+++ b/tapsets.cxx
@@ -737,11 +737,15 @@ base_query::base_query(dwflpp & dw, literal_map_t const & params):
       // Library probe? Let's target that instead if it is fully resolved (such
       // as what query_one_library() would have done for us). Otherwise, we
       // resort to iterate_over_libraries().
-      if (has_library && is_fully_resolved(library_name, sess.sysroot, sess.sysenv,
-                                           "LD_LIBRARY_PATH"))
-        {
-          path = path_remove_sysroot(sess, module_val);
-          module_val = library_name;
+      if (has_library)
+	{
+	  string library = find_executable (library_name, sess.sysroot,
+					    sess.sysenv, "LD_LIBRARY_PATH");
+	  if (is_fully_resolved(library, "", sess.sysenv, "LD_LIBRARY_PATH"))
+	    {
+	      path = path_remove_sysroot(sess, module_val);
+	      module_val = library;
+	    }
         }
     }
 
@@ -8162,9 +8166,15 @@ dwarf_builder::build(systemtap_session & sess,
       // do this only if the library path is already fully resolved (such as
       // what query_one_library() would have done for us). Otherwise, we resort
       // to iterate_over_libraries.
-      if (get_param (parameters, TOK_LIBRARY, user_lib) && !user_lib.empty()
-          && is_fully_resolved(user_lib, sess.sysroot, sess.sysenv, "LD_LIBRARY_PATH"))
-        module_name = user_lib;
+      if (get_param (parameters, TOK_LIBRARY, user_lib) && !user_lib.empty())
+	{
+	  string library = find_executable (user_lib, sess.sysroot,
+					    sess.sysenv, "LD_LIBRARY_PATH");
+	  if (is_fully_resolved(library, "", sess.sysenv, "LD_LIBRARY_PATH"))
+	    {
+	      module_name = library;
+	    }
+	}
       else
         module_name = user_path; // canonicalize it
 
-- 
1.7.4.1


[-- Attachment #6: Type: text/plain, Size: 2 bytes --]