Aw: Re: Prelinking on ARM with Debug Link

["Torsten Polle" <Torsten.Polle@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 8113 bytes --]

Hi Mark,

> Gesendet: Mittwoch, 13. April 2016 um 11:25 Uhr
> Von: "Mark Wielaard" <mjw@redhat.com>
> An: "Torsten Polle" <Torsten.Polle@gmx.de>
> Cc: systemtap@sourceware.org
> Betreff: Re: Prelinking on ARM with Debug Link
> Hi Torsten,
> 
> On Tue, 2016-04-12 at 22:26 +0200, Torsten Polle wrote:
> > > Am 11.04.2016 um 23:01 schrieb Mark Wielaard <mjw@redhat.com>:
> > >
> > > On Mon, Apr 11, 2016 at 08:47:00PM +0200, Torsten Polle wrote:
> > >> I’ve checked your patch. As a result the backtrace calculations
> > >> break in my environment.
> > >
> > > I assume this is an in-kernel backtrace, does it involve kernel
> > modules?
> > > Or is it a user backtrace, executable only? shared libraries?
> > > Could you show a probe script and example backtrace?
> >
> > in principle, I’m talking about the example in [1]. In short, I’m
> > talking about getting a proper backtrace in the prelinked library
> > libc-2.18.so. I’m saying „in principle“ because the example does not
> > set a real probe, but only makes SystemTap include the necessary
> > unwinding information for libc-2.18.so.
> >
> > The real (actually already stripped down) script can be found in [2].
> >
> > In the script taptrek_run_izA4.stp in line 240, I’m using the
> > following statement
> > uaddr = __ustack_raw(depth);
> > to get backtrace information. Please don’t get distracted by the usage
> > of the function __ustack_raw(). The function print_ubacktrace() would
> > provide a similar result.
> 
> I am not sure why the example needs to be so complicated.
> The usage of mixed user/kernel space probes/backtraces makes it really
> hard to understand what is going on. Can't you show what you get with a
> simple probe inside libc.so with a print_ubacktrace?

This is what I had in mind. ;-) I use the following probe. Please find the result below.

probe process("/lib/libc-2.18.so").function("_int_malloc").call
{
    print_ubacktrace();
}

> > > Please do include some verbose output. If you can provide the output
> > > of stap -DDEBUG_UNWIND=1 that might be helpful.

_stp_stack_unwind_one_user:462: STARTING user unwind
 0x4df2b1b8 : _int_malloc+0x0/0x15f0 [/lib/libc-2.18.so]
_stp_stack_unwind_one_user:478: CONTINUING user unwind to depth 1
unwind:1481: pc=4df2b1b8, 4df2b1b8
unwind:1524: trying debug_frame
_stp_search_unwind_hdr:778: binary search for 4df2b1b8
_stp_search_unwind_hdr:843: fde off=7294
_stp_search_unwind_hdr:853: returning fde=7f4275b8 startLoc=4df2b1b8
unwind_frame:1197: /lib/libc-2.18.so: fde=7f4275b8
unwind_frame:1202: /lib/libc-2.18.so: cie=7f427354
parse_fde_cie:156: map retAddrReg value 14 to reg_info idx 14
unwind_frame:1217: startLoc: 4df2b1b8, endLoc: 4df2c7a8
unwind_frame:1266: cie=7f427354 fde=7f4275b8 startLoc=4df2b1b8 endLoc=4df2c7a8, pc=4df2b1b8
unwind_frame:1286: processCFI for CIE
processCFI:313: targetLoc=0 state->loc=4df2b1b8
processCFI:519: map DW_CFA_def_cfa value 13 to reg_info idx 13
processCFI:521: DW_CFA_def_cfa reg=13
processCFI:530: DW_CFA_def_cfa_offset offs=0
processCFI:649: targetLoc=0 state->loc=4df2b1b8
processCFI:650: result: 1
unwind_frame:1294: processCFI for FDE
processCFI:313: targetLoc=4df2b1b8 state->loc=4df2b1b8
advance_loc:241: state->loc=4df2b1bc
processCFI:617: DW_CFA_advance_loc
processCFI:649: targetLoc=4df2b1b8 state->loc=4df2b1bc
processCFI:650: result: 1
unwind_frame:1313: cfa reg=13, off=0
unwind_frame:1318: cfa=7ed94438 startLoc=7ed94438, endLoc=7ed94438
unwind_frame:1325: cie=7f427354 fde=7f4275b8
unwind_frame:1449: returning 0 (4df2de08)
_stp_stack_unwind_one_user:503: ret=0 PC=4df2de08 SP=7ed94438
 0x4df2de08 : __libc_malloc+0x74/0x204 [/lib/libc-2.18.so]
_stp_stack_unwind_one_user:478: CONTINUING user unwind to depth 2
unwind:1481: pc=4df2de07, 4df2de08
unwind:1524: trying debug_frame
_stp_search_unwind_hdr:778: binary search for 4df2de07
_stp_search_unwind_hdr:843: fde off=73e4
_stp_search_unwind_hdr:853: returning fde=7f427708 startLoc=4df2dd94
unwind_frame:1197: /lib/libc-2.18.so: fde=7f427708
unwind_frame:1202: /lib/libc-2.18.so: cie=7f427354
parse_fde_cie:156: map retAddrReg value 14 to reg_info idx 14
unwind_frame:1217: startLoc: 4df2dd94, endLoc: 4df2df98
unwind_frame:1266: cie=7f427354 fde=7f427708 startLoc=4df2dd94 endLoc=4df2df98, pc=4df2de07
unwind_frame:1286: processCFI for CIE
processCFI:313: targetLoc=0 state->loc=4df2dd94
processCFI:519: map DW_CFA_def_cfa value 13 to reg_info idx 13
processCFI:521: DW_CFA_def_cfa reg=13
processCFI:530: DW_CFA_def_cfa_offset offs=0
processCFI:649: targetLoc=0 state->loc=4df2dd94
processCFI:650: result: 1
unwind_frame:1294: processCFI for FDE
processCFI:313: targetLoc=4df2de07 state->loc=4df2dd94
advance_loc:241: state->loc=4df2dd98
processCFI:617: DW_CFA_advance_loc
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:530: DW_CFA_def_cfa_offset offs=18
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 3 to reg_info idx 3
set_offset_rule:259: reg=3, where=2, svalue=ffffffe8
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 4 to reg_info idx 4
set_offset_rule:259: reg=4, where=2, svalue=ffffffec
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 5 to reg_info idx 5
set_offset_rule:259: reg=5, where=2, svalue=fffffff0
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 6 to reg_info idx 6
set_offset_rule:259: reg=6, where=2, svalue=fffffff4
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 7 to reg_info idx 7
set_offset_rule:259: reg=7, where=2, svalue=fffffff8
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:627: map DW_CFA_offset value 14 to reg_info idx 14
set_offset_rule:259: reg=e, where=2, svalue=fffffffc
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
processCFI:322: DW_CFA_nop
processCFI:649: targetLoc=4df2de07 state->loc=4df2dd98
processCFI:650: result: 1
unwind_frame:1313: cfa reg=13, off=18
unwind_frame:1318: cfa=7ed94450 startLoc=7ed94438, endLoc=7ed94450
unwind_frame:1325: cie=7f427354 fde=7f427708
unwind_frame:1439: set register 3 to 7ed94488
unwind_frame:1439: set register 4 to 47
unwind_frame:1439: set register 5 to 7ed94474
unwind_frame:1439: set register 6 to 2f580
unwind_frame:1439: set register 7 to 8
unwind_frame:1439: set register 14 to 23a30
unwind_frame:1449: returning 0 (23a30)
_stp_stack_unwind_one_user:503: ret=0 PC=23a30 SP=7ed94450
 0x23a30 [/lib/systemd/systemd-udevd+0x1ba30/0x35000]
_stp_stack_unwind_one_user:478: CONTINUING user unwind to depth 3
unwind:1481: pc=23a2f, 23a30
unwind:1520: No module found for pc=23a2f
_stp_stack_unwind_one_user:503: ret=-22 PC=23a30 SP=7ed94450


> > I’ll try my best and come back to you. I would provide the
> > information with my patch applied, because otherwise the result is
> > simply garbage.
> 
> Please do include the exact patch you are using.

Please find the patch attached.

> Thanks, that is interesting.
> But it is also wrong that we use section offsets in user space modules.
> The sections really shouldn't play a role, except for kernel modules.
> User space modules are mapped into memory according to their segments
> (phdrs). Could you show the segments plus section to segment mapping of
> the libc-2.18.so and libc-2.18.so.debug files used with eu-readelf -Sl

Please check [1] for the requested output.

This means sec_load_offset should be 0 as you proposed in your
patch. But then the question is where you want to adjust for the
necessary difference between offsets.

> Thanks,
> 
> Mark

Thanks,
Torsten

[1] https://sourceware.org/ml/systemtap/2015-q4/msg00219.html

[-- Attachment #2: 0001-PATCH-v2-Fix-Compilation-fails-for-prelinked-librari.patch --]
[-- Type: application/octet-stream, Size: 3021 bytes --]

From ce284b4398627305714a10c5f436a4c0b2ae39e8 Mon Sep 17 00:00:00 2001
Message-Id: <ce284b4398627305714a10c5f436a4c0b2ae39e8.1460557495.git.tpolle@de.adit-jv.com>
From: Torsten Polle <Torsten.Polle@gmx.de>
Date: Sun, 28 Feb 2016 21:45:33 +0100
Subject: [PATCH] [PATCH v2] Fix: Compilation fails for prelinked libraries.

The offset to the section ".text" in a prelinked library might increase due to
changed relocations that need more space. If the prelinked library contains a
debug link and the linked file is not prelinked, the offset to the ".text"
section is different. If the difference between the offset of the ".text"
section in the prelinked library and non-prelinked library (with e.g. debug
information) becomes negative, the representation of this negative number has
to fit into an uint32_t. If the width of the host is larger than the
width of the target, the (cross-) compilation for the target fails.

Changes since v1:
* Use hex values for the 32bit case.

Signed-off-by: Torsten Polle <tpolle@de.adit-jv.com>
---
 translate.cxx |   34 ++++++++++++++++++++++++++++++----
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/translate.cxx b/translate.cxx
index 5daf725..fcd2d8e 100644
--- a/translate.cxx
+++ b/translate.cxx
@@ -6912,9 +6912,22 @@ dump_unwindsym_cxt (Dwfl_Module *m,
           c->output << ".debug_hdr_len = " << debug_frame_hdr_len << ", \n";
 
 	  Dwarf_Addr dwbias = 0;
+	  Dwarf_Addr elf_bias;
+	  GElf_Ehdr *ehdr, ehdr_mem;
+	  Elf *elf;
+	  elf = dwfl_module_getelf(m, &elf_bias);
+	  ehdr = gelf_getehdr(elf, &ehdr_mem);
 	  dwfl_module_getdwarf (m, &dwbias);
-	  c->output << ".sec_load_offset = 0x"
-		    << hex << debug_frame_off - dwbias << dec << "\n";
+	  if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
+	    {
+	      c->output << ".sec_load_offset = 0x"
+			<< hex << (uint32_t)(debug_frame_off - dwbias) << dec << "\n";
+	    }
+	  else
+	    {
+	      c->output << ".sec_load_offset = 0x"
+			<< hex << debug_frame_off - dwbias << dec << "\n";
+	    }
 
 	  c->output << "#else\n";
 	  c->output << ".debug_hdr = NULL,\n";
@@ -6932,9 +6945,22 @@ dump_unwindsym_cxt (Dwfl_Module *m,
             {
               c->output << "#if defined(STP_NEED_LINE_DATA)\n";
               Dwarf_Addr dwbias = 0;
+	      Dwarf_Addr elf_bias;
+	      GElf_Ehdr *ehdr, ehdr_mem;
+	      Elf *elf;
+	      elf = dwfl_module_getelf(m, &elf_bias);
+	      ehdr = gelf_getehdr(elf, &ehdr_mem);
               dwfl_module_getdwarf (m, &dwbias);
-              c->output << ".sec_load_offset = 0x"
-                        << hex << debug_frame_off - dwbias << dec << "\n";
+	      if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
+		{
+		  c->output << ".sec_load_offset = 0x"
+			    << hex << (uint32_t)(debug_frame_off - dwbias) << dec << "\n";
+		}
+	      else
+		{
+		  c->output << ".sec_load_offset = 0x"
+			    << hex << debug_frame_off - dwbias << dec << "\n";
+		}
               c->output << "#else\n";
             }
 	  c->output << ".sec_load_offset = 0\n";
-- 
1.7.9.5