setting a shared library static

setting a shared library static

[Stan Cox]

[-- Attachment #1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2: ,stap --]
[-- Type: text/plain, Size: 2161 bytes --]

Trying to get a static variable in a .so, in this case it controls the firing of
a marker, to get set from within stap.  First strace is run via strace
and we see an mmap of the entire .so followed by an mprotect of the
protected segment followed by an mmap of the writeable segment.  

strace 
 mmap 0x7f7713392000 length 2102464
 mprotect 0x7f7713394000 length 2093056 PROT_NONE
 mmap 0x7f7713593000 length 4096 

7f7713392000-7f7713394000 libsdt.so rxp
7f7713394000-7f7713593000 libsdt.so p
7f7713593000-7f7713594000 libsdt.so rwp

For the stap run the semaphore is accessed at various points and the
epoch time in microseconds displayed.  For the run we see:
 mmap of the entire .so 
 mmap of the writeable segment
 munmap of the segment preceding the first .so segment
 mmap of the first segment of the .so
 mmap of the protected segment of the .so
 mmap of the writeable segment of the .so
 munmapping of segments that don't seem to correspond to the .so

stap
 __stp_call_mmap_callbacks 0x7f1f8d377000  0x202000  r-xp libsdt.so
 __stp_call_mmap_callbacks 0x7f1f8d578000  0x1000    rw-p libsdt.so
 stap_uprobe_munmap_found  0x7f1f8d360000 length 92486 libsdt.so
__stp_call_mmap_callbacks  0x7f1f8d377000  0x2000    r-xp libsdt.so
__stp_call_mmap_callbacks  0x7f1f8d379000  0x2000    ---p libsdt.so
			   Time 1,256,655,823,883,308
__stp_call_mmap_callbacks  0x7f1f8d578000  0x1000    rw-p libsdt.so
			   Time 1,256,655,823,883,317
			   The semaphore can be accessed here
stap_uprobe_munmap_found   0x7f3ff181a000 length 92486 libsdt.so
			   Time 1,256,655,823,924,199
			   The semaphore cannot be accessed here
stap_uprobe_munmap_found   0x7f3ff1830000 length 4096 libsdt.so
			   Time 1,256,655,823,924,914
stap_uprobe_munmap_found   0x7f26aec7a000 length 92486 libsdt.so
			   Time 1,256,655,823,938,678
stap_uprobe_munmap_found   0x7f26aec90000 length 4096 libsdt.so 
			   Time 1,256,655,823,939,013
program starts running: Time 1,256,655,824,250,531

 
Map
7f1f8d377000-7f1f8d379000 r-xp 00000000 08:05 458339 libsdt.so
7f1f8d379000-7f1f8d578000 ---p 00002000 08:05 458339 libsdt.so
7f1f8d578000-7f1f8d579000 rw-p 00001000 08:05 458339 libsdt.so

Re: setting a shared library static

[Stan Cox]

The static variable address for the strace run of test is 0x7f7713593494, i.e.
7f7713593000-7f7713594000 rwp 00001000  08:05  458339 libsdt.so

The static variable address for the stap run of test is 0x7f1f8d578494, i.e.
7f1f8d578000-7f1f8d579000 rw-p 00001000 08:05 458339 libsdt.so

For this stap run at Time 1,256,655,823,883,317 the static can be accessed and 
after 1,256,655,823,924,199 it cannot but I don't (yet) see anything obvious 
that causes that behavior.

Re: setting a shared library static

[Stan Cox]

The test program via the macros in sdt.h, and help from stap, save away 
dwfl_module_relocate_address(&static variable).  When stap runs it relocates 
&static_variable and when the writeable segment of the .so is mmapped it 
increments it via __access_process_vm and decrements it via 
__access_process_vm.  If static_variable is not in a .so then it works as 
expected.  Otherwise static_variable gets set in stap_uprobe_mmap_found via 
task_finder.c::__stp_call_mmap_callbacks.  The value of &static_variable is 
checked at the mmap and munmap callbacks.  The behavior I'm noticing is that 
at the first callback for munmap the value of &static_variable has become 
unavailable.

eu-readelf -lS libsdt-uprobe.so
There are 36 section headers, starting at offset 0x2518:

Section Headers:
[Nr] Name                 Type         Addr             Off      Size     ES 
Flags Lk Inf Al
[ 0]                      NULL         0000000000000000 00000000 00000000  0 
       0   0  0
[ 1] .note.gnu.build-id   NOTE         0000000000000190 00000190 00000024  0 A 
      0   0  4
[ 2] .gnu.hash            GNU_HASH     00000000000001b8 000001b8 00000064  0 A 
      3   0  8
[ 3] .dynsym              DYNSYM       0000000000000220 00000220 000001f8 24 A 
      4   2  8
[ 4] .dynstr              STRTAB       0000000000000418 00000418 00000119  0 A 
      0   0  1
[ 5] .gnu.version         GNU_versym   0000000000000532 00000532 0000002a  2 A 
      3   0  2
[ 6] .gnu.version_r       GNU_verneed  0000000000000560 00000560 00000020  0 A 
      4   1  8
[ 7] .rela.dyn            RELA         0000000000000580 00000580 00000258 24 A 
      3   0  8
[ 8] .rela.plt            RELA         00000000000007d8 000007d8 00000048 24 A 
      3  10  8
[ 9] .init                PROGBITS     0000000000000820 00000820 00000018  0 
AX     0   0  4
[10] .plt                 PROGBITS     0000000000000838 00000838 00000040 16 
AX     0   0  4
[11] .text                PROGBITS     0000000000000880 00000880 00000718  0 
AX     0   0 16
[12] .fini                PROGBITS     0000000000000f98 00000f98 0000000e  0 
AX     0   0  4
[13] .rodata              PROGBITS     0000000000000fa8 00000fa8 00000119  0 A 
      0   0  8
[14] .eh_frame_hdr        PROGBITS     00000000000010c4 000010c4 0000001c  0 A 
      0   0  4
[15] .eh_frame            PROGBITS     00000000000010e0 000010e0 0000005c  0 A 
      0   0  8
[16] .ctors               PROGBITS     0000000000201140 00001140 00000010  0 
WA     0   0  8
[17] .dtors               PROGBITS     0000000000201150 00001150 00000010  0 
WA     0   0  8
[18] .jcr                 PROGBITS     0000000000201160 00001160 00000008  0 
WA     0   0  8
[19] .data.rel.ro         PROGBITS     0000000000201168 00001168 00000008  0 
WA     0   0  8
[20] .dynamic             DYNAMIC      0000000000201170 00001170 00000190 16 
WA     4   0  8
[21] .got                 PROGBITS     0000000000201300 00001300 00000050  8 
WA     0   0  8
[22] .got.plt             PROGBITS     0000000000201350 00001350 00000030  8 
WA     0   0  8
[23] .probes              PROGBITS     0000000000201380 00001380 00000122  0 
WA     0   0  8
[24] .bss                 NOBITS       00000000002014a8 000014a2 00000018  0 
WA     0   0  8
[25] .comment             PROGBITS     0000000000000000 000014a2 0000010e  0 
       0   0  1
[26] .debug_aranges       PROGBITS     0000000000000000 000015b0 00000060  0 
       0   0  1
[27] .debug_pubnames      PROGBITS     0000000000000000 00001610 000000e5  0 
       0   0  1
[28] .debug_info          PROGBITS     0000000000000000 000016f5 000007e2  0 
       0   0  1
[29] .debug_abbrev        PROGBITS     0000000000000000 00001ed7 0000017e  0 
       0   0  1
[30] .debug_line          PROGBITS     0000000000000000 00002055 0000010b  0 
       0   0  1
[31] .debug_str           PROGBITS     0000000000000000 00002160 000001cc  1 
MS     0   0  1
[32] .debug_loc           PROGBITS     0000000000000000 0000232c 00000098  0 
       0   0  1
[33] .shstrtab            STRTAB       0000000000000000 000023c4 00000152  0 
       0   0  1
[34] .symtab              SYMTAB       0000000000000000 00002e18 000006d8 24 
      35  54  8
[35] .strtab              STRTAB       0000000000000000 000034f0 0000024a  0 
       0   0  1

Program Headers:
   Type           Offset   VirtAddr           PhysAddr           FileSiz 
MemSiz   Flg Align
   LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x00113c 
0x00113c R E 0x200000
   LOAD           0x001140 0x0000000000201140 0x0000000000201140 0x000362 
0x000380 RW  0x200000
   DYNAMIC        0x001170 0x0000000000201170 0x0000000000201170 0x000190 
0x000190 RW  0x8
   NOTE           0x000190 0x0000000000000190 0x0000000000000190 0x000024 
0x000024 R   0x4
   GNU_EH_FRAME   0x0010c4 0x00000000000010c4 0x00000000000010c4 0x00001c 
0x00001c R   0x4
   GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 
0x000000 RW  0x8

  Section to Segment mapping:
   Segment Sections...
    00      [RO: .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version 
.gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata 
.eh_frame_hdr .eh_frame]
    01      .ctors .dtors .jcr .data.rel.ro .dynamic .got .got.plt .probes .bss
    02      .dynamic
    03      [RO: .note.gnu.build-id]
    04      [RO: .eh_frame_hdr]
    05

Re: setting a shared library static

[Roland McGrath]

> at the first callback for munmap the value of &static_variable has become 
> unavailable.

Define this, please.  access_process_vm doesn't have a useful return value
for errors, but it works by calling get_user_pages and any failure should
be coming from there.  Can you cite the exact arguments and return value
for this get_user_pages call?


Thanks,
Roland

Re: setting a shared library static

[Stan Cox]

[-- Attachment #1: Type: text/plain, Size: 133 bytes --]

On 10/27/2009 05:24 PM, Roland McGrath wrote:
> Can you cite the exact arguments and return value
> for this get_user_pages call?




[-- Attachment #2: ,mail0 --]
[-- Type: text/plain, Size: 2297 bytes --]

This first hunk is when stap is successful accessing address the
static variable at address 0xfafeee03494.  

__access_process_vm (via probing with stap)
 __get_user_pages
  find_extend_vma 
   find_vma 

__access_process_vm: tsk 0xffff88000c188000 mm 0xffff88002ec309c0 addr 0x7fafeee03494 write 0 1256849310691081
__get_user_pages@mm/memory.c:1226") tsk=0xffff88000c188000 mm=? start=0x7fafeee03494 len=0x1 flags=? pages=0xffff880026761b90 vmas=0xffff880026761b98
find_vma@mm/mmap.c:1555").return mm=0xffff88002ec309c0 addr=0x7fafeee03000 ffff8800260d1b00
# this is the stap "return" probe showing -131940756940032 is returned
find_extend_vma@mm/mmap.c:1818").return mm=0xffff88002ec309c0 addr=0x7fafeee03494 -131940756940032
# return probe, 1 is returned
__get_user_pages@mm/memory.c:1209").return tsk=0xffff88000c188000 mm=0xffff88002ec309c0 start=0x7fafeee03494 len=0x1 flags=0x2 pages=0xffff880026761b90 vmas=0xffff880026761b98 1
could access saved 0x7fafeee03494 0x1 for reading 1256849310691202 


This second hunk is when stap is unsuccessful accessing address the
static variable at address 0xfafeee03494.

__access_process_vm 
 __get_user_pages
  find_extend_vma 
   find_vma 
    anon_vma_prepare 
     expand_downwards

__access_process_vm: tsk 0xffff88000c188000 mm 0xffff88002ec309c0 addr 0x7fafeee03494 write 0 1256849310730608
__get_user_pages@mm/memory.c:1226") tsk=0xffff88000c188000 mm=? start=0x7fafeee03494 len=0x1 flags=? pages=0xffff880026761d58 vmas=0xffff880026761d60
find_vma@mm/mmap.c:1555").return mm=0xffff88002ec309c0 addr=0x7fafeee03000 ffff88000e033a50
find_extend_vma@mm/mmap.c:1829") mm=? addr=0x7fafeee03000 100173
anon_vma_prepare@mm/rmap.c:97").return vma=0xffff88000581f9a0 0
expand_downwards@mm/mmap.c:1741").return vma=0xffff88000581f9a0 address=0x7f22f00b3000 fffffffffffffff4 
# #define ENOMEM          12      /* Out of memory */
# find_extend_vma returns 0
find_extend_vma@mm/mmap.c:1818").return mm=0xffff88002ec309c0 addr=0x7fafeee03494 0
__get_user_pages@mm/memory.c:1209").return tsk=0xffff88000c188000 mm=0xffff88002ec309c0 start=0x7fafeee03494 len=0x1 flags=0x2 pages=0xffff880026761d58 vmas=0xffff880026761d60 -14
# #define EFAULT          14      /* Bad address */
couldn't access saved 0x7fafeee03494 0xffffffff 0 for reading 1256849310730716

Re: setting a shared library static

[Roland McGrath]

[before]
> find_vma@mm/mmap.c:1555").return mm=0xffff88002ec309c0 addr=0x7fafeee03000 ffff8800260d1b00

[after]
> find_vma@mm/mmap.c:1555").return mm=0xffff88002ec309c0 addr=0x7fafeee03000 ffff88000e033a50

That return value is a struct vm_area_struct *.  So this means that it
found a different vma.  It would be helpful to see ->vm_{start,end}, etc.
of that using @cast syntax magic I can't exactly recall (do we have a
tapset "print_vma"?).  Also could perhaps try to stop in between and dump
the /proc/pid/maps (or internal equivalent) before each access_process_vm call.


Thanks,
Roland