[Bug nptl/15771] New: __lll_unlock_wake break perf/libunwind unwinding

[Bug nptl/15771] New: __lll_unlock_wake break perf/libunwind unwinding

[andi-bz at firstfloor dot org]

http://sourceware.org/bugzilla/show_bug.cgi?id=15771

            Bug ID: 15771
           Summary: __lll_unlock_wake break perf/libunwind unwinding
           Product: glibc
           Version: 2.17
            Status: NEW
          Severity: normal
          Priority: P2
         Component: nptl
          Assignee: unassigned at sourceware dot org
          Reporter: andi-bz at firstfloor dot org
                CC: drepper.fsp at gmail dot com

Created attachment 7113
  --> http://sourceware.org/bugzilla/attachment.cgi?id=7113&action=edit
mutex test case

Something in the way the x86-64 dwarf unwind table is written for
__lll_unlock_wake breaks perf's libunwind implementation

This is a problem because it makes it hard to track down lock contention in
pthread programs.

When profiling the attached test case with 

perf record -g dwarf ...
perf report 

we get

perf/libunwind:
 |--57.28%-- futex_wake
                 |          do_futex
                 |          sys_futex
                 |          system_call_fastpath
                 |          __lll_unlock_wake
                 |          |         
                 |          |--80.89%-- 0x0
                 |          |         
                 |          |--18.73%-- _L_unlock_701
                 |           --0.38%-- [...]


The unwind always stops inside __lll_unlock_wake

gdb can backtrace through it

#0  0x000000326fe0de8a in __lll_unlock_wake () from /lib64/libpthread.so.0
#1  0x000000326fe0aae3 in _L_unlock_701 () from /lib64/libpthread.so.0
#2  0x000000326fe0aa6e in pthread_mutex_unlock () from /lib64/libpthread.so.0
#3  0x0000000000400809 in thread (arg=0x0) at tmutex.c:14
#4  0x000000326fe07c53 in start_thread () from /lib64/libpthread.so.0
#5  0x000000326faf513d in clone () from /lib64/libc.so.6

The glibc backtrace used by mutrace also seems to work.

This is on FC19 with 3.10 perf and libunwind-1.1-2.fc19.x86_64

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/15771] __lll_unlock_wake break perf/libunwind unwinding

[andi-bz at firstfloor dot org]

http://sourceware.org/bugzilla/show_bug.cgi?id=15771

--- Comment #1 from Andi Kleen <andi-bz at firstfloor dot org> ---
So it looks like libunwind sometimes gets it right, and sometimes wrong.
Depending on the IP hit?

Here's the dwarf for __lll_unlock_wake

00002948 00000024 0000294c FDE cie=00000000 pc=0000e090..0000e0ad
  DW_CFA_advance_loc: 1 to 0000e091
  DW_CFA_def_cfa_offset: 16
  DW_CFA_advance_loc: 1 to 0000e092
  DW_CFA_def_cfa_offset: 24
  DW_CFA_offset: r4 (rsi) at cfa-16
  DW_CFA_offset: r1 (rdx) at cfa-24
  DW_CFA_advance_loc: 25 to 0000e0ab
  DW_CFA_def_cfa_offset: 16
  DW_CFA_restore: r1 (rdx)
  DW_CFA_advance_loc: 1 to 0000e0ac
  DW_CFA_def_cfa_offset: 8
  DW_CFA_restore: r4 (rsi)
  DW_CFA_nop
  DW_CFA_nop
  DW_CFA_nop
  DW_CFA_nop
  DW_CFA_nop

and the assembler

000000326fe0de70 <__lll_unlock_wake>:
  326fe0de70:   56                      push   %rsi
  326fe0de71:   52                      push   %rdx
  326fe0de72:   c7 07 00 00 00 00       movl   $0x0,(%rdi)
  326fe0de78:   81 f6 81 00 00 00       xor    $0x81,%esi
  326fe0de7e:   ba 01 00 00 00          mov    $0x1,%edx
  326fe0de83:   b8 ca 00 00 00          mov    $0xca,%eax
  326fe0de88:   0f 05                   syscall 
  326fe0de8a:   5a                      pop    %rdx
  326fe0de8b:   5e                      pop    %rsi
  326fe0de8c:   c3                      retq   
  326fe0de8d:   0f 1f 00                nopl   (%rax)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/15771] __lll_unlock_wake break perf/libunwind unwinding

[arun@sharma-home.net]

https://sourceware.org/bugzilla/show_bug.cgi?id=15771

Arun Sharma <arun@sharma-home.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |arun@sharma-home.net

--- Comment #2 from Arun Sharma <arun@sharma-home.net> ---
A patch was submitted to libunwind-devel for this recently.

http://lists.gnu.org/archive/html/libunwind-devel/2014-01/msg00003.html

This code was committed in 2006 to glibc:

http://repo.or.cz/w/glibc.git/commitdiff/cd248c3f0dff50ed7a9421fa3b152401050e6a6c?hp=71213dc2589554dc8f8061e9b04e80c55d098b6a

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug nptl/15771] __lll_unlock_wake break perf/libunwind unwinding

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=15771

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.