[Bug dyninst/15101] New: [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[Bug dyninst/15101] New: [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[jiri.horky at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

             Bug #: 15101
           Summary: [PATCH] Stap sefaults when calling NSS_shutdown()
                    which was not previously initialized
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: dyninst
        AssignedTo: systemtap@sourceware.org
        ReportedBy: jiri.horky@gmail.com
    Classification: Unclassified


Created attachment 6845
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6845
GDB backtrace

Systemtap git version as of 20130105, SLES SP1, 2.6.32.59-0.7.1.du2-default.

Problem: stap segfaults when running probe for NFSD code (see attachment) in
NSS_shutdown(). This is because NSS was previously initiliazed. Please attached
GDB backtrace as an example of such code. 

I also include patch that fixes this behavior. The patch was originally written
by my colleague Lukas Hejtmanek.

Jiri Horky

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug dyninst/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[jiri.horky at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #1 from Jiri Horky <jiri.horky at gmail dot com> 2013-02-05 14:22:09 UTC ---
Created attachment 6846
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6846
Patch that fixes the problem

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug dyninst/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[jiri.horky at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #2 from Jiri Horky <jiri.horky at gmail dot com> 2013-02-05 14:23:18 UTC ---
Created attachment 6847
  --> http://sourceware.org/bugzilla/attachment.cgi?id=6847
stp script used to reproduce the problem

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug dyninst/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[brolley at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Dave Brolley <brolley at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |brolley at redhat dot com

--- Comment #3 from Dave Brolley <brolley at redhat dot com> 2013-02-05 15:08:23 UTC ---
The patch is for the function nssCleanup in cscommon.cxx.

While the patch will certainly suppress the failure, the intent is that calls
to NSS_Init and NSS_Shutdown against a given database be balanced. This is
because we need to be able to initialize/shutdown NSS multiple times against
different databases. There is an accommodation for the one known unbalanced
call to nssCleanup (from the rpm finder).

I think that we should try to find the source of the unbalanced call before we
resort to checking the state of NSS within this function.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[jistone at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Josh Stone <jistone at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|dyninst                     |server

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[fche at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #6847|application/octet-stream    |text/plain
          mime type|                            |

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[fche at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Frank Ch. Eigler <fche at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING
                 CC|                            |fche at redhat dot com

--- Comment #4 from Frank Ch. Eigler <fche at redhat dot com> 2013-02-06 18:21:50 UTC ---
Jiri, can you spell out how you invoked stap (and maybe add a half-dozen -v's
for extra information)?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[fche at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #5 from Frank Ch. Eigler <fche at redhat dot com> 2013-02-06 21:00:57 UTC ---
Also, can you identify the versions of nss/nspr in use?

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[brolley at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #6 from Dave Brolley <brolley at redhat dot com> 2013-02-06 21:41:43 UTC ---
It turns out that the call to nssCleanup is not unbalanced after all. It came
from rpm_finder.cxx:missing_rpm_enlist.

Modern versions of NSS (3.11 and later) self check that NSS has been
initialized when NSS_Shutdown is been called, however, some do not (that's why
we're curious about the version being used here).

I have committed your patch (commit 87f566e2ac69fc00091df758f0ceb01f890fa00a)
and have added code to warn about unbalanced NSS Init/Shutdown calls (should
never happen) (commit 9447d9164c4430981cefc50148b31d72da700104).

Both commits have been pushed. Please let us know whether these solve the
problem for you.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[brolley at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Dave Brolley <brolley at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |ASSIGNED

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[fche at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #7 from Frank Ch. Eigler <fche at redhat dot com> 2013-02-07 03:58:13 UTC ---
(Whoops, the authorship credit for commit 87f566e2 credits Jiri rather than
Lukas.  Shall we fix that up with an additional dummy commit in Lukas's name?)

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[jiri.horky at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

--- Comment #8 from Jiri Horky <jiri.horky at gmail dot com> 2013-02-07 08:19:20 UTC ---
Hi all,

I confirm that the latest git version fixes the problem  for us. As of
authorship, I would prefer if Lukas was listed as he was the one who found the
cause of segfault and fixed it.

Just to complete the picture, here are the requested information (without -vvvv
output of stap - I would have to downgrade stap and I understood it is no
longer necessary):

root@fe6.du1 ~/systemtap # rpm -qa | grep nss
nss_ldap-262-11.32.31.1
php5-openssl-5.2.14-0.7.30.42.1
libopenssl1_0_0-1.0.0c-18.21.1
insserv-1.12.0-25.3.20
libnsssharedhelper0-1.0.6-1.6
openssl-certs-0.9.8h-27.5.1
openssl-0.9.8j-0.44.1
libopenssl0_9_8-0.9.8j-0.44.1
libnsssharedhelper0-32bit-1.0.6-1.6
libopenssl0_9_8-32bit-0.9.8j-0.44.1
openssh-5.1p1-41.55.1.hpn4
mozilla-nss-32bit-3.13.5-0.4.2
openssh-askpass-5.1p1-41.55.1.hpn4
mozilla-nss-3.13.5-0.4.2

root@fe6.du1 ~/systemtap # rpm -qa | grep nspr
mozilla-nspr-32bit-4.9.1-0.5.1
mozilla-nspr-4.9.1-0.5.1

The stap was invoked normally just as:
stap -v -g /cxfs/clusterbase/admin/accounting/nfsdacc-fullpath.stp

One needs to have a patched nfsd tapset script to be able to use the
"nfsdacc-fullpath.stp" script:


--- tapset/linux/nfsd.stp       2013-01-04 21:22:00.718422707 +0100
+++ tapset/linux/nfsd.stp-lh    2013-01-04 21:10:50.328889987 +0100
@@ -536,6 +536,8 @@
        vec = $rqstp->rq_vec
        vlen = $write->wr_vlen
        stable = $write->wr_stable_how
+       uid = $rqstp->rq_cred->cr_uid;
+       gid = $rqstp->rq_cred->cr_gid;

        name = "nfsd.proc4.write"
        argstr = sprintf("%d,%d",count,offset)

Thank you
Jiri Horky

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug server/15101] [PATCH] Stap sefaults when calling NSS_shutdown() which was not previously initialized

[brolley at redhat dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=15101

Dave Brolley <brolley at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #9 from Dave Brolley <brolley at redhat dot com> 2013-02-08 18:13:27 UTC ---
The originator of this PR has confirmed the fix.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.