Problem setting up systemtap 1.6 client/server on Ubutu 11.10

Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Alex Wolfson]

[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]

Hi all,

I am having problems setting up systemtap client/server and would like 
to find out,
  if it is already known problem before I start debugging the systemtap 
code.

Environment:
------------------

I am using ubuntu oneiric 11.10 and systemtap 1.6.
I got the source from git tagged "release-1.6" and built myself. I 
believe this is the version that debian uses
The build, in case someone needs it, is available in 
https://launchpad.net/~awolfson/+archive/systemtap

Unfortunately, I did not find instructions on setting stap-client/server 
on Debian derivatives.
I tried to make a script to automate setup. It is attached

Couple of things seems to go wrong:
-------------------------------------------------

1. sudo su -s /bin/sh - stap-server -c  /usr/lib/systemtap/stap-gen-cert

As far as I understand, it suppose to ask for password but it does not 
happened.
However the certificate is generated and put in 
/var/lib/stapserver/.systemtap/ssl/server/stap.cert

2. sudo su -s /bin/sh - stap-server -c /usr/lib/systemtap/stap-start-server

[sudo] password for awolfson:
/usr/lib/systemtap/stap-start-server: line 51:  6460 
Aborted                 ${stap_pkglibexecdir}stap-serverd "$@" < 
/dev/null > /dev/null 2> /dev/null
Unable to start a systemtap server: /usr/lib/systemtap/stap-serverd did 
not start

3. Attempt to manually start the daemon
-------------------------------------------------------
awolfson@awolfson-T410:/usr/lib/systemtap$ ./stap-serverd
terminate called after throwing an instance of 'std::logic_error'
   what():  basic_string::_S_construct null not valid
Aborted

Sorry for that long Email, but I would be greatly appreciated some feedback.
If this is unknown issue, I will try to debug it.
I did not try other systemtap versions yet.

Thanks,

Alex


[-- Attachment #2: stap-setup-server.sh --]
[-- Type: application/x-shellscript, Size: 2439 bytes --]

Re: Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Josh Stone]

On 10/20/2011 12:57 PM, Alex Wolfson wrote:
> 3. Attempt to manually start the daemon
> -------------------------------------------------------
> awolfson@awolfson-T410:/usr/lib/systemtap$ ./stap-serverd
> terminate called after throwing an instance of 'std::logic_error'
>    what():  basic_string::_S_construct null not valid
> Aborted

I believe that's a programming error, trying to initialize some string
with a (char*)NULL.

> Sorry for that long Email, but I would be greatly appreciated some feedback.
> If this is unknown issue, I will try to debug it.
> I did not try other systemtap versions yet.

This is not a known issue AFAIK, so we'd welcome your debugging.  Even
just a backtrace will probably let us track down the NULL.

Thanks!
Josh

Re: Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Josh Stone]

For some reason, your mailer or the list server mangled your reply, but
picking out pieces...

Alex Wolfson wrote:
> Hi Josh,
> 
> Thank you for the feedback.
> 
> I found the place when stap-serverd is failing
> it is in stap-serverd.cxx
> 
> static void
> initialize (int argc, char **argv) {
> .....
>    string login = getlogin ();
> getlogin returns NULL

> |getlogin| is an unsafe and deprecated way of determining the logged-in 
> user. It's probably trying to open a record of logged-in users, perhaps 
> |utmp| or something. The correct way to determine the user you're 
> running as (which might not be the same as the logged-in user, but is 
> almost always better to use anyway) is |getpwuid(getuid())|
> 
> So after I replaced
> 
> string login = getlogin ();
> 
> with
> 
> struct passwd *pwd;
> pwd = getpwuid(getuid());
> string login = pwd->pw_name;
> 
> I can now manualy run
> 
> ./stap-serverd
> 
>   without std::logic_error
> 
> Looks like this bug is specific ether to Ubuntu or my setup.
> 
> I hope to find time to finish my script next week and provide a patch 
> for getlogin and whatever else will show up.
> Probably replacing getlogin() call is a good idea anyway.
> 
> Thanks,
> 
> Alex

It looks like we did run into this bug, and Dave already fixed it:

https://bugzilla.redhat.com/show_bug.cgi?id=737095

commit 3845290458cd498e490078318ae5306911cb47a9
Author: Dave Brolley <brolley@redhat.com>
Date:   Tue Sep 13 11:07:11 2011 -0400

    RHBZ 737095: Unable to start a systemtap server

    - Don't use getlogin to obtain user name because it can fail.
    - Establish server log before issuing any messages.
    - Failure of avahi to advertise is now only a warning. Avahi is not
      always available and the server can still be used directly.

There are two other commits mentioned in the bug too, but this is the
one make a similar getlogin change as you propose.

HTH,
Josh

Re: Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Alex Wolfson]

Hi Josh,

I had "compose in HTML" set in thunderbird, but never had message 
screwed up so badly before. I disabled HTML, hope it will prevent 
messages from screwing again.

I applied the pacth you mentioned and was able to build and setup 
client/server. I used my laptop as a server. The only minor discrepancy 
- server was not discovered automatically, so I need  add "--use-server 
IP" to stap command. I like client/server a lot - much nicer then 
manually copying modules around or install all debug symbols on system 
under test.

I need to test more the scripts and the build and will put client/server 
setups scripts to that list, hopefully next week. I will also put 
prebuilt .deb packages into PPA, so ubuntu users would be able to use 
version 1.6 of the systemtap now, while it makes its way to official 
pool. It is currently 1.4

Thank you very match for your help,

Alex

On 10/21/2011 02:18 AM, Josh Stone wrote:
> For some reason, your mailer or the list server mangled your reply, but
> picking out pieces...
>
> Alex Wolfson wrote:
>> Hi Josh,
>>
>> Thank you for the feedback.
>>
>> I found the place when stap-serverd is failing
>> it is in stap-serverd.cxx
>>
>> static void
>> initialize (int argc, char **argv) {
>> .....
>>     string login = getlogin ();
>> getlogin returns NULL
>
>> |getlogin| is an unsafe and deprecated way of determining the logged-in
>> user. It's probably trying to open a record of logged-in users, perhaps
>> |utmp| or something. The correct way to determine the user you're
>> running as (which might not be the same as the logged-in user, but is
>> almost always better to use anyway) is |getpwuid(getuid())|
>>
>> So after I replaced
>>
>> string login = getlogin ();
>>
>> with
>>
>> struct passwd *pwd;
>> pwd = getpwuid(getuid());
>> string login = pwd->pw_name;
>>
>> I can now manualy run
>>
>> ./stap-serverd
>>
>>    without std::logic_error
>>
>> Looks like this bug is specific ether to Ubuntu or my setup.
>>
>> I hope to find time to finish my script next week and provide a patch
>> for getlogin and whatever else will show up.
>> Probably replacing getlogin() call is a good idea anyway.
>>
>> Thanks,
>>
>> Alex
>
> It looks like we did run into this bug, and Dave already fixed it:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=737095
>
> commit 3845290458cd498e490078318ae5306911cb47a9
> Author: Dave Brolley<brolley@redhat.com>
> Date:   Tue Sep 13 11:07:11 2011 -0400
>
>      RHBZ 737095: Unable to start a systemtap server
>
>      - Don't use getlogin to obtain user name because it can fail.
>      - Establish server log before issuing any messages.
>      - Failure of avahi to advertise is now only a warning. Avahi is not
>        always available and the server can still be used directly.
>
> There are two other commits mentioned in the bug too, but this is the
> one make a similar getlogin change as you propose.
>
> HTH,
> Josh
>

Re: Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Dave Brolley]

This is a known problem and has been fixed. See: 
https://bugzilla.redhat.com/show_bug.cgi?id=737095

Dave

On 10/20/2011 04:28 PM, Josh Stone wrote:
> On 10/20/2011 12:57 PM, Alex Wolfson wrote:
>> 3. Attempt to manually start the daemon
>> -------------------------------------------------------
>> awolfson@awolfson-T410:/usr/lib/systemtap$ ./stap-serverd
>> terminate called after throwing an instance of 'std::logic_error'
>>     what():  basic_string::_S_construct null not valid
>> Aborted
> I believe that's a programming error, trying to initialize some string
> with a (char*)NULL.
>
>> Sorry for that long Email, but I would be greatly appreciated some feedback.
>> If this is unknown issue, I will try to debug it.
>> I did not try other systemtap versions yet.
> This is not a known issue AFAIK, so we'd welcome your debugging.  Even
> just a backtrace will probably let us track down the NULL.
>
> Thanks!
> Josh

Re: Problem setting up systemtap 1.6 client/server on Ubutu 11.10

[Alex Wolfson]

Thanks David,

I already tried that fix and it indeed made client/server working

Alex

On 10/23/2011 11:02 PM, Dave Brolley wrote:
> This is a known problem and has been fixed. See:
> https://bugzilla.redhat.com/show_bug.cgi?id=737095
>
> Dave
>
> On 10/20/2011 04:28 PM, Josh Stone wrote:
>> On 10/20/2011 12:57 PM, Alex Wolfson wrote:
>>> 3. Attempt to manually start the daemon
>>> -------------------------------------------------------
>>> awolfson@awolfson-T410:/usr/lib/systemtap$ ./stap-serverd
>>> terminate called after throwing an instance of 'std::logic_error'
>>> what(): basic_string::_S_construct null not valid
>>> Aborted
>> I believe that's a programming error, trying to initialize some string
>> with a (char*)NULL.
>>
>>> Sorry for that long Email, but I would be greatly appreciated some
>>> feedback.
>>> If this is unknown issue, I will try to debug it.
>>> I did not try other systemtap versions yet.
>> This is not a known issue AFAIK, so we'd welcome your debugging. Even
>> just a backtrace will probably let us track down the NULL.
>>
>> Thanks!
>> Josh
>