Linking with external lib

Linking with external lib

[Mahmoud Hanafi]

In my stap script I have some embedded C function that needs to make 
some calls to external libraries. Is there I can link my script with the 
external libs? Or do i need to add all the code I need into my script.


-- 
Mahmoud Hanafi

Re: Linking with external lib

[David Smith]

On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
> In my stap script I have some embedded C function that needs to make
> some calls to external libraries. Is there I can link my script with the
> external libs? Or do i need to add all the code I need into my script.

Basically you can't. A systemtap script gets compiled into a kernel
module (assuming you are using the default 'linux' runtime). A kernel
module can't access *any* user library.

Perhaps you should back up a bit and let us know what your external
library is trying to do.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Linking with external lib

[Mahmoud Hanafi]

On 09/06/2016 11:55 AM, David Smith wrote:
> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>> In my stap script I have some embedded C function that needs to make
>> some calls to external libraries. Is there I can link my script with the
>> external libs? Or do i need to add all the code I need into my script.
> Basically you can't. A systemtap script gets compiled into a kernel
> module (assuming you are using the default 'linux' runtime). A kernel
> module can't access *any* user library.
>
> Perhaps you should back up a bit and let us know what your external
> library is trying to do.
>
The library is part of lustre. I need the functions that does address 
translation to string values.

-- 
Mahmoud Hanafi
NASA Ames Research Center
Mail Stop 258-6
Bldg. 258
P.O. Box 1
Moffett Field, CA 94035-0001

Office: 650-604-1454
Cell: 510-386-0615

Re: Linking with external lib

[David Smith]

On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
> 
> 
> On 09/06/2016 11:55 AM, David Smith wrote:
>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>> In my stap script I have some embedded C function that needs to make
>>> some calls to external libraries. Is there I can link my script with the
>>> external libs? Or do i need to add all the code I need into my script.
>> Basically you can't. A systemtap script gets compiled into a kernel
>> module (assuming you are using the default 'linux' runtime). A kernel
>> module can't access *any* user library.
>>
>> Perhaps you should back up a bit and let us know what your external
>> library is trying to do.
>>
> The library is part of lustre. I need the functions that does address
> translation to string values.

Can you point me to the library/function on lustre.org that you are
interested in?

You can't really copy userspace C code into systemtap. Systemtap tries
very hard to handle memory accesses safely (since otherwise you can
crash the kernel). We might be able to "translate" the userspace C code
into embedded C.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Linking with external lib

[Mahmoud Hanafi]

On 09/06/2016 12:34 PM, David Smith wrote:
> On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
>>
>> On 09/06/2016 11:55 AM, David Smith wrote:
>>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>>> In my stap script I have some embedded C function that needs to make
>>>> some calls to external libraries. Is there I can link my script with the
>>>> external libs? Or do i need to add all the code I need into my script.
>>> Basically you can't. A systemtap script gets compiled into a kernel
>>> module (assuming you are using the default 'linux' runtime). A kernel
>>> module can't access *any* user library.
>>>
>>> Perhaps you should back up a bit and let us know what your external
>>> library is trying to do.
>>>
>> The library is part of lustre. I need the functions that does address
>> translation to string values.
> Can you point me to the library/function on lustre.org that you are
> interested in?
>
> You can't really copy userspace C code into systemtap. Systemtap tries
> very hard to handle memory accesses safely (since otherwise you can
> crash the kernel). We might be able to "translate" the userspace C code
> into embedded C.
the function is defined here
http://osxr.org:8080/linux/source/drivers/staging/lustre/lustre/libcfs/nidstrings.c#0382

"libcfs_nid2str"

It would be nice to compile in something like this, because if there are 
changes later in the function it wouldn't require stap script changes.


-- 
Mahmoud Hanafi
NASA Ames Research Center
Mail Stop 258-6
Bldg. 258
P.O. Box 1
Moffett Field, CA 94035-0001

Office: 650-604-1454
Cell: 510-386-0615

Re: Linking with external lib

[Josh Stone]

On 09/06/2016 12:39 PM, Mahmoud Hanafi wrote:
> 
> 
> On 09/06/2016 12:34 PM, David Smith wrote:
>> On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
>>>
>>> On 09/06/2016 11:55 AM, David Smith wrote:
>>>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>>>> In my stap script I have some embedded C function that needs to make
>>>>> some calls to external libraries. Is there I can link my script with the
>>>>> external libs? Or do i need to add all the code I need into my script.
>>>> Basically you can't. A systemtap script gets compiled into a kernel
>>>> module (assuming you are using the default 'linux' runtime). A kernel
>>>> module can't access *any* user library.
>>>>
>>>> Perhaps you should back up a bit and let us know what your external
>>>> library is trying to do.
>>>>
>>> The library is part of lustre. I need the functions that does address
>>> translation to string values.
>> Can you point me to the library/function on lustre.org that you are
>> interested in?
>>
>> You can't really copy userspace C code into systemtap. Systemtap tries
>> very hard to handle memory accesses safely (since otherwise you can
>> crash the kernel). We might be able to "translate" the userspace C code
>> into embedded C.
> the function is defined here
> http://osxr.org:8080/linux/source/drivers/staging/lustre/lustre/libcfs/nidstrings.c#0382
> 
> "libcfs_nid2str"
> 
> It would be nice to compile in something like this, because if there are 
> changes later in the function it wouldn't require stap script changes.

What are your probe locations?  Would it be possible for you to use SDT
probes, where you prepare these strings as arguments ahead of time?

That's what the python function probes do, for instance.  See
systemtap_function_entry() in this patch:
http://pkgs.fedoraproject.org/cgit/rpms/python3.git/tree/00055-systemtap.patch

Re: Linking with external lib

[William Cohen]

On 09/06/2016 03:39 PM, Mahmoud Hanafi wrote:
> 
> 
> On 09/06/2016 12:34 PM, David Smith wrote:
>> On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
>>>
>>> On 09/06/2016 11:55 AM, David Smith wrote:
>>>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>>>> In my stap script I have some embedded C function that needs to make
>>>>> some calls to external libraries. Is there I can link my script with the
>>>>> external libs? Or do i need to add all the code I need into my script.
>>>> Basically you can't. A systemtap script gets compiled into a kernel
>>>> module (assuming you are using the default 'linux' runtime). A kernel
>>>> module can't access *any* user library.
>>>>
>>>> Perhaps you should back up a bit and let us know what your external
>>>> library is trying to do.
>>>>
>>> The library is part of lustre. I need the functions that does address
>>> translation to string values.
>> Can you point me to the library/function on lustre.org that you are
>> interested in?
>>
>> You can't really copy userspace C code into systemtap. Systemtap tries
>> very hard to handle memory accesses safely (since otherwise you can
>> crash the kernel). We might be able to "translate" the userspace C code
>> into embedded C.
> the function is defined here
> http://osxr.org:8080/linux/source/drivers/staging/lustre/lustre/libcfs/nidstrings.c#0382
> 
> "libcfs_nid2str"
> 
> It would be nice to compile in something like this, because if there are changes later in the function it wouldn't require stap script changes.
> 
> 

Hi,

Isn't libcfs_nid2str a kernel function?  For some kernel functions one could use systemtap guru mode to directly call c functions in the kernel.  However, it doesn't look like this particular function is visible to code outside the lustre module.

You might be able to include <linux/libcfs/libcfs.h> and clone the libcfs_nid2str function using guru mode (-g) as a work around. At least that would allow access to the various defines and macros.  Use the systemtap/tapset/linux/tcp.stp as an example of how to do the include and systemtap functions (https://www.sourceware.org/git/?p=systemtap.git;a=blob;f=tapset/linux/tcp.stp;h=22fe2f08fda55c989d33cdcd776fa00736f4b7dd;hb=HEAD).  Not an ideal solution, but it might get you closer to what you want.

-Will

Re: Linking with external lib

[David Smith]

On 09/07/2016 10:24 AM, William Cohen wrote:
> On 09/06/2016 03:39 PM, Mahmoud Hanafi wrote:
>>
>>
>> On 09/06/2016 12:34 PM, David Smith wrote:
>>> On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
>>>>
>>>> On 09/06/2016 11:55 AM, David Smith wrote:
>>>>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>>>>> In my stap script I have some embedded C function that needs to make
>>>>>> some calls to external libraries. Is there I can link my script with the
>>>>>> external libs? Or do i need to add all the code I need into my script.
>>>>> Basically you can't. A systemtap script gets compiled into a kernel
>>>>> module (assuming you are using the default 'linux' runtime). A kernel
>>>>> module can't access *any* user library.
>>>>>
>>>>> Perhaps you should back up a bit and let us know what your external
>>>>> library is trying to do.
>>>>>
>>>> The library is part of lustre. I need the functions that does address
>>>> translation to string values.
>>> Can you point me to the library/function on lustre.org that you are
>>> interested in?
>>>
>>> You can't really copy userspace C code into systemtap. Systemtap tries
>>> very hard to handle memory accesses safely (since otherwise you can
>>> crash the kernel). We might be able to "translate" the userspace C code
>>> into embedded C.
>> the function is defined here
>> http://osxr.org:8080/linux/source/drivers/staging/lustre/lustre/libcfs/nidstrings.c#0382
>>
>> "libcfs_nid2str"
>>
>> It would be nice to compile in something like this, because if there are changes later in the function it wouldn't require stap script changes.
>>
>>
> 
> Hi,
> 
> Isn't libcfs_nid2str a kernel function?  For some kernel functions one
> could use systemtap guru mode to directly call c functions in the kernel.
> However, it doesn't look like this particular function is visible to code
> outside the lustre module.
> 
> You might be able to include <linux/libcfs/libcfs.h> and clone the libcfs_nid2str
> function using guru mode (-g) as a work around. At least that would
allow access
> to the various defines and macros.  Use the
systemtap/tapset/linux/tcp.stp as an
> example of how to do the include and systemtap functions
>
(https://www.sourceware.org/git/?p=systemtap.git;a=blob;f=tapset/linux/tcp.stp;h=22fe2f08fda55c989d33cdcd776fa00736f4b7dd;hb=HEAD).
> Not an ideal solution, but it might get you closer to what you want.

I've looked at the kernel's libcfs_nid2str() function a bit. Cloning the
function really isn't going to work. The function uses an internal
lookup table that you won't have access to.

If the nid can't be found in the lookup table, libcfs_nid2str() returns
a default string. Here's a simplified version of libcfs_nid2str() that
shows how it creates the default string:

	__u32 addr = LNET_NIDADDR(nid);
	__u32 net = LNET_NIDNET(nid);
	__u32 nnum = LNET_NETNUM(net);
	__u32 lnd = LNET_NETTYP(net);
	...
	snprintf(buf, buf_size, "%x@<%u:%u>", addr, lnd, nnum);

If that's acceptable, it should be fairly easy to do in systemtap.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: Linking with external lib

[Mahmoud Hanafi]

Thank you I'll give that a try.



On 09/08/2016 11:17 AM, David Smith wrote:
> On 09/07/2016 10:24 AM, William Cohen wrote:
>> On 09/06/2016 03:39 PM, Mahmoud Hanafi wrote:
>>>
>>> On 09/06/2016 12:34 PM, David Smith wrote:
>>>> On 09/06/2016 02:08 PM, Mahmoud Hanafi wrote:
>>>>> On 09/06/2016 11:55 AM, David Smith wrote:
>>>>>> On 09/06/2016 01:37 PM, Mahmoud Hanafi wrote:
>>>>>>> In my stap script I have some embedded C function that needs to make
>>>>>>> some calls to external libraries. Is there I can link my script with the
>>>>>>> external libs? Or do i need to add all the code I need into my script.
>>>>>> Basically you can't. A systemtap script gets compiled into a kernel
>>>>>> module (assuming you are using the default 'linux' runtime). A kernel
>>>>>> module can't access *any* user library.
>>>>>>
>>>>>> Perhaps you should back up a bit and let us know what your external
>>>>>> library is trying to do.
>>>>>>
>>>>> The library is part of lustre. I need the functions that does address
>>>>> translation to string values.
>>>> Can you point me to the library/function on lustre.org that you are
>>>> interested in?
>>>>
>>>> You can't really copy userspace C code into systemtap. Systemtap tries
>>>> very hard to handle memory accesses safely (since otherwise you can
>>>> crash the kernel). We might be able to "translate" the userspace C code
>>>> into embedded C.
>>> the function is defined here
>>> http://osxr.org:8080/linux/source/drivers/staging/lustre/lustre/libcfs/nidstrings.c#0382
>>>
>>> "libcfs_nid2str"
>>>
>>> It would be nice to compile in something like this, because if there are changes later in the function it wouldn't require stap script changes.
>>>
>>>
>> Hi,
>>
>> Isn't libcfs_nid2str a kernel function?  For some kernel functions one
>> could use systemtap guru mode to directly call c functions in the kernel.
>> However, it doesn't look like this particular function is visible to code
>> outside the lustre module.
>>
>> You might be able to include <linux/libcfs/libcfs.h> and clone the libcfs_nid2str
>> function using guru mode (-g) as a work around. At least that would
> allow access
>> to the various defines and macros.  Use the
> systemtap/tapset/linux/tcp.stp as an
>> example of how to do the include and systemtap functions
>>
> (https://www.sourceware.org/git/?p=systemtap.git;a=blob;f=tapset/linux/tcp.stp;h=22fe2f08fda55c989d33cdcd776fa00736f4b7dd;hb=HEAD).
>> Not an ideal solution, but it might get you closer to what you want.
> I've looked at the kernel's libcfs_nid2str() function a bit. Cloning the
> function really isn't going to work. The function uses an internal
> lookup table that you won't have access to.
>
> If the nid can't be found in the lookup table, libcfs_nid2str() returns
> a default string. Here's a simplified version of libcfs_nid2str() that
> shows how it creates the default string:
>
> 	__u32 addr = LNET_NIDADDR(nid);
> 	__u32 net = LNET_NIDNET(nid);
> 	__u32 nnum = LNET_NETNUM(net);
> 	__u32 lnd = LNET_NETTYP(net);
> 	...
> 	snprintf(buf, buf_size, "%x@<%u:%u>", addr, lnd, nnum);
>
> If that's acceptable, it should be fairly easy to do in systemtap.
>

-- 
Mahmoud Hanafi
NASA Ames Research Center
Mail Stop 258-6
Bldg. 258
P.O. Box 1
Moffett Field, CA 94035-0001

Office: 650-604-1454
Cell: 510-386-0615