SystemTap + Android | Compiling Script | ERROR

SystemTap + Android | Compiling Script | ERROR

[ayush goel]

I’ve been trying to compile stap scripts so that I can run them on my
android. I’ve installed system tap on my linux system, android ndk and
the systemtap android application on my android device. However the
script compilation process is continuously running into error.
I’ve tried the following things:

As per the documentation
(https://sourceware.org/systemtap/wiki/SystemTapOnAndroidARM) I used
the build-module executable providing path to my kernel source code
and the script as arguments however that fails for me giving the
following error:
config/omap.conf: line 1: /root/sukritk/kernel/goldfish/: Is a directory
./build-module: line 92: [: x86: unary operator expected
./build-module: line 92: [: arm: unary operator expected
unknown kernel architecture ''. Check the corresponding file: config/omap.conf

After searching the net, I found that the stap binary inside
installed/bin can also be used instead, therefore I’m running the
following command now
installed/bin/stap  -p 4 -v   -a arm  -B
CROSS_COMPILE=/root/sukritk/android-ndk-r12b/toolchains/bin/arm-linux-androideabi-
 -r /root/sukritk/kernel/goldfish/ -m measure
scripts/monitor_fopen.stp

However the above command fails, with the very first error being
"#error "DebugFS is required and was not found in the kernel.”
I’m assuming I have to recompile my kernel source with some kind of
DEBUGFS flag enabled.

However I’m not really sure how to do that.
Any kind of help or comprehensive steps to compile stap scripts so
that I can run them my android device would be sincerely appreciated.

Thanks and Regards
-Ayush

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 10/25/2016 11:44 AM, ayush goel wrote:
> I’ve been trying to compile stap scripts so that I can run them on my
> android. I’ve installed system tap on my linux system, android ndk and
> the systemtap android application on my android device. However the
> script compilation process is continuously running into error.
> I’ve tried the following things:

... stuff deleted ...

> However the above command fails, with the very first error being
> "#error "DebugFS is required and was not found in the kernel.”
> I’m assuming I have to recompile my kernel source with some kind of
> DEBUGFS flag enabled.
> 
> However I’m not really sure how to do that.
> Any kind of help or comprehensive steps to compile stap scripts so
> that I can run them my android device would be sincerely appreciated.

OK, I've never run systemtap on Android, so I won't be much help there.
As far as DebugFS being required, the flag you are looking for is
CONFIG_DEBUG_FS.

As far as running systemtap on Android, have you looked at the following
link?

<https://sourceware.org/systemtap/wiki/SystemTapOnAndroidARM>

It was updated fairly recently.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: SystemTap + Android | Compiling Script | ERROR

[Alexander Lochmann]

[-- Attachment #1.1: Type: text/plain, Size: 2381 bytes --]

Hi Ayush!

On 10/25/2016 06:44 PM, ayush goel wrote:
> I’ve been trying to compile stap scripts so that I can run them on my
> android. I’ve installed system tap on my linux system, android ndk and
> the systemtap android application on my android device. However the
> script compilation process is continuously running into error.
> I’ve tried the following things:
> 
> As per the documentation
> (https://sourceware.org/systemtap/wiki/SystemTapOnAndroidARM) I used
> the build-module executable providing path to my kernel source code
> and the script as arguments however that fails for me giving the
> following error:
> config/omap.conf: line 1: /root/sukritk/kernel/goldfish/: Is a directory
> ./build-module: line 92: [: x86: unary operator expected
> ./build-module: line 92: [: arm: unary operator expected
> unknown kernel architecture ''. Check the corresponding file: config/omap.conf
It seems you have not set the KERNEL_ARCH variable in your config. Can
you pls paste your omap.conf file?
> 
> After searching the net, I found that the stap binary inside
> installed/bin can also be used instead, therefore I’m running the
> following command now
> installed/bin/stap  -p 4 -v   -a arm  -B
> CROSS_COMPILE=/root/sukritk/android-ndk-r12b/toolchains/bin/arm-linux-androideabi-
>  -r /root/sukritk/kernel/goldfish/ -m measure
> scripts/monitor_fopen.stp
> 
> However the above command fails, with the very first error being
> "#error "DebugFS is required and was not found in the kernel.”
> I’m assuming I have to recompile my kernel source with some kind of
> DEBUGFS flag enabled.
> 
> However I’m not really sure how to do that.
> Any kind of help or comprehensive steps to compile stap scripts so
> that I can run them my android device would be sincerely appreciated.
> 
Have a look at my tutorial on how to build your own Android kernel:
https://sourceware.org/systemtap/wiki/SystemTapOnAndroidARM/BuildingKernel

Although this tutorial might be a bit outdated, it is a good starting
point. :-)

Cheers,
Alex
> Thanks and Regards
> -Ayush
> 


-- 
Technische Universität Dortmund
Alexander Lochmann                PGP key: 0xBC3EF6FD
Otto-Hahn-Str. 16                 phone:  +49.231.7556141
D-44227 Dortmund                  fax:    +49.231.7556116
http://ess.cs.tu-dortmund.de/Staff/al


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 850 bytes --]

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

> > unknown kernel architecture ''. Check the corresponding file: config/omap.conf
> It seems you have not set the KERNEL_ARCH variable in your config. Can
> you pls paste your omap.conf file?

Well, as per the documentation, the conf file should contain the path
to the kernel source tree, that’s it, right?
my omap.conf file contents: /root/sukritk/kernel/goldfish/

> Have a look at my tutorial on how to build your own Android kernel:
> https://sourceware.org/systemtap/wiki/SystemTapOnAndroidARM/BuildingKernel
>
> Although this tutorial might be a bit outdated, it is a good starting
> point. :-)

Sure, though I think I just need to recompile my kernel with
CONFIG_DEBUG_FS flag as David suggested and that error would go away.

Also, which tool should I be using specifically for compiling stap
scripts into modules, for android.
Your documentation talks about using build_module executable, and on
the net I found this another way using the ./installed/bin/stap
executable.

Thanks for getting back to me.

-Ayush

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

>
> Also, which tool should I be using specifically for compiling stap scripts into modules,
> for android.
> Your documentation talks about using build_module executable, and on the net I found
> this another way using the ./installed/bin/stap executable.
>
 Kindly ignore the above comment. I just went through the code of
build-module and it invokes the installed/bin/stap executable itself.

Also, I added the KERNEL_ARCH variable to my conf file and now at
least the script gets fired.

I also recompiled my kernel the DEBUG_FS flag and I no longer see that
error, so that’s some progress.

However the compilation process is still giving me some errors.
---------------------------------------------------------------------------------------------------------------
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:345:14:
error: static declaration of 'vzalloc' follows non-static declaration
 static void *vzalloc(unsigned long size)
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
include/linux/vmalloc.h:68:14: note: previous declaration of 'vzalloc' was here
 extern void *vzalloc(unsigned long size);
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:212:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:380:14:
error: static declaration of 'vmalloc_node' follows non-static
declaration
 static void *vmalloc_node(unsigned long size, int node __attribute__((unused)))
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
include/linux/vmalloc.h:70:14: note: previous declaration of
'vmalloc_node' was here
 extern void *vmalloc_node(unsigned long size, int node);

This is the first of many.

Any kind of assistance would be appreciated.

Thanks.

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 11/02/2016 02:38 PM, ayush goel wrote:
>>
>> Also, which tool should I be using specifically for compiling stap scripts into modules,
>> for android.
>> Your documentation talks about using build_module executable, and on the net I found
>> this another way using the ./installed/bin/stap executable.
>>
>  Kindly ignore the above comment. I just went through the code of
> build-module and it invokes the installed/bin/stap executable itself.
> 
> Also, I added the KERNEL_ARCH variable to my conf file and now at
> least the script gets fired.
> 
> I also recompiled my kernel the DEBUG_FS flag and I no longer see that
> error, so that’s some progress.
> 
> However the compilation process is still giving me some errors.
> ---------------------------------------------------------------------------------------------------------------
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:345:14:
> error: static declaration of 'vzalloc' follows non-static declaration
>  static void *vzalloc(unsigned long size)
>               ^
> In file included from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
>                  from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
>                  from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
> include/linux/vmalloc.h:68:14: note: previous declaration of 'vzalloc' was here
>  extern void *vzalloc(unsigned long size);
>               ^
> In file included from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:212:0,
>                  from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
>                  from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:380:14:
> error: static declaration of 'vmalloc_node' follows non-static
> declaration
>  static void *vmalloc_node(unsigned long size, int node __attribute__((unused)))
>               ^
> In file included from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
>                  from
> /root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
>                  from /tmp/stapeKp7TT/monitor_fopen_src.c:26:
> include/linux/vmalloc.h:70:14: note: previous declaration of
> 'vmalloc_node' was here
>  extern void *vmalloc_node(unsigned long size, int node);

What's going on here is that some linux kernels have a vzalloc()
function and some don't. We attempt to use the kernel's vzalloc()
function if available. If it isn't available, we roll our own.

In your case, the android kernel's include/linux/vmalloc.h file declared
a vzalloc() function but the systemtap build process didn't find it and
tried to use its own version.

Does the script you are using to run systemtap use the '-r BUILD_TREE'
option? I ask because it might be that your host kernel doesn't have a
vzalloc() function but your android kernel does and systemtap is looking
in the wrong place.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

> What's going on here is that some linux kernels have a vzalloc()
> function and some don't. We attempt to use the kernel's vzalloc()
> function if available. If it isn't available, we roll our own.
>
> In your case, the android kernel's include/linux/vmalloc.h file declared
> a vzalloc() function but the systemtap build process didn't find it and
> tried to use its own version.
>
> Does the script you are using to run systemtap use the '-r BUILD_TREE'
> option? I ask because it might be that your host kernel doesn't have a
> vzalloc() function but your android kernel does and systemtap is looking
> in the wrong place.
>
No my script doesn’t use `-r BUILD_TREE` option.
I’m using the arm-linux-androideabi-gcc compiler from android’s ndk
toolchain, so I would like to assume that it’s linker would
automatically search through the required android kernel’s libraries
and headers and find the required definitions.
This is a strange behaviour.

Any possible environment variables I need to set, for the compiler to
look at the correct place?


-Ayush

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

> >
> No my script doesn’t use `-r BUILD_TREE` option.
> I’m using the arm-linux-androideabi-gcc compiler from android’s ndk toolchain, so
> I would like to assume that it’s linker would automatically search through the required
> android kernel’s libraries and headers and find the required definitions.
> This is a strange behaviour.

Sorry for the confusion, but yes my script does make use of the -r
BUILD_TREE option. Should I get rid of that?

-Ayush

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 11/06/2016 01:05 AM, ayush goel wrote:
>>>
>> No my script doesn’t use `-r BUILD_TREE` option.
>> I’m using the arm-linux-androideabi-gcc compiler from android’s ndk toolchain, so
>> I would like to assume that it’s linker would automatically search through the required
>> android kernel’s libraries and headers and find the required definitions.
>> This is a strange behaviour.
> 
> Sorry for the confusion, but yes my script does make use of the -r
> BUILD_TREE option. Should I get rid of that?

No, you need the '-r BUILD_TREE' option so that systemtap will compile a
module for your target kernel, not for your running kernel.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

> No, you need the '-r BUILD_TREE' option so that systemtap will compile a
> module for your target kernel, not for your running kernel.
>

Well is there a work around then, because I can see this error getting
repeated for a couple of definitions.

/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:345:14:
error: static declaration of 'vzalloc' follows non-static declaration
 static void *vzalloc(unsigned long size)
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapEaz4QM/monitor_fopen_src.c:26:
include/linux/vmalloc.h:68:14: note: previous declaration of 'vzalloc' was here
 extern void *vzalloc(unsigned long size);
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:212:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapEaz4QM/monitor_fopen_src.c:26:
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:380:14:
error: static declaration of 'vmalloc_node' follows non-static
declaration
 static void *vmalloc_node(unsigned long size, int node __attribute__((unused)))
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:23:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapEaz4QM/monitor_fopen_src.c:26:
include/linux/vmalloc.h:70:14: note: previous declaration of
'vmalloc_node' was here
 extern void *vmalloc_node(unsigned long size, int node);
              ^
In file included from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/runtime.h:212:0,
                 from
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/runtime.h:24,
                 from /tmp/stapEaz4QM/monitor_fopen_src.c:26:
/root/sukritk/systemtap-android/installed//share/systemtap/runtime/linux/alloc.c:387:14:
error: static declaration of 'vzalloc_node' follows non-static
declaration

Manually going into the alloc.c and commenting these definitions
doesn’t feel like the best solution?

-Ayush

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 11/07/2016 12:12 PM, ayush goel wrote:
>> No, you need the '-r BUILD_TREE' option so that systemtap will compile a
>> module for your target kernel, not for your running kernel.
>>
> 
> Well is there a work around then, because I can see this error getting
> repeated for a couple of definitions.

To fix this problem, you'll have to dig into how systemtap works a bit.
When invoked, one of the things it does is try to figure out if your
kernel exports vzalloc. Systemtap does this by reading your kernel's
Module.symvers file, located in your kernel build tree.

On my 32-bit rawhide system, that file is located in
/usr/src/kernels/4.8.0-0.rc6.git0.1.fc26.i686+PAE/Module.symvers. If I
look for vzalloc in that file, I see the following:

====
# fgrep -i vzalloc
/usr/src/kernels/4.8.0-0.rc6.git0.1.fc26.i686+PAE/Module.symvers
0x00000000	vzalloc_node	vmlinux	EXPORT_SYMBOL
0x00000000	vzalloc	vmlinux	EXPORT_SYMBOL
====

Which reports that vzalloc is present and exported.

If systemtap can't find the Module.symvers file, it will report the
following error:

====
Checking "FILE" failed with error: ERROR
Ensure kernel development headers & makefiles are installed
====

Questions:

1) Do you see that error (or something similar) in your systemtap
invocation output?

2) Can you find the Module.symvers file for your host kernel and target
android kernel?

3) Is vzalloc present in the Module.symvers file for your host kernel
and target android kernel?

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: SystemTap + Android | Compiling Script | ERROR

[Frank Ch. Eigler]

dsmith wrote:

> [...] Questions: [...]

I wonder if we could extend stap-report to collect answers to
those questions.  (e.g., it could take an -r PATH argument and
describe it.)

- FChE

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

>
> 1) Do you see that error (or something similar) in your systemtap
> invocation output?
>
> 2) Can you find the Module.symvers file for your host kernel and target
> android kernel?
>
> 3) Is vzalloc present in the Module.symvers file for your host kernel
> and target android kernel?
>
 Well I just reinstalled the systemtap framework from scratch. And now
when I run the script, I’m getting the error you just mentioned.
--------------------
Checking "/root/sukritk/kernel/samsung/Module.symvers" failed with
error: No such file or directory
Ensure kernel development headers & makefiles are installed
———————————

This is extremely strange, because my kernel hasn’t been changed, and
previously it was working fine.

My kernel compile steps:
make clean -j4 ARCH=arm SUBARCH=arm CROSS_COMPILE=arm-linux-androideabi-
make ARCH=arm SUBARCH=arm CROSS_COMPILE=arm-linux-androideabi-
slim_i9300_defconfig
gedit .config

    # Enable config parameters:
    CONFIG_DEBUG_INFO, CONFIG_KPROBES, CONFIG_RELAY, CONFIG_DEBUG_FS,
CONFIG_MODULES, CONFIG_MODULE_UNLOAD

make -j4 ARCH=arm SUBARCH=arm CROSS_COMPILE=arm-linux-androideabi-

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 11/09/2016 12:33 PM, ayush goel wrote:
>>
>> 1) Do you see that error (or something similar) in your systemtap
>> invocation output?
>>
>> 2) Can you find the Module.symvers file for your host kernel and target
>> android kernel?
>>
>> 3) Is vzalloc present in the Module.symvers file for your host kernel
>> and target android kernel?
>>
>  Well I just reinstalled the systemtap framework from scratch. And now
> when I run the script, I’m getting the error you just mentioned.
> --------------------
> Checking "/root/sukritk/kernel/samsung/Module.symvers" failed with
> error: No such file or directory
> Ensure kernel development headers & makefiles are installed

OK, what is in the /root/sukritk/kernel/samsung directory? Let's make
sure it is finding the right directory.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

Re: SystemTap + Android | Compiling Script | ERROR

[ayush goel]

>
> OK, what is in the /root/sukritk/kernel/samsung directory? Let's make
> sure it is finding the right directory.
>
Well it contained the source code of my samsung kernel.

Anyhow, I just compiled a different kernel (goldfish).
It contains the Module.symvars file.

However the systemtap script still gives me the vzalloc error.

Using fgrep like you suggested gives me nothing.

————
root@irldxph010:goldfish# fgrep -i vzalloc ../../kernel/goldfish/Module.symvers
root@irldxph010:goldfish#
————

Re: SystemTap + Android | Compiling Script | ERROR

[Alexander Lochmann]

[-- Attachment #1.1: Type: text/plain, Size: 1003 bytes --]

On 11/10/2016 07:28 AM, ayush goel wrote:
>>
>> OK, what is in the /root/sukritk/kernel/samsung directory? Let's make
>> sure it is finding the right directory.
>>
> Well it contained the source code of my samsung kernel.
Mhmh. That sounds strange.
Can you pls have a look at your kernel directory, and ensure that the
kernel is build completely?

- ALex
> 
> Anyhow, I just compiled a different kernel (goldfish).
> It contains the Module.symvars file.
> 
> However the systemtap script still gives me the vzalloc error.
> 
> Using fgrep like you suggested gives me nothing.
> 
> ————
> root@irldxph010:goldfish# fgrep -i vzalloc ../../kernel/goldfish/Module.symvers
> root@irldxph010:goldfish#
> ————
> 


-- 
Technische Universität Dortmund
Alexander Lochmann                PGP key: 0xBC3EF6FD
Otto-Hahn-Str. 16                 phone:  +49.231.7556141
D-44227 Dortmund                  fax:    +49.231.7556116
http://ess.cs.tu-dortmund.de/Staff/al


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 850 bytes --]

Re: SystemTap + Android | Compiling Script | ERROR

[David Smith]

On 11/10/2016 12:28 AM, ayush goel wrote:
>>
>> OK, what is in the /root/sukritk/kernel/samsung directory? Let's make
>> sure it is finding the right directory.
>>
> Well it contained the source code of my samsung kernel.

I was looking for other files that should have been there, like System.map.

> Anyhow, I just compiled a different kernel (goldfish).
> It contains the Module.symvars file.
> 
> However the systemtap script still gives me the vzalloc error.
> 
> Using fgrep like you suggested gives me nothing.
> 
> ————
> root@irldxph010:goldfish# fgrep -i vzalloc ../../kernel/goldfish/Module.symvers
> root@irldxph010:goldfish#
> ————

OK, that's weird. According to your Module.symvers file, your kernel
doesn't have a vzalloc. But your compilation errors seemed to indicate
your kernel header files contain a declaration for vzalloc.

-- 
David Smith
dsmith@redhat.com
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)