arm64 kprobes patches

arm64 kprobes patches

[Sandeepa Prabhu]

Hi all,

We have uploaded arm64 kprobes latest changes on linaro git which
include some fixes for recursive kprobes. These are rebased on top of
3.14-rc3.  Kprobes support can be enabled by enabling
"General setup  --->  [*] Kprobes" config option.

Git location:  git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git
 Branch: arm64_kprobes_3.14-rc3

Please let us know if you are interested in pulling this branch for
systemtap test-suite verification, while we are trying to include
systemtap test-suite on our linaro open-embedded platforms.

Also, please let us know if there are documentation on howto run full
test-suite for systemtap and various criteria  kprobes should meet?

Thanks & regards,
Sandeepa

Re: arm64 kprobes patches

[Frank Ch. Eigler]

Sandeepa Prabhu <sandeepa.prabhu@linaro.org> writes:

> Hi all,
> We have uploaded arm64 kprobes latest changes on linaro git which
> include some fixes for recursive kprobes.  [...]

Great.

> Please let us know if you are interested in pulling this branch for
> systemtap test-suite verification, while we are trying to include
> systemtap test-suite on our linaro open-embedded platforms.

We will try soon.

> Also, please let us know if there are documentation on howto run full
> test-suite for systemtap and various criteria  kprobes should meet?

The systemtap README includes these words:

  [...]
  To run a simple test.
  # stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

  To run the full test suite from the build tree.
  # make installcheck
  [...]

The full installcheck suite exercises kprobes some, but not in a
stressful mode, like

      probe kernel.function("*").* {}

That's partly because the x86 kernel has never been able to survive
such a test.  It would be great if arm kprobes were robust enough.


- FChE

Re: arm64 kprobes patches

[Masami Hiramatsu]

(2014/02/27 0:02), Sandeepa Prabhu wrote:
> Hi all,
> 
> We have uploaded arm64 kprobes latest changes on linaro git which
> include some fixes for recursive kprobes. These are rebased on top of
> 3.14-rc3.  Kprobes support can be enabled by enabling
> "General setup  --->  [*] Kprobes" config option.
> 
> Git location:  git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git
>  Branch: arm64_kprobes_3.14-rc3

Great! Thank you for working on that!

> 
> Please let us know if you are interested in pulling this branch for
> systemtap test-suite verification, while we are trying to include
> systemtap test-suite on our linaro open-embedded platforms.
> 
> Also, please let us know if there are documentation on howto run full
> test-suite for systemtap and various criteria  kprobes should meet?

One possible testing is putting kprobes on all functions in the
kernel by using ftrace interface. (On x86, current kprobes still has
some problems, and I solved, will send the series later).

You can put kprobes on all kernel functions
  ----
#!/bin/sh
TRACE_DIR=/sys/kernel/debug/tracing/
grep -iw t /proc/kallsyms | tr -d . | awk 'BEGIN{i=0};{print("p:"$3"_"i, "0x"$1); i++}' | \
while read l; do echo $l >> $TRACE_DIR/kprobe_events || echo "Failed($?): $l" ; done
  ----
Note that this just register *disabled* kprobes on each function.
To enable each one, you need to write "1" to $TRACE_DIR/events/kprobes/<EVENT>/enable.
# I recommend you to enable one by one, there still be a performance problem.

Thank you,
-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: arm64 kprobes patches

[Sandeepa Prabhu]

On 26 February 2014 22:06, Frank Ch. Eigler <fche@redhat.com> wrote:
> Sandeepa Prabhu <sandeepa.prabhu@linaro.org> writes:
>
>> Hi all,
>> We have uploaded arm64 kprobes latest changes on linaro git which
>> include some fixes for recursive kprobes.  [...]
>
> Great.
>
>> Please let us know if you are interested in pulling this branch for
>> systemtap test-suite verification, while we are trying to include
>> systemtap test-suite on our linaro open-embedded platforms.
>
> We will try soon.
Thanks, we will start verifying from our end too.
>
>> Also, please let us know if there are documentation on howto run full
>> test-suite for systemtap and various criteria  kprobes should meet?
>
> The systemtap README includes these words:
>
>   [...]
>   To run a simple test.
>   # stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
>
>   To run the full test suite from the build tree.
>   # make installcheck
>   [...]
>
> The full installcheck suite exercises kprobes some, but not in a
> stressful mode, like
>
>       probe kernel.function("*").* {}
>
> That's partly because the x86 kernel has never been able to survive
> such a test.  It would be great if arm kprobes were robust enough.
Even on arm64, there are kernel functions that fails for kprobes,
(example: memcpy, memset etc) that are referenced by kprobe handler
themselves.
>
>
> - FChE

Re: arm64 kprobes patches

[Sandeepa Prabhu]

On 27 February 2014 07:44, Masami Hiramatsu
<masami.hiramatsu.pt@hitachi.com> wrote:
> (2014/02/27 0:02), Sandeepa Prabhu wrote:
>> Hi all,
>>
>> We have uploaded arm64 kprobes latest changes on linaro git which
>> include some fixes for recursive kprobes. These are rebased on top of
>> 3.14-rc3.  Kprobes support can be enabled by enabling
>> "General setup  --->  [*] Kprobes" config option.
>>
>> Git location:  git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git
>>  Branch: arm64_kprobes_3.14-rc3
>
> Great! Thank you for working on that!
>
>>
>> Please let us know if you are interested in pulling this branch for
>> systemtap test-suite verification, while we are trying to include
>> systemtap test-suite on our linaro open-embedded platforms.
>>
>> Also, please let us know if there are documentation on howto run full
>> test-suite for systemtap and various criteria  kprobes should meet?
>
> One possible testing is putting kprobes on all functions in the
> kernel by using ftrace interface. (On x86, current kprobes still has
> some problems, and I solved, will send the series later)
OK we will try to verify this with arm64 ftrace patches (from linaro)
which are close to get accepted upstream.
Has your earlier patches (kprobes blacklist related) been queued-up
for upstream? Are you planning generic kprobes API changes ?

>
> You can put kprobes on all kernel functions
>   ----
> #!/bin/sh
> TRACE_DIR=/sys/kernel/debug/tracing/
> grep -iw t /proc/kallsyms | tr -d . | awk 'BEGIN{i=0};{print("p:"$3"_"i, "0x"$1); i++}' | \
> while read l; do echo $l >> $TRACE_DIR/kprobe_events || echo "Failed($?): $l" ; done
>   ----
> Note that this just register *disabled* kprobes on each function.
> To enable each one, you need to write "1" to $TRACE_DIR/events/kprobes/<EVENT>/enable.
> # I recommend you to enable one by one, there still be a performance problem.
>
> Thank you,
> --
> Masami HIRAMATSU
> IT Management Research Dept. Linux Technology Center
> Hitachi, Ltd., Yokohama Research Laboratory
> E-mail: masami.hiramatsu.pt@hitachi.com
>
>

Re: arm64 kprobes patches

[Masami Hiramatsu]

(2014/02/27 13:56), Sandeepa Prabhu wrote:
> On 27 February 2014 07:44, Masami Hiramatsu
> <masami.hiramatsu.pt@hitachi.com> wrote:
>> (2014/02/27 0:02), Sandeepa Prabhu wrote:
>>> Hi all,
>>>
>>> We have uploaded arm64 kprobes latest changes on linaro git which
>>> include some fixes for recursive kprobes. These are rebased on top of
>>> 3.14-rc3.  Kprobes support can be enabled by enabling
>>> "General setup  --->  [*] Kprobes" config option.
>>>
>>> Git location:  git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git
>>>  Branch: arm64_kprobes_3.14-rc3
>>
>> Great! Thank you for working on that!
>>
>>>
>>> Please let us know if you are interested in pulling this branch for
>>> systemtap test-suite verification, while we are trying to include
>>> systemtap test-suite on our linaro open-embedded platforms.
>>>
>>> Also, please let us know if there are documentation on howto run full
>>> test-suite for systemtap and various criteria  kprobes should meet?
>>
>> One possible testing is putting kprobes on all functions in the
>> kernel by using ftrace interface. (On x86, current kprobes still has
>> some problems, and I solved, will send the series later)
> OK we will try to verify this with arm64 ftrace patches (from linaro)
> which are close to get accepted upstream.
> Has your earlier patches (kprobes blacklist related) been queued-up
> for upstream?

Yes, I updated and will post the series soon. :)

> Are you planning generic kprobes API changes ?

No, actually generic API is still same. But internal API will
be added.

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com

Re: arm64 kprobes patches

[Sandeepa Prabhu]

On 27 February 2014 11:34, Masami Hiramatsu
<masami.hiramatsu.pt@hitachi.com> wrote:
> (2014/02/27 13:56), Sandeepa Prabhu wrote:
>> On 27 February 2014 07:44, Masami Hiramatsu
>> <masami.hiramatsu.pt@hitachi.com> wrote:
>>> (2014/02/27 0:02), Sandeepa Prabhu wrote:
>>>> Hi all,
>>>>
>>>> We have uploaded arm64 kprobes latest changes on linaro git which
>>>> include some fixes for recursive kprobes. These are rebased on top of
>>>> 3.14-rc3.  Kprobes support can be enabled by enabling
>>>> "General setup  --->  [*] Kprobes" config option.
>>>>
>>>> Git location:  git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git
>>>>  Branch: arm64_kprobes_3.14-rc3
>>>
>>> Great! Thank you for working on that!
>>>
>>>>
>>>> Please let us know if you are interested in pulling this branch for
>>>> systemtap test-suite verification, while we are trying to include
>>>> systemtap test-suite on our linaro open-embedded platforms.
>>>>
>>>> Also, please let us know if there are documentation on howto run full
>>>> test-suite for systemtap and various criteria  kprobes should meet?
>>>
>>> One possible testing is putting kprobes on all functions in the
>>> kernel by using ftrace interface. (On x86, current kprobes still has
>>> some problems, and I solved, will send the series later)
>> OK we will try to verify this with arm64 ftrace patches (from linaro)
>> which are close to get accepted upstream.
>> Has your earlier patches (kprobes blacklist related) been queued-up
>> for upstream?
>
> Yes, I updated and will post the series soon. :)
>
>> Are you planning generic kprobes API changes ?
>
> No, actually generic API is still same. But internal API will
> be added.
Ok thanks, this mean we have to update arch-specific code after your
series gets mainlined, we can help testing your patches on arm64,
guess your major verification platform is x86 :)
Cheers,
~Sandeepa
>
> Thank you,
>
> --
> Masami HIRAMATSU
> IT Management Research Dept. Linux Technology Center
> Hitachi, Ltd., Yokohama Research Laboratory
> E-mail: masami.hiramatsu.pt@hitachi.com
>
>

Re: arm64 kprobes patches

[Frank Ch. Eigler]

Hi -

On Thu, Feb 27, 2014 at 10:12:26AM +0530, Sandeepa Prabhu wrote:
> [...]
> Even on arm64, there are kernel functions that fails for kprobes,
> (example: memcpy, memset etc) that are referenced by kprobe handler
> themselves.

That is normal & fine, if "fails for kprobes" means only that the
kprobes refuse to be registered.  A failure that brings down the
kernel is much worse.

- FChE