get backtrace of KVM VM from host

get backtrace of KVM VM from host

[Justin Cinkelj]

(crossposting from KVM maillist)

Hi

Is it possible to get stack backtrace into KVM VM from the host side? So 
if I run './stack -p PID' (stack from elfutilfs 
https://sourceware.org/elfutils/), I get backtrace of some process. I 
would like to do the same for VM. I can assume VM will run only a kernel 
(a unikernel, like OSv or IncludeOS), so most/all debug symbols will be 
there in a single file, and at least IncludeOS doesnt load any code 
beside its own kernel.

I did notice KVM_GET_REGS and KVM_SET_MEMORY_REGION, and at least for 
trivial examples (like https://github.com/dpw/kvm-hello-world) this 
provides enough information to track which code was loaded into VM, 
observe current stack content and registers. I can only guess much more 
work is required to get similar result with qemu-kvm. Hence I'm asking 
if this is already implemented.

Thank you,
Justin

Re: get backtrace of KVM VM from host

[Mark Wielaard]

Hi,

On Mon, 2018-05-21 at 10:26 +0200, Justin Cinkelj wrote:
> Is it possible to get stack backtrace into KVM VM from the host side?
> So 
> if I run './stack -p PID' (stack from elfutilfs 
> https://sourceware.org/elfutils/), I get backtrace of some process. I 
> would like to do the same for VM. I can assume VM will run only a kernel 
> (a unikernel, like OSv or IncludeOS), so most/all debug symbols will be 
> there in a single file, and at least IncludeOS doesnt load any code 
> beside its own kernel.
> 
> I did notice KVM_GET_REGS and KVM_SET_MEMORY_REGION, and at least for 
> > trivial examples (like https://github.com/dpw/kvm-hello-world) this 
> provides enough information to track which code was loaded into VM, 
> observe current stack content and registers. I can only guess much more 
> work is required to get similar result with qemu-kvm. Hence I'm asking 
> if this is already implemented.

Providing the registers and memory view inside the KVM VM would be the
first step. elfutils would also need to know the memory/ELF process
layout. For a normal process that would come from e.g. /proc/pid/maps.
Using such a layout eu-stack would then be able to find the unwind
tables and symbols associated with a particular address.

I believe qemu already has an gdb stub that gdb can use to get at the
registers, memory and process layout. Maybe you could adapt that
provide the information needed.

Cheers,

Mark

Re: get backtrace of KVM VM from host

[Justin Cinkelj]

Something like that was suggested at KVM devel list too. I was able to 
get an useful backtrace for a trivial VM (a single ELF file, VM code 
runs directly from (virtual) physical memory). Well, that was more to 
learn a bit about elfutils than anything else. A more realistic VM will 
be more difficult, I guess.

Justin

On 05/22/2018 04:04 PM, Mark Wielaard wrote:
> Hi,
>
> On Mon, 2018-05-21 at 10:26 +0200, Justin Cinkelj wrote:
>> Is it possible to get stack backtrace into KVM VM from the host side?
>> So
>> if I run './stack -p PID' (stack from elfutilfs
>> https://sourceware.org/elfutils/), I get backtrace of some process. I
>> would like to do the same for VM. I can assume VM will run only a kernel
>> (a unikernel, like OSv or IncludeOS), so most/all debug symbols will be
>> there in a single file, and at least IncludeOS doesnt load any code
>> beside its own kernel.
>>
>> I did notice KVM_GET_REGS and KVM_SET_MEMORY_REGION, and at least for
>>> trivial examples (like https://github.com/dpw/kvm-hello-world) this
>> provides enough information to track which code was loaded into VM,
>> observe current stack content and registers. I can only guess much more
>> work is required to get similar result with qemu-kvm. Hence I'm asking
>> if this is already implemented.
> Providing the registers and memory view inside the KVM VM would be the
> first step. elfutils would also need to know the memory/ELF process
> layout. For a normal process that would come from e.g. /proc/pid/maps.
> Using such a layout eu-stack would then be able to find the unwind
> tables and symbols associated with a particular address.
>
> I believe qemu already has an gdb stub that gdb can use to get at the
> registers, memory and process layout. Maybe you could adapt that
> provide the information needed.
>
> Cheers,
>
> Mark