From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 11537 invoked by alias); 3 Jun 2015 14:29:16 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Received: (qmail 11528 invoked by uid 89); 3 Jun 2015 14:29:16 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=0.9 required=5.0 tests=BAYES_50,KAM_ASCII_DIVIDERS,RCVD_IN_DNSWL_LOW,SPF_PASS autolearn=no version=3.3.2 X-HELO: mail-wg0-f54.google.com Received: from mail-wg0-f54.google.com (HELO mail-wg0-f54.google.com) (74.125.82.54) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-GCM-SHA256 encrypted) ESMTPS; Wed, 03 Jun 2015 14:29:06 +0000 Received: by wgbgq6 with SMTP id gq6so10793899wgb.3 for ; Wed, 03 Jun 2015 07:29:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-type:content-disposition:content-transfer-encoding :user-agent; bh=keCqxWFFVODQ/zvGiVzDM+p0WRgr09gathKet5VCqYk=; b=eIpf77P/ym8FfcQrg3ZU7I/sf5CQaY0x+DgZzWOamXpclmyOBqoba1i+mAUb5jACU0 kFS6XdMg71eOtZoceSrBelymfCNVP4cdcbLEUdvLwBsTpxMwboOuFP9Gzd3liI6Z+ARc O/mutM0NNGBd8WfRr0ubHgEuxguhamjL86DiTR9+mDDaKQWKS3hXm5chhV7E4c1RwNti uE826uN7FOt60q5YVoS4U7Yxv83fWcGJNw7DPAE73+3KMp587/gfPn7DLDzVFhqLUTWN u4+eNqhSsqzpfG31IXVZC0T2//zIrC+IypWOagsbtreguMDbX4ahEvtijnG7z40TS053 d4XQ== X-Gm-Message-State: ALoCoQkI266lg4orDwWYhsOtz18P8fyjtEo7et41ssHpVX+0FnK6qCda9tjtm5V+e8SYkLe3A+7F X-Received: by 10.181.25.234 with SMTP id it10mr13739436wid.0.1433341742705; Wed, 03 Jun 2015 07:29:02 -0700 (PDT) Received: from griffinp-ThinkPad-X1-Carbon-2nd (cpc14-aztw22-2-0-cust189.18-1.cable.virginm.net. [82.45.1.190]) by mx.google.com with ESMTPSA id ej5sm1345499wjd.22.2015.06.03.07.29.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 03 Jun 2015 07:29:01 -0700 (PDT) Date: Wed, 03 Jun 2015 14:29:00 -0000 From: Peter Griffin To: gdb-patches@sourceware.org Cc: lee.jones@linaro.org Subject: RFC GDB Linux Awareness analysis Message-ID: <20150603142858.GA19370@griffinp-ThinkPad-X1-Carbon-2nd> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.21 (2010-09-15) X-SW-Source: 2015-06/txt/msg00040.txt.bz2 Hi GDB community, Overview ======== The purpose of this email is to describe a useful feature which has been developed by STMicroelectronics inside of GDB for debugging Linux kernels. I will cover at a high level some of the implementation details and what I see as the advantages / disadvantages of the implementation. I will also cover some other alternative approaches that I'm aware of. The purpose is to facilitate discussion with the GDB experts on this mailing list as to what the "correct" way to implement this functionality would be. The end goal is to have an upstream implementation of this functionality. Introduction ============ STMicroelectronics has a patchset on top of vanilla GDB which adds much better Linux kernel awareness into GDB. They have called this GDB extension LKD (Linux Kernel Debugger). This GDB extension is primarily used in conjunction with ST's JTAG debugger for debugging ARM / SH4 SoCs, and is implemented as an internal GDB extension, written in C. The LKD extension is nicely abstracted from the underlying JTAG interface, and I have used it to debug a ARMv7 kernel running in QEMU via gdbremote. ST would like to contribute these patches back to GDB, as we think it could be useful not only for ARM Linux debugging, but also other CPU/OS combinations in the future. LKD can be broadly split into the following parts: - 1) Linux task awareness 2) Loadable module support 3) Various Linux helper commands The next section looks at each of the three parts, and any other implementations I'm aware of that currently exist. 1) Task Awareness ================= 1.1) LKD Linux Task awareness ============================= When using mainline GDB for debugging a Linux kernel via JTAG GDB will typically only show the actual hardware threads. The LKD task awareness extension (lkd-process.c) adds the ability for GDB to parse some kernel data structures, so it can build a thread list of all kernel threads in the running Linux kernel. When halting the processor (via a breakpoint, ctrl+c etc) the thread list is re-enumerated, so new tasks are visible to GDB and tasks which have exited are removed from the thread list. To achieve this GDB has to know about various Linux kernel data structures, and also various fields within these structures (mainly task_struct, thread_info, mm_struct). Code has been implemented to parse these structs, and also do virtual to physical address translations and cache handling. Various frame unwinders are also implemented to stop backtracing on various exceptions, and entry points to the kernel (these would be a useful addition regardless of which task awareness approach is taken). Advantages - Adds Linux kernel task awareness to GDB - Supports symbolic debugging of user processes - Contextual information (structs / field offsets) are readily available inside of GDB - Has been used and well tested inside ST for some time Disadvantages - Being implemented in C within GDB creates a dependency between GDB and the Linux kernel - Mainly tested on 3.10 and 2.6.30 with ARM and SH4 kernels, being upstreamed would expose it to many differing kernel versions and architectures. - I can't see any other "OS awareness" support currently in the GDB code base 1.2) OpenOCD / GDB Linux task awareness ======================================= Whilst looking for any prior art in this area I found that OpenOCD already implements some basic Linux task awareness. See here: - http://openocd.org/doc/doxygen/html/linux_8c_source.html I've this working to the extent where I can connect via JTAG to a ARMv7 U8500 Snowball board and enumerate a thread list in GDB. It required some debugging & hacking to get this far. This implementation passes the thread list to GDB via the gdbremote protocol and as such changes required in GDB are minimal. Advantages - OpenOCD already supports many target types (ARM / MIPS), and has support for virt to phys translations / cache handling etc. - OpenOCD also implements task awareness for other RTOS’s (ThreadX / FreeRTOS / eCos) - Using gdbremote means GDB changes are (so far) minimal. - OpenOCD / Linux Kernel dependency already exists Disadvantages - Creates a dependency between Linux kernel data structures and OpenOCD. - I believe finding field offsets within structs is currently not possible via gdbremote protocol. Currently OpenOCD generates these offsets at compile time which is ugly and needs fixing. See here http://openocd.org/doc-release/doxygen/linux__header_8h_source.html. Being able to find field offsets would in my opinion be a useful addition to the gdbremote protocol which would allow OpenOCD task awareness to work much better at runtime. - Doesn't support debugging user processes. I think this would still require some GDB changes, and also gdbremote protocol changes to get working correctly. - Needs to be made more generic 1.3) Python Task awareness ========================== Jan Kiszka from Siemens has implemented some basic Linux kernel task awareness using the GDB Python interface. See here https://lwn.net/Articles/631167/ and here https://github.com/torvalds/linux/blob/master/Documentation/gdb-kernel-debugging.txt This support is currently limited to the following commands: - lx_current -- Return current task lx_per_cpu -- Return per-cpu variable lx_task_by_pid -- Find Linux task by PID and return the task_struct variable lx_thread_info -- Calculate Linux thread_info from task variable However this could be extended to build up a thread list. The GDB Python interface would I believe need to be extended to allow a thread list to be passed back to GDB via Python. Advantages - Code parsing Linux kernel structures lives in the kernel source tree - Contextual information is easily available - Works with all OCD implementations not just OpenOCD Disadvantages - Doesn't exist yet - Python / GDB interface would need to be extended to support threads Questions ======== 1) What do GDB community think about having Linux OS task awareness in the GDB codebase? 2) Is there a preferred way to implement this (C extension / gdbremote / Python / something else)? 3) Any other implementations / advantages / disadvantages that I'm not currently aware of? 2) Loadable module support ========================== 2.1) LKD Loadable module support ================================ lkd-modules.c adds better Linux kernel module symbolic symbol support to GDB using the GDB shared libraries infrastructure (solib). It has hooks to enable the debugging of module __init sections to reflect that these pages get discarded post-load. It also implements the same section layout algorithm as the kernel to speed up symbol resolution, only inspecting the target’s memory if there is a mismatch (inspecting target memory can be slow). I think this part could be upstreamed separately to the task awareness support, although I’ve not tried separating it yet from the other LKD patches. Advantages - Allows full symbolic debugging of Linux loadable modules including init sections - Would be independent of underlying communication mechanism (JTAG / gdbremote etc) Disadvantages - Some dependency between GDB and Linux kernel is still present Questions: Are GDB community happy to have a Linux specific solib functionality in the GDB code base? 2.2) Python Loadable module support =================================== I've not managed to look to much at this, but some basic support exists, see here https://github.com/torvalds/linux/blob/master/Documentation/gdb-kernel-debugging.txt Having read the LKD modules implementation I don’t think this can be anywhere near as functionally complete. 3) Linux Helper commands ======================== 3.1) LKD helper commands ======================== LKD implements various Linux helper commands inside GDB such as: - - dmesg - dump dmesg log buffer from kernel - process_info - prints various info about current process - pmap - prints memory map of current process - vm_translate - translates virtual to physical address - proc_interrupts - prints interrupt statistics - proc_iomem - prints I/O mem map - proc_cmdline - prints the contents of /proc/cmdline - proc_version - prints the contents of /proc/version - proc_mounts - print the contents of /proc/mounts - proc_meminfo - print the contents of /proc/meminfo. Advantages - Can be used by all GDB based debug solutions - Some precedent of OS related commands in the GDB code base https://sourceware.org/gdb/current/onlinedocs/gdb/OS-Information.html#OS-Information Disadvantages - Creates a GDB dependency with the kernel Python helper commands ====================== Jan’s python scripts also implement some of the same LKD commands such as 'dmesg'. See here https://lwn.net/Articles/631167/ and https://github.com/torvalds/linux/blob/master/Documentation/gdb-kernel-debugging.txt Advantages - Code that is parsing Linux kernel structures lives in the kernel source tree - Can be used by all GDB based debug solutions Questions - Do GDB community mind Linux specific custom commands being added to GDB code base? My current opinion is that helper commands which can be, should be migrated from C code into Python, and merged into the kernel source tree (and then retired from the LKD patchset). If you got here, thanks for reading this far! Like I said at the beginning, the purpose of this email is to stimulate some discussion on what you folks consider the 'correct' way to implement this OS awareness functionality is. All feedback is welcome. Kind regards, Peter.