From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 59624 invoked by alias); 6 May 2019 16:40:26 -0000 Mailing-List: contact binutils-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: binutils-owner@sourceware.org Received: (qmail 59616 invoked by uid 89); 6 May 2019 16:40:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-9.6 required=5.0 tests=AWL,BAYES_00,SPF_HELO_PASS,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 spammy=GAS, chase, continuous, HTo:U*nickc X-HELO: aserp2130.oracle.com Received: from aserp2130.oracle.com (HELO aserp2130.oracle.com) (141.146.126.79) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 06 May 2019 16:40:23 +0000 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x46GdHZS185653; Mon, 6 May 2019 16:40:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : references : date : in-reply-to : message-id : mime-version : content-type; s=corp-2018-07-02; bh=TsRYu+nsafSdjML7PsucY7r9b4u7BfV3XhWfvUdPIQI=; b=dLjBc0TQ9lw1NZcqHvTwc2bTXPP0JmzsccM62tuX31rDT/gvF7HHAiSU+l1xSppVhl2o 47NIMsKVLcjX0nuXjjLE0KNqSjqCBMZhA9C1Q2lqmjDRRiX7p7P5dfE6UZjlXDh8GiKk YeNa/wnuIQsDJOevYseksHRaIFehYPIORHEKLjjiuC8bKT7Dsh98AeMhZYjOn41w/u8P MUTqTx6QndT2AHiRtO2mEb8XD/Ns/rtOrV/H5QdUYgHH+fkGaJasToMgz+MlH/LKImsQ DXN27CFnvo0XL7dcNJKP6L8xQU5mrJY3cEvuzylRN4ESDfaxxEqdZqUHBQvAY8396vCX QA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 2s94b5qtvu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 06 May 2019 16:40:10 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x46GdGmr057127; Mon, 6 May 2019 16:40:09 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 2s94af0g2t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 06 May 2019 16:40:09 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x46Ge8FK029985; Mon, 6 May 2019 16:40:08 GMT Received: from loom (/81.187.191.129) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 06 May 2019 09:40:07 -0700 From: Nick Alcock To: Nick Clifton Cc: Joseph Myers , binutils@sourceware.org Subject: Re: [PATCH 00/19] libctf, and CTF support for objdump and readelf References: <20190430225706.159422-1-nick.alcock@oracle.com> <534c2613-da1e-7b53-338c-2988b44b1b6b@redhat.com> Date: Mon, 06 May 2019 16:40:00 -0000 In-Reply-To: <534c2613-da1e-7b53-338c-2988b44b1b6b@redhat.com> (Nick Clifton's message of "Fri, 3 May 2019 13:33:10 +0100") Message-ID: <878svjtu3e.fsf@esperi.org.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-IsSubscribed: yes X-SW-Source: 2019-05/txt/msg00142.txt.bz2 On 3 May 2019, Nick Clifton verbalised: > Hi Nick, > > OK, so I am going to stop my review of this patch series until we have > some answers to a few high level questions/requirements. Specifically: > > * Hosting > > Is the binutils project the right place for this code ? As Joseph > has already mentioned libctf appears to use code from the elfutils > project, so maybe that would be a better home for libctf ? It uses no code from elfutils: It uses one datatype from elfutils, and shouldn't really even use that (it should use the stuff from from glibc). (Thanks for spotting that! It'll be fixed this week.) > * Testing > > I did not see a testsuite for the library, nor any additions to > the binutils testsuite for the extensions to objdump and readelf. > I would not be comfortable adding such a large body of code to the > project without some Agreed!!! Writing a testsuite is high on our priority list, but we need compiler and linker support first. There will definitely be a testsuite in place, probably in large part autogenerated. (The code is not at all untested: the testing largely consists of generating CTF for the entire Linux kernel and then letting the fairly sizeable DTrace testsuite run over it. However, this is obviously not suitable for upstreaming into binutils: something more systematic is needed, and will exist.) > * Documentation > > It would be really good to have the CTF format documented somewhere > (semi) permanent and publicly accessible. It would also be good if > there was a libctf.texi document describing how consumers are expected > to use the library, and, ideally, providing code examples. Agreed! > * Usefulness > > This may be a bit contentious - but is the CTF format actually being > used anywhere, or likely to be used in the near future ? If this is > a project that is just going to be used by a small group, or even just > a single company, then I am worried that the code will just bit rot > away and not be actively maintained. It would be useful for many projects, but it was not easy to adopt it, and it has been relatively unknown outside of Solaris and FreeBSD circles. We have been using it for many years with the Linux kernel and DTrace. Other projects may adopt it. It is *useful* for many other projects, but it was quite difficult to generate CTF-format data (from DWARF using a variety of painful-to-port Solaris tools). I know I wanted it in the past for various other profilers I was writing, but I didn't use it because I had no idea it existed. It seems likely to be *useful* for debuggers and tracers and introspectors of all sorts: anything that wants to know what type an ELF object is, or what type function arguments or return types are, and who wants to be able to decompose that type into its constitutent pieces and chase pointers from it and make sense of the results seems likely to be able to make use of CTF. In particular, GDB, perf, and sysprof could definitely make use of it, as could systemtap, rr, and honestly I'm only limited here by not knowing the names of more obscure debugger projects. Right now these either cannot do useful things with datatypes at all, or require all the DWARF debuginfo to be present to do anything, and given that many of these things are systemwide continuous debuggers, the debuginfo is almost never present when the debugger points at a victim program and tries to print out argument info or whatever. I'm fairly confident that having type information for C programs as widely and easily available as it is, for, say, Lisp programs (with a cost, often, of only a few kilobytes) is a generally good thing. We're making a start by adding CTF support to GCC, GNU ld and GDB. Size-wise it really is pretty small. Let's try a few samples (with a notably unscientific construction method). The first five numeric columns are a count of types, and CTF sizes in bytes (the sizes of specific sections: all uncompressed sizes but the first 'size' number. "DWARF size" is the size of .debug_info.) CTF Program types size (uncompressed) stringsize typesize DWARF size coreutils ls 396 10324 (26216) 13148 13068 74241 GAS 2.30 1123 55748 (172731) 106079 66652 1001453 emacs 26.1.50 3546 104276 (284479) 142231 142248 3912261 X.org 1.20.3 7266 152196 (473797) 201421 272376 4163434 GhostScript 9.26 7873 181036 (538901) 243293 295608 7943132 Gtk 3.24.7 9236 208612 (620926) 328174 292752 6106925 So a shrinkage over DWARF of roughly 80%, and usually less than 5% of the size of the binary whose types are described. With planned format and generation improvements, I expect that a 90% shrinkage over DWARF is probably achievable without too much effort. (More than that might require pruning out individual unused bits, a fairly radical change.)