From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 12770 invoked by alias); 22 Feb 2012 21:56:47 -0000 Mailing-List: contact archer-help@sourceware.org; run by ezmlm Sender: Precedence: bulk List-Post: List-Help: List-Subscribe: List-Id: Received: (qmail 12761 invoked by uid 22791); 22 Feb 2012 21:56:46 -0000 X-SWARE-Spam-Status: No, hits=-6.9 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_HI,SPF_HELO_PASS,TW_BJ,T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org From: Tom Tromey To: Jan Kratochvil Cc: archer@sourceware.org, Jakub Jelinek Subject: Re: Inter-CU DWARF size optimizations and gcc -flto References: <20120201132307.GA32578@host2.jankratochvil.net> Date: Wed, 22 Feb 2012 21:56:00 -0000 In-Reply-To: <20120201132307.GA32578@host2.jankratochvil.net> (Jan Kratochvil's message of "Wed, 1 Feb 2012 14:23:09 +0100") Message-ID: <87hayio7ld.fsf@fleche.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.93 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-SW-Source: 2012-q1/txt/msg00009.txt.bz2 Jan> (b) .gdb_index will have limited scope, only to select which Jan> objfiles to expand, no longer to select which CUs to expand. I suspect we are going to need a better approach here anyway. I sometimes hear about programs with more than 800 shared libraries. If you assume separate debuginfo this means 1600 objfiles. I think this will just crush most of the existing algorithms in gdb. Jan> (c) Partial CU expansion Tom Tromey talks about is a must in such case. I realized I never wrote up how this could work. The below is sort of a sketch that devolves into random thoughts. I have been thinking about it since we discussed it and I think it has a potentially severe problem. The basic idea is simple: right now we have two DWARF readers in dwarf2read.c, the psymtab reader and the full symbol reader. Right now when we find a psymbol, we expand the whole CU to full symbols. This normally isn't too bad -- but there are some CUs out there in practice that are quite large, and the delay reading them is noticeable. So, what if we unified the two readers -- eliminating one source of bugs -- and also changed CU expansion to be DIE-based. That is, in symtab.c, before returning a symbol from a symtab, we would call some back-end function to expand the symbol. The DWARF reader would then just read the DIEs needed to instantiate that one particular symbol plus whatever dependencies (types usually) it has. Ok, that sounds good, but there is a problem: struct symbol is really big, much bigger than a psymbol. We could just read psymbol-like structs on our first pass, but we need somewhere to store the DIE offset for efficient expansion. We can solve that by updating and applying an old patch that shrinks psymbol. Then we can use the saved space to store the DIE -- so this change can be space-neutral. However, this neglects the bcache. In fact, the bcache sinks the whole project, since DIE offsets will vary by definition. Well, the DIE offset sinks this particular approach. Maybe there is another approach, not space-neutral but also not too bad, that can be used. For example, keeping the bcache but having the symtabs contain both {psymbol+DIE} pairs and fully-expanded symbols (depending on what has been expanded). If we went a bit deeper and had hierarchical symbol tables, we could skip whole DIE subtrees even in the partial reader. A related idea here that I was idly wondering about is whether we could make the psymtab reader hierarchical without touching full symbols. The deeper rewrite seems eventually necessary. The symbol table code is pretty horrible, in multiple ways. However, at least for me it hasn't yet reached the pain point where we can justify spending months and months on it, which I think is what it would take. Your thoughts welcome. Tom