From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 35529 invoked by alias); 21 Jun 2019 12:57:22 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Received: (qmail 35512 invoked by uid 89); 21 Jun 2019 12:57:22 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-5.2 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.3.1 spammy=H*i:sk:bd3013b, H*f:sk:bd3013b X-HELO: nikam.ms.mff.cuni.cz Received: from nikam.ms.mff.cuni.cz (HELO nikam.ms.mff.cuni.cz) (195.113.20.16) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 21 Jun 2019 12:57:20 +0000 Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id 3605E2825CA; Fri, 21 Jun 2019 14:57:18 +0200 (CEST) Date: Fri, 21 Jun 2019 12:57:00 -0000 From: Jan Hubicka To: Martin =?iso-8859-2?Q?Li=B9ka?= Cc: Richard Biener , Thomas Koenig , Andrew Pinski , Jeff Law , GCC Development , GCC Patches Subject: Re: [PATCH] Add .gnu.lto_.meta section. Message-ID: <20190621125718.qv2m5iygqj2b43fe@kam.mff.cuni.cz> References: <20190619192954.edwdfxns3gx2gt5m@kam.mff.cuni.cz> <122f53d2-5ae0-b801-81ab-36ce69f9efda@netcologne.de> <9417a329-c71c-4682-872c-60ff3524c47e@suse.cz> <20190620120225.s7ouywid2mntmhlm@kam.mff.cuni.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: NeoMutt/20170113 (1.7.2) X-SW-Source: 2019-06/txt/msg01301.txt.bz2 > On 6/21/19 2:34 PM, Richard Biener wrote: > > On Fri, Jun 21, 2019 at 12:20 PM Martin Liška wrote: > >> > >> Hi. > >> > >> The patch is about a new ELF section that will contain information > >> about LTO version. And for the future, used compression will be stored > >> here. The patch removes streaming of the version into each section. > > > > I'd like each section to have a header containing a compression method > > (compressed or not, and now zlib vs. zstd). We currently have a mix > > and knowledge is hidden. > > That would be possible, good idea. > > > > > I also remember I had old patches to make the data streamer compress > > the stream on-the-fly, not requiring the full uncompressed stream in > > memory and then re-access it for compression (needing more temporary > > memory). But IIRC the speedup was marginal. It may be more interesting as memory optimization now we do parallel WPA streaming BTW. > > > > My thought is also that we should have exactly _one_ ELF section > > for the LTO bytecode and our own container inside (much like > > simple-object does for non-ELF). So having another one is, well, ugly ;) > > Having N sections for all symbols (functions and variables) is handy because > we read some of the during WPA (by IPA ICF). So having all in a single section > would make decompression more complicated. There is nothing preventing us from implementing our own subsections in that section where each is compressed independently and can be copied to output file and still save size of headers needed for that. For example all sections are keyed by the lto section enum + possibly a decl that can be both streamed as integers rather than quite lenghtly strings. (the decl to name is streamed into the global decl stream and available at all time). I guess one can go for one global LTO major/minor in the section header and then section type/is compressed/size + optional decl ID for each subsection reducing headers to about 1+8+optional 8 bytes per header. This can be all placed at the begining of the file followed by RAW data of individual sections. I bet this is a lot smaller than ELF overhead :) > > I'm going to prepare a patch that will pull out a LTO section header > from compressed stream. This looks like good step (and please stream it in host independent way). I suppose all these issues can be done one-by-one. Honza > > Martin > > > > >> Disadvantage is a format change that will lead to following errors when > >> LTO bytecode is used from a different version: > >> > >> $ gcc x.o > >> lto1: fatal error: bytecode stream in file ‘x.o’ generated with GCC compiler older than 10.0 > >> > >> $ gcc-9 main.o > >> lto1: fatal error: bytecode stream in file ‘main.o’ generated with LTO version 850.0 instead of the expected 8.0 > >> > >> I've been testing the patch. > >> Thoughts? > >> Martin >