From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 51097 invoked by alias); 19 Jun 2019 19:26:00 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 50808 invoked by uid 89); 19 Jun 2019 19:25:59 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-2.0 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.1 spammy=H*f:sk:1BBDAED, H*i:sk:1BBDAED, H*F:U*pinskia X-HELO: mail-wm1-f41.google.com Received: from mail-wm1-f41.google.com (HELO mail-wm1-f41.google.com) (209.85.128.41) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 19 Jun 2019 19:25:58 +0000 Received: by mail-wm1-f41.google.com with SMTP id s3so645127wms.2 for ; Wed, 19 Jun 2019 12:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TkqIVjW2quOuwe5hYG56/D2IWk20n//jM0zhPJhfcvA=; b=mi/8fIb7459In6eCVsNDjbaCFIJN8oWTpdD2WM1qxwVtwU1BKp77HDCsZVQtzbKsFP vNo4ZbXzSlLhFFfAEKOrIyLs+o9rGO4FvggjLTdHRwt7AQEMW2ordhfh1vztfd6wIULl bR+PE+WU3RRBBZMjjBwX1b+jcUMjklWKBk+tU7Pj1fl6ms5TC+5jyvL+XhUI2IdWW1hH r+FWte0RdTKZWGWhhdTmdM8nujPfcSLWKqXzaZr9itjUYQIAaaMeuupnTq6QNQWGBKZ1 zsfrCZw6R+JPK9mAfYvCmt+OP0E6N18oxo8bWzhUL3ljE2A91IdZZDIZPR+RAYRb/6Ff fNXg== MIME-Version: 1.0 References: <89e98dc0-e766-ffef-da0f-263c3b284e96@suse.cz> <1BBDAEDD-9432-4B12-BA20-63A6E047FDB6@gmail.com> In-Reply-To: <1BBDAEDD-9432-4B12-BA20-63A6E047FDB6@gmail.com> From: Andrew Pinski Date: Wed, 19 Jun 2019 19:26:00 -0000 Message-ID: Subject: Re: [RFC] zstd as a compression algorithm for LTO To: Richard Biener Cc: Jeff Law , =?UTF-8?Q?Martin_Li=C5=A1ka?= , GCC Development , Jan Hubicka Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-IsSubscribed: yes X-SW-Source: 2019-06/txt/msg00200.txt.bz2 On Wed, Jun 19, 2019 at 11:55 AM Richard Biener wrote: > > On June 19, 2019 6:03:21 PM GMT+02:00, Jeff Law wrote: > >On 6/19/19 3:21 AM, Martin Li=C5=A1ka wrote: > >> Hi. > >> > >> I've written a patch draft that replaces zlib with the zstd > >compression algorithm ([1]) > >> in LTO. I'm also sending statistics that are collected for couple of > >quite big C++ source > >> files. Observation I did: > >> > >> - LTO stream compression takes 3-4% of LGEN compile time > >> - zstd in default compression level (3) generated slighly smaller LTO > >elf files > >> - zstd compression is 4-8x faster > >> - decompression is quite negligible, but for a bigger project (godot) > >I can > >> reduction from 1.37 to 0.53 seconds > >> - ZSTD API is much simpler to use > >> > >> Suggestion based on the observation: > >> - I would suggest to make zstd optional (--enable-zstd) and one would > >> use #include + -lzstd > >> - I like the default level as we want to mainly speed up LTO > >compilation > >> - we can provide an option to control algorithm > >(-flto-compression-algorithm), > >> similarly to -flto-compression-level > >> - we can discuss possible compression of LTO bytecode that is > >distributed between WPA > >> stage and individual LTRANS phases. > >Presumably the reason we're not being more aggressive about switching > >is > >the build/run time dependency on zstd? I wonder if we could default to > >zstd and fallback to zlib when zstd isn't available? > > Is zstd too big to include into the repository? But yes, we can properly = encode the compression format in the LTO section header and use dlopen to '= find' the default to use. At least allow it to be built as part of the normal build like GMP, etc. are done. And include it in downloading using contrib/download_prerequisites like the libraries are done. Thanks, Andrew Pinski > > Richard. > > >jeff >