From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 28044 invoked by alias); 19 Jun 2019 18:55:26 -0000 Mailing-List: contact gcc-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Archive: List-Post: List-Help: Sender: gcc-owner@gcc.gnu.org Received: (qmail 28036 invoked by uid 89); 19 Jun 2019 18:55:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-14.4 required=5.0 tests=AWL,BAYES_00,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.1 spammy=H*f:sk:ecf95d2, H*i:sk:ecf95d2 X-HELO: mail-wm1-f54.google.com Received: from mail-wm1-f54.google.com (HELO mail-wm1-f54.google.com) (209.85.128.54) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 19 Jun 2019 18:55:24 +0000 Received: by mail-wm1-f54.google.com with SMTP id w9so5258696wmd.1 for ; Wed, 19 Jun 2019 11:55:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:user-agent:in-reply-to:references:mime-version :content-transfer-encoding:subject:to:cc:from:message-id; bh=J1zZIHIT0/VbVtJ7GyKbUPKYYiD5sCOsy2JoMeSQ4MQ=; b=lCZcNOyaKvRTTJLxgGN6WCJOLhHtbTuSmLv/TvMiZLVL30qgXRrEoYEmw8KtH9bN03 Nqlyiyr9JMtzKE0LxM11fKO4mt+X2zaRXOzFgiRrV7A4BOfrVgSZ9lkQAhg4VswfpCGs dwrYUvPPpkrzPa9VfXuXQPdGUOu0AxpE18eJmgiqvStrcxf7Ay1RU6CoQGSZz0ubswHN KRUWlEq279O21jjqG9PR3//GCpJ4zQyOx/QxvWO2J0PztEASYyOozRboOfNfIqloszLK 0Pn6N9sr5iBkwl9y244hpkmsLbJBL+e4PkhDihXnLn1nJkCoW+YoeBckTTO+D/EyNZQL 5T8g== Return-Path: Received: from [192.168.178.32] (x5f757922.dyn.telefonica.de. [95.117.121.34]) by smtp.gmail.com with ESMTPSA id p26sm16557959wrp.58.2019.06.19.11.55.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Jun 2019 11:55:21 -0700 (PDT) Date: Wed, 19 Jun 2019 18:55:00 -0000 User-Agent: K-9 Mail for Android In-Reply-To: References: <89e98dc0-e766-ffef-da0f-263c3b284e96@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [RFC] zstd as a compression algorithm for LTO To: gcc@gcc.gnu.org,Jeff Law ,=?UTF-8?Q?Martin_Li=C5=A1ka?= ,GCC Development CC: Jan Hubicka From: Richard Biener Message-ID: <1BBDAEDD-9432-4B12-BA20-63A6E047FDB6@gmail.com> X-IsSubscribed: yes X-SW-Source: 2019-06/txt/msg00196.txt.bz2 On June 19, 2019 6:03:21 PM GMT+02:00, Jeff Law wrote: >On 6/19/19 3:21 AM, Martin Li=C5=A1ka wrote: >> Hi. >>=20 >> I've written a patch draft that replaces zlib with the zstd >compression algorithm ([1]) >> in LTO. I'm also sending statistics that are collected for couple of >quite big C++ source >> files. Observation I did: >>=20 >> - LTO stream compression takes 3-4% of LGEN compile time >> - zstd in default compression level (3) generated slighly smaller LTO >elf files >> - zstd compression is 4-8x faster >> - decompression is quite negligible, but for a bigger project (godot) >I can >> reduction from 1.37 to 0.53 seconds >> - ZSTD API is much simpler to use >>=20 >> Suggestion based on the observation: >> - I would suggest to make zstd optional (--enable-zstd) and one would >> use #include + -lzstd >> - I like the default level as we want to mainly speed up LTO >compilation >> - we can provide an option to control algorithm >(-flto-compression-algorithm), >> similarly to -flto-compression-level >> - we can discuss possible compression of LTO bytecode that is >distributed between WPA >> stage and individual LTRANS phases. >Presumably the reason we're not being more aggressive about switching >is >the build/run time dependency on zstd? I wonder if we could default to >zstd and fallback to zlib when zstd isn't available? Is zstd too big to include into the repository? But yes, we can properly en= code the compression format in the LTO section header and use dlopen to 'fi= nd' the default to use.=20 Richard.=20 >jeff