Re: [RFC] zstd as a compression algorithm for LTO

[Jeff Law <law@redhat.com>]

On 6/19/19 3:21 AM, Martin Liška wrote:
> Hi.
> 
> I've written a patch draft that replaces zlib with the zstd compression algorithm ([1])
> in LTO. I'm also sending statistics that are collected for couple of quite big C++ source
> files. Observation I did:
> 
> - LTO stream compression takes 3-4% of LGEN compile time
> - zstd in default compression level (3) generated slighly smaller LTO elf files
> - zstd compression is 4-8x faster
> - decompression is quite negligible, but for a bigger project (godot) I can
>   reduction from 1.37 to 0.53 seconds
> - ZSTD API is much simpler to use
> 
> Suggestion based on the observation:
> - I would suggest to make zstd optional (--enable-zstd) and one would
>   use #include <zstd> + -lzstd
> - I like the default level as we want to mainly speed up LTO compilation
> - we can provide an option to control algorithm (-flto-compression-algorithm),
>   similarly to -flto-compression-level
> - we can discuss possible compression of LTO bytecode that is distributed between WPA
>   stage and individual LTRANS phases.
Presumably the reason we're not being more aggressive about switching is
the build/run time dependency on zstd?  I wonder if we could default to
zstd and fallback to zlib when zstd isn't available?

jeff