New ch_type value ELFCOMPRESS_ZSTD?

New ch_type value ELFCOMPRESS_ZSTD?

[Fangrui Song]

I created https://groups.google.com/g/generic-abi/c/satyPkuMisk ("Add new
ch_type value: ELFCOMPRESS_ZSTD") after I saw that on LLVM side, Cole Kissane
proposes that we add Zstandard as new compression method (mainly for .debug*
sections, but also for some LLVM internal things)
https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399

This makes a lot of sense to me as I know compression/uncompression takes a lot
of compile/link time.  If people agree that this is a good direction, I wish
that we can have a generic-abi value: ELFCOMPRESS_ZSTD 2.  If that does not
work, perhaps we need a GNU ABI value.

We will need features from a bunch of tools:

* GCC/Clang: perhaps we can introduce a new driver option -gz=zstd
* GNU assembler: add --compress-debug-sections=zstd
* GNU ld: add --compress-debug-sections=zstd
* objcopy/llvm-objcopy: add --compress-debug-sections=zstd
* gdb: support Zstandard
* elfutils: support Zstandard

Re: New ch_type value ELFCOMPRESS_ZSTD?

[Nick Alcock]

On 27 Jun 2022, Fangrui Song stated:

> I created https://groups.google.com/g/generic-abi/c/satyPkuMisk ("Add new
> ch_type value: ELFCOMPRESS_ZSTD") after I saw that on LLVM side, Cole Kissane
> proposes that we add Zstandard as new compression method (mainly for .debug*
> sections, but also for some LLVM internal things)
> https://discourse.llvm.org/t/rfc-zstandard-as-a-second-compression-method-to-llvm/63399

The next CTF format will be compressed with zstd too (if available),
fwiw. It's pretty obviously the right thing to go with these days for
anything but specialized use cases. So (though my opinion counts for
nothing) I think this is an excellent direction to go in.

(Of course, if zstd isn't folded into the binutils source tree like zlib
is, we have the interesting problem of what to do if it's not available
at build time. If zstd compression is optional, and not available in the
existing binutils, and gdb or objdump or something needs to read a
zstd-compressed debug section, what do we do? Error out? Whatever it is
we end up doing, I'll make libctf do the same thing in the same
situation.

Pointless detail: this will hardly ever impair use of things like ld
that must keep working because ld inputs are usually compiler output,
and the CTF in there is entirely uncompressed and undeduplicated. Only
using ld -r outputs as inputs to ld would be affected, and that's a rare
use case that is usually done as part of the same build process on the
same machine, and that use case would *also* keep working whether zstd
was available or not. The only case that might be impacted is users like
GDB being asked to read zstd-compressed sections when GDB was not
compiled with zstd support, and that's the same problem we'd have if the
same binary contained DWARF debuginfo and the same GDB were asked to
read it.)

-- 
NULL && (void)