From: "Martin Liška" <mliska@suse.cz>
To: "Dmitry V. Levin" <ldv@altlinux.org>
Cc: elfutils-devel@sourceware.org, Mark Wielaard <mark@klomp.org>,
Fangrui Song <maskray@google.com>
Subject: Re: [PATCH][RFC] readelf: partial support of ZSTD compression
Date: Mon, 24 Oct 2022 20:16:09 +0200 [thread overview]
Message-ID: <6c1ce1f1-2e45-20bb-e98d-6d35692addfb@suse.cz> (raw)
In-Reply-To: <20221024164806.GA21412@altlinux.org>
[-- Attachment #1: Type: text/plain, Size: 993 bytes --]
On 10/24/22 18:48, Dmitry V. Levin wrote:
> On Mon, Oct 24, 2022 at 02:17:17PM +0200, Martin Liška wrote:
>> On 10/24/22 13:41, Dmitry V. Levin wrote:
>>> On Mon, Oct 24, 2022 at 01:09:59PM +0200, Martin Liška wrote:
>>> [...]
>>>> One TODO I see is that:
>>>> +libelf_so_LDLIBS = $(libelf_so_DEPS) -lz -lzstd
>>>>
>>>> should be conditional based on HAVE_ZSTD. But I don't know how to do that?
>>>
>>> I suppose you're talking about libzstd_LIBS.
>>
>> Hm, can't see it after autoreconf -fi and ./configure.
>
> That's because you do
> PKG_CHECK_MODULES(ZSTD, [libzstd], ...)
> and this defines ZSTD_CFLAGS and ZSTD_LIBS instead of libzstd_CFLAGS
> and libzstd_LIBS because PKG_CHECK_MODULES() uses its first argument
> as the prefix for these variables.
>
>
Thank you. Apparently, I collided with the existing:
eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)])
Anyway, I'm sending V2 that works fine --with-zstd and --without-zstd as expected.
Ready for master?
Thanks,
Martin
[-- Attachment #2: 0001-readelf-partial-support-of-ZSTD-compression.patch --]
[-- Type: text/x-patch, Size: 8117 bytes --]
From 4aea412783b9b0dcaf0f887947bf2e8ee6c5368b Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 24 Oct 2022 11:53:13 +0200
Subject: [PATCH] readelf: partial support of ZSTD compression
Support decompression of ZSTD sections and add support
for it when -SWz is used:
...
[30] .debug_abbrev PROGBITS 0000000000000000 00001f9d 00000168 0 C 0 0 1
[ELF ZSTD (2) 000002fc 1]
...
ChangeLog:
* configure.ac: Add zstd_LIBS.
libelf/ChangeLog:
* Makefile.am: Use zstd_LIBS.
* elf.h (ELFCOMPRESS_ZSTD): Add new value.
* elf_compress.c (__libelf_decompress): Dispatch based
on the compression algorithm.
(__libelf_decompress_zlib): New.
(__libelf_decompress_zstd): New.
(__libelf_decompress_elf): Pass type of compression to
__libelf_decompress.
* elf_compress_gnu.c (elf_compress_gnu): Use ELFCOMPRESS_ZLIB
as .z* sections can be only compressed with ZLIB.
* libelfP.h (__libelf_decompress): Change signature.
src/ChangeLog:
* readelf.c (elf_ch_type_name): Use switch and support ZSTD.
---
configure.ac | 8 +++---
libelf/Makefile.am | 2 +-
libelf/elf.h | 3 +++
libelf/elf_compress.c | 56 ++++++++++++++++++++++++++++++++++++---
libelf/elf_compress_gnu.c | 2 +-
libelf/libelfP.h | 2 +-
src/readelf.c | 18 ++++++++-----
7 files changed, 75 insertions(+), 16 deletions(-)
diff --git a/configure.ac b/configure.ac
index 03b67a9d..803876e2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -410,6 +410,11 @@ dnl Test for bzlib and xz/lzma/zstd, gives BZLIB/LZMALIB/ZSTD .am
dnl conditional and config.h USE_BZLIB/USE_LZMALIB/USE_ZSTD #define.
save_LIBS="$LIBS"
LIBS=
+eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)])
+AS_IF([test "x$with_zstd" = xyes], [LIBZSTD="libzstd"], [LIBLZSTD=""])
+AC_SUBST([LIBZSTD])
+zstd_LIBS="$LIBS"
+AC_SUBST([zstd_LIBS])
eu_ZIPLIB(bzlib,BZLIB,bz2,BZ2_bzdopen,bzip2)
# We need this since bzip2 doesn't have a pkgconfig file.
BZ2_LIB="$LIBS"
@@ -417,9 +422,6 @@ AC_SUBST([BZ2_LIB])
eu_ZIPLIB(lzma,LZMA,lzma,lzma_auto_decoder,[LZMA (xz)])
AS_IF([test "x$with_lzma" = xyes], [LIBLZMA="liblzma"], [LIBLZMA=""])
AC_SUBST([LIBLZMA])
-eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)])
-AS_IF([test "x$with_zstd" = xyes], [LIBZSTD="libzstd"], [LIBLZSTD=""])
-AC_SUBST([LIBZSTD])
zip_LIBS="$LIBS"
LIBS="$save_LIBS"
AC_SUBST([zip_LIBS])
diff --git a/libelf/Makefile.am b/libelf/Makefile.am
index 560ed45f..24c25cf8 100644
--- a/libelf/Makefile.am
+++ b/libelf/Makefile.am
@@ -106,7 +106,7 @@ libelf_pic_a_SOURCES =
am_libelf_pic_a_OBJECTS = $(libelf_a_SOURCES:.c=.os)
libelf_so_DEPS = ../lib/libeu.a
-libelf_so_LDLIBS = $(libelf_so_DEPS) -lz
+libelf_so_LDLIBS = $(libelf_so_DEPS) -lz $(zstd_LIBS)
if USE_LOCKS
libelf_so_LDLIBS += -lpthread
endif
diff --git a/libelf/elf.h b/libelf/elf.h
index 02a1b3f5..f0f0ec7d 100644
--- a/libelf/elf.h
+++ b/libelf/elf.h
@@ -506,6 +506,9 @@ typedef struct
/* Legal values for ch_type (compression algorithm). */
#define ELFCOMPRESS_ZLIB 1 /* ZLIB/DEFLATE algorithm. */
+#define ELFCOMPRESS_ZSTD 2 /* Compressed with zstd */
+ /* (see http://www.zstandard.org). */
+
#define ELFCOMPRESS_LOOS 0x60000000 /* Start of OS-specific. */
#define ELFCOMPRESS_HIOS 0x6fffffff /* End of OS-specific. */
#define ELFCOMPRESS_LOPROC 0x70000000 /* Start of processor-specific. */
diff --git a/libelf/elf_compress.c b/libelf/elf_compress.c
index d7f53af2..62b41b20 100644
--- a/libelf/elf_compress.c
+++ b/libelf/elf_compress.c
@@ -39,6 +39,10 @@
#include <string.h>
#include <zlib.h>
+#ifdef USE_ZSTD
+#include <zstd.h>
+#endif
+
/* Cleanup and return result. Don't leak memory. */
static void *
do_deflate_cleanup (void *result, z_stream *z, void *out_buf,
@@ -207,7 +211,7 @@ __libelf_compress (Elf_Scn *scn, size_t hsize, int ei_data,
void *
internal_function
-__libelf_decompress (void *buf_in, size_t size_in, size_t size_out)
+__libelf_decompress_zlib (void *buf_in, size_t size_in, size_t size_out)
{
/* Catch highly unlikely compression ratios so we don't allocate
some giant amount of memory for nothing. The max compression
@@ -260,6 +264,50 @@ __libelf_decompress (void *buf_in, size_t size_in, size_t size_out)
return buf_out;
}
+#ifdef USE_ZSTD
+void *
+internal_function
+__libelf_decompress_zstd (void *buf_in, size_t size_in, size_t size_out)
+{
+ /* Malloc might return NULL when requestion zero size. This is highly
+ unlikely, it would only happen when the compression was forced.
+ But we do need a non-NULL buffer to return and set as result.
+ Just make sure to always allocate at least 1 byte. */
+ void *buf_out = malloc (size_out ?: 1);
+ if (unlikely (buf_out == NULL))
+ {
+ __libelf_seterrno (ELF_E_NOMEM);
+ return NULL;
+ }
+
+ size_t ret = ZSTD_decompress (buf_out, size_out, buf_in, size_in);
+ if (ZSTD_isError (ret))
+ {
+ free (buf_out);
+ __libelf_seterrno (ELF_E_DECOMPRESS_ERROR);
+ return NULL;
+ }
+ else
+ return buf_out;
+}
+#endif
+
+void *
+internal_function
+__libelf_decompress (int chtype, void *buf_in, size_t size_in, size_t size_out)
+{
+ if (chtype == ELFCOMPRESS_ZLIB)
+ return __libelf_decompress_zlib (buf_in, size_in, size_out);
+ else
+ {
+#ifdef USE_ZSTD
+ return __libelf_decompress_zstd (buf_in, size_in, size_out);
+#else
+ return 0;
+#endif
+ }
+}
+
void *
internal_function
__libelf_decompress_elf (Elf_Scn *scn, size_t *size_out, size_t *addralign)
@@ -268,7 +316,7 @@ __libelf_decompress_elf (Elf_Scn *scn, size_t *size_out, size_t *addralign)
if (gelf_getchdr (scn, &chdr) == NULL)
return NULL;
- if (chdr.ch_type != ELFCOMPRESS_ZLIB)
+ if (chdr.ch_type != ELFCOMPRESS_ZLIB && chdr.ch_type != ELFCOMPRESS_ZSTD)
{
__libelf_seterrno (ELF_E_UNKNOWN_COMPRESSION_TYPE);
return NULL;
@@ -295,7 +343,9 @@ __libelf_decompress_elf (Elf_Scn *scn, size_t *size_out, size_t *addralign)
? sizeof (Elf32_Chdr) : sizeof (Elf64_Chdr));
size_t size_in = data->d_size - hsize;
void *buf_in = data->d_buf + hsize;
- void *buf_out = __libelf_decompress (buf_in, size_in, chdr.ch_size);
+ void *buf_out
+ = __libelf_decompress (chdr.ch_type, buf_in, size_in, chdr.ch_size);
+
*size_out = chdr.ch_size;
*addralign = chdr.ch_addralign;
return buf_out;
diff --git a/libelf/elf_compress_gnu.c b/libelf/elf_compress_gnu.c
index 3d2977e7..be9e990e 100644
--- a/libelf/elf_compress_gnu.c
+++ b/libelf/elf_compress_gnu.c
@@ -178,7 +178,7 @@ elf_compress_gnu (Elf_Scn *scn, int inflate, unsigned int flags)
size_t size = gsize;
size_t size_in = data->d_size - hsize;
void *buf_in = data->d_buf + hsize;
- void *buf_out = __libelf_decompress (buf_in, size_in, size);
+ void *buf_out = __libelf_decompress (ELFCOMPRESS_ZLIB, buf_in, size_in, size);
if (buf_out == NULL)
return -1;
diff --git a/libelf/libelfP.h b/libelf/libelfP.h
index d88a613c..ab82357c 100644
--- a/libelf/libelfP.h
+++ b/libelf/libelfP.h
@@ -577,7 +577,7 @@ extern void * __libelf_compress (Elf_Scn *scn, size_t hsize, int ei_data,
size_t *size, bool force)
internal_function;
-extern void * __libelf_decompress (void *buf_in, size_t size_in,
+extern void * __libelf_decompress (int chtype, void *buf_in, size_t size_in,
size_t size_out) internal_function;
extern void * __libelf_decompress_elf (Elf_Scn *scn,
size_t *size_out, size_t *addralign)
diff --git a/src/readelf.c b/src/readelf.c
index a206e60e..1af20e35 100644
--- a/src/readelf.c
+++ b/src/readelf.c
@@ -1229,13 +1229,17 @@ get_visibility_type (int value)
static const char *
elf_ch_type_name (unsigned int code)
{
- if (code == 0)
- return "NONE";
-
- if (code == ELFCOMPRESS_ZLIB)
- return "ZLIB";
-
- return "UNKNOWN";
+ switch (code)
+ {
+ case 0:
+ return "NONE";
+ case ELFCOMPRESS_ZLIB:
+ return "ZLIB";
+ case ELFCOMPRESS_ZSTD:
+ return "ZSTD";
+ default:
+ return "UNKNOWN";
+ }
}
/* Print the section headers. */
--
2.38.0
next prev parent reply other threads:[~2022-10-24 18:16 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-24 11:09 Martin Liška
2022-10-24 11:41 ` Dmitry V. Levin
2022-10-24 12:17 ` Martin Liška
2022-10-24 16:48 ` Dmitry V. Levin
2022-10-24 18:16 ` Martin Liška [this message]
2022-10-28 22:21 ` Mark Wielaard
2022-11-28 13:16 ` Martin Liška
2022-11-28 22:29 ` Mark Wielaard
2022-11-29 9:34 ` Martin Liška
2022-11-29 12:05 ` [PATCHv2] support ZSTD compression algorithm Martin Liška
2022-12-09 10:17 ` Martin Liška
2022-12-15 13:17 ` Mark Wielaard
2022-12-19 14:19 ` Martin Liška
2022-12-19 15:09 ` Mark Wielaard
2022-12-21 11:13 ` Martin Liška
2023-01-10 17:44 ` [PATCH] readelf: Check compression status of .debug section data Mark Wielaard
2023-01-16 19:39 ` Mark Wielaard
2022-12-16 0:32 ` [PATCHv2] support ZSTD compression algorithm Mark Wielaard
2022-12-19 14:21 ` Martin Liška
2022-12-19 15:16 ` Mark Wielaard
2022-12-21 11:09 ` Martin Liška
2022-12-21 23:14 ` Mark Wielaard
2022-12-22 9:17 ` Martin Liška
2022-12-22 18:36 ` Mark Wielaard
2022-12-23 8:44 ` Martin Liška
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6c1ce1f1-2e45-20bb-e98d-6d35692addfb@suse.cz \
--to=mliska@suse.cz \
--cc=elfutils-devel@sourceware.org \
--cc=ldv@altlinux.org \
--cc=mark@klomp.org \
--cc=maskray@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).