From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by sourceware.org (Postfix) with ESMTPS id EC3753858D1E for ; Mon, 19 Dec 2022 14:19:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org EC3753858D1E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=suse.cz Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id ECCD960CF0; Mon, 19 Dec 2022 14:19:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1671459566; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H5Y6GZ0R6K1moTTQKBMCNydC4p+quhN91xLn0xq3X+I=; b=yHhXiI+V6TWckrCyb9e5i0PHRrKhLgkQHIYtJbCfsd4UbzaHOJ9+gram3t48u7s6fYL6hU aZQJ+Ot4lfE6pIsUA7ZdvqtaONaUk42wD6Zk/9Rty9gfo6m5d58PD36+DhbUz7dfffSy+9 gNxs38j3TadMulWyS04uI1pjvaAsJr4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1671459566; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H5Y6GZ0R6K1moTTQKBMCNydC4p+quhN91xLn0xq3X+I=; b=ujFjO2As/sL7fqU/bLtxmzmKl8qaun8SHc90JpQ1jI+ANz6YlFeWU433SkEBrNcBBfNAaq gZjK+7YNTW675XDA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DD3E913910; Mon, 19 Dec 2022 14:19:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id JNUtNe5yoGNYNQAAMHmgww (envelope-from ); Mon, 19 Dec 2022 14:19:26 +0000 Message-ID: Date: Mon, 19 Dec 2022 15:19:26 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: [PATCHv2] support ZSTD compression algorithm Content-Language: en-US To: Mark Wielaard , elfutils-devel@sourceware.org References: <24d7165b-b8ac-ea5d-a046-aec2203696a9@suse.cz> From: =?UTF-8?Q?Martin_Li=c5=a1ka?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00,BODY_8BITS,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 12/15/22 14:17, Mark Wielaard wrote: > Hi Martin, > > On Tue, 2022-11-29 at 13:05 +0100, Martin Liška wrote: >> There's second version of the patch that fully support both >> compression and decompression. >> >> Changes from the v1: >> - compression support added >> - zstd detection is fixed >> - new tests are added >> - builds fine w/ and w/o the ZSTD library >> >> What's currently missing and where I need a help: >> >> 1) When I build ./configure --without-zstd, I don't have >> a reasonable error message (something similar to binutils's readelf: >> readelf: Warning: section '.debug_str' has unsupported compress type: >> 2) >> even though, __libelf_decompress returns NULL and __libelf_seterrno). >> One can see a garbage in the console. >> >> How to handle that properly? > > Is there a particular way you are running eu-readelf? Is it with > generic -w or -a, or decoding a specific section type? Hello. $ LD_LIBRARY_PATH=./libelf ./src/readelf -w ~/Programming/testcases/a.out where I get: ./src/readelf: cannot get debug context descriptor: No DWARF information found DWARF section [37] '.debug_info' at offset 0x1ab2: [Offset] ./src/readelf: cannot get next unit: no error Call frame information section [13] '.eh_frame' at offset 0x4a8: ... t��o5��=I�iAp@a����S^R/<�����^�qi�ַp@ E���Z �O��Š�w��/#�!���7�6�#�I����*���굮R�v妗�/Z���O����Oի���E��z�����K��(��9���:{ѧ�zOa;̥�O޾�����+O�̰҆ˑ}��C��ۋ_�k9�� Jv�>;)`F�� :!�-�ˏQ@�L, V��6cI�Ъ^�6z��6�4�Fz���n���}~�U��R�])��zF��#�V��E�eȹ/� Z�A!DJP%"�H�$i� Bc3�" *** error, missing string terminator So basically a garbage. And I don't know how to bail out properly? > >> 2) How should I run my newly added tests conditionally if zstd >> configuration support is enabled? > > eu_ZIP will define an AM_CONDITIONAL for ZSTD (note this is different > from the HAVE_ZSTD, which is the am conditional added for having the > zstd program itself). You could use it in tests/Makefile.am as: > > if ZSTD > TESTS += run-zstd-compress-test.sh > endif > > If you had a separate test... Otherwise add some variable to > TESTS_ENVIRONMENT (and installed_TESTS_ENVIRONMENT) based on the Make > variable that is tested inside the shell script (somewhat like > USE_VALGRIND) maybe. All right, I have a working version where I utilize an env. variable. > >> configure.ac | 8 +- >> libelf/Makefile.am | 2 +- >> libelf/elf_compress.c | 294 ++++++++++++++++++++++++++++++-- >> ----- >> libelf/elf_compress_gnu.c | 5 +- >> libelf/libelfP.h | 4 +- >> src/elfcompress.c | 144 ++++++++++-------- >> src/readelf.c | 18 ++- >> tests/run-compress-test.sh | 24 +++ >> 8 files changed, 373 insertions(+), 126 deletions(-) >> >> diff --git a/configure.ac b/configure.ac >> index 59be27ac..07cfa54b 100644 >> --- a/configure.ac >> +++ b/configure.ac >> @@ -410,6 +410,11 @@ dnl Test for bzlib and xz/lzma/zstd, gives BZLIB/LZMALIB/ZSTD .am >> dnl conditional and config.h USE_BZLIB/USE_LZMALIB/USE_ZSTD #define. >> save_LIBS="$LIBS" >> LIBS= >> +eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)]) >> +AS_IF([test "x$with_zstd" = xyes], [LIBZSTD="libzstd"], [LIBLZSTD=""]) >> +AC_SUBST([LIBZSTD]) >> +zstd_LIBS="$LIBS" >> +AC_SUBST([zstd_LIBS]) >> eu_ZIPLIB(bzlib,BZLIB,bz2,BZ2_bzdopen,bzip2) >> # We need this since bzip2 doesn't have a pkgconfig file. >> BZ2_LIB="$LIBS" >> @@ -417,9 +422,6 @@ AC_SUBST([BZ2_LIB]) >> eu_ZIPLIB(lzma,LZMA,lzma,lzma_auto_decoder,[LZMA (xz)]) >> AS_IF([test "x$with_lzma" = xyes], [LIBLZMA="liblzma"], [LIBLZMA=""]) >> AC_SUBST([LIBLZMA]) >> -eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)]) >> -AS_IF([test "x$with_zstd" = xyes], [LIBZSTD="libzstd"], [LIBLZSTD=""]) >> -AC_SUBST([LIBZSTD]) >> zip_LIBS="$LIBS" >> LIBS="$save_LIBS" >> AC_SUBST([zip_LIBS]) > > Doing AC_SUBST([zstd_LIBS]) seems correct. Why is the test moved > earlier? > > You are testing for ZSTD_decompress, is that enough? Asking because I > see you are using ZSTD_compressStream2, which seems to requires libzstd > v1.4.0+. > > In general how stable is the libzstd api? It's hopefully stable, but yes, I should check for ZSTD_compressStream2 in configure.ac. I'm going to update that. > > You'll also need to use the AC_SUBST for LIBZSTD in config/libelf.pc.in > now, as used in the libdw.pc.in: > > Requires.private: zlib @LIBZSTD@ Oh, thanks. Cheers, Martin > >> diff --git a/libelf/Makefile.am b/libelf/Makefile.am >> index 560ed45f..24c25cf8 100644 >> --- a/libelf/Makefile.am >> +++ b/libelf/Makefile.am >> @@ -106,7 +106,7 @@ libelf_pic_a_SOURCES = >> am_libelf_pic_a_OBJECTS = $(libelf_a_SOURCES:.c=.os) >> >> libelf_so_DEPS = ../lib/libeu.a >> -libelf_so_LDLIBS = $(libelf_so_DEPS) -lz >> +libelf_so_LDLIBS = $(libelf_so_DEPS) -lz $(zstd_LIBS) >> if USE_LOCKS >> libelf_so_LDLIBS += -lpthread >> endif > > OK. > > Haven't read the actual code yet. I'll get back to that later today. > > Cheers, > > Mark