public inbox for elfutils@sourceware.org
 help / color / mirror / Atom feed
* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
       [not found] <20221221111547.74C4C385B50B@sourceware.org>
@ 2022-12-21 14:20 ` Martin Liška
  2022-12-21 14:42   ` Mark Wielaard
  0 siblings, 1 reply; 8+ messages in thread
From: Martin Liška @ 2022-12-21 14:20 UTC (permalink / raw)
  To: Mark Wielaard; +Cc: elfutils-devel

On 12/21/22 12:15, builder@sourceware.org wrote:
> A failed build has been detected on builder elfutils-try-debian-armhf while building elfutils.
> 
> Full details are available at:
>     https://builder.sourceware.org/buildbot/#builders/201/builds/47
> 
> Build state: failed compile (failure)
> Revision: 1b845fdf4ca4d5367969cc5b33cbfa60338cf51b
> Worker: debian-armhf
> Build Reason: (unknown)
> Blamelist: Martin Liska <mliska@suse.cz>
> 
> Steps:
> 
> - 0: worker_preparation ( success )
> 
> - 1: set package name ( success )
> 
> - 2: git checkout ( success )
>     Logs:
>         - stdio: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/2/logs/stdio
> 
> - 3: autoreconf ( success )
>     Logs:
>         - stdio: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/3/logs/stdio
> 
> - 4: configure ( success )
>     Logs:
>         - stdio: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/4/logs/stdio
> 
> - 5: get version ( success )
>     Logs:
>         - stdio: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/5/logs/stdio
>         - property changes: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/5/logs/property_changes
> 
> - 6: make ( failure )
>     Logs:
>         - stdio: https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/6/logs/stdio
> 

Mark, do you know what can be wrong about the configure test:

elf_compress.c: In function ‘__libelf_compress_zstd’:
elf_compress.c:205:3: error: unknown type name ‘ZSTD_EndDirective’; did you mean ‘ZSTD_DDict’?
   ZSTD_EndDirective mode = ZSTD_e_continue;
   ^~~~~~~~~~~~~~~~~
   ZSTD_DDict
elf_compress.c:205:28: error: ‘ZSTD_e_continue’ undeclared (first use in this function)
   ZSTD_EndDirective mode = ZSTD_e_continue;
                            ^~~~~~~~~~~~~~~
elf_compress.c:205:28: note: each undeclared identifier is reported only once for each function it appears in
elf_compress.c:237:9: error: ‘ZSTD_e_end’ undeclared (first use in this function); did you mean ‘ZSTD_DDict’?
  mode = ZSTD_e_end;
         ^~~~~~~~~~
         ZSTD_DDict
elf_compress.c:243:17: error: implicit declaration of function ‘ZSTD_compressStream2’; did you mean ‘ZSTD_compressStream’? [-Werror=implicit-function-declaration]
    size_t ret = ZSTD_compressStream2 (cctx, &ob, &ib, mode);
                 ^~~~~~~~~~~~~~~~~~~~
                 ZSTD_compressStream

Thanks,
Martin

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 14:20 ` ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2) Martin Liška
@ 2022-12-21 14:42   ` Mark Wielaard
  2022-12-21 14:54     ` Frank Ch. Eigler
  2022-12-21 15:21     ` Mark Wielaard
  0 siblings, 2 replies; 8+ messages in thread
From: Mark Wielaard @ 2022-12-21 14:42 UTC (permalink / raw)
  To: Martin Liška; +Cc: elfutils-devel

Hi Martin,

On Wed, 2022-12-21 at 15:20 +0100, Martin Liška wrote:
> On 12/21/22 12:15, builder@sourceware.org wrote:
> > A failed build has been detected on builder elfutils-try-debian-
> > armhf while building elfutils.
> > 
> > Full details are available at:
> >     https://builder.sourceware.org/buildbot/#builders/201/builds/47
> > 
> > Build state: failed compile (failure)
> > Revision: 1b845fdf4ca4d5367969cc5b33cbfa60338cf51b
> > Worker: debian-armhf
> > Build Reason: (unknown)
> > Blamelist: Martin Liska <mliska@suse.cz>
> > 
> > Steps:
> > 
> > - 0: worker_preparation ( success )
> > 
> > - 1: set package name ( success )
> > 
> > - 2: git checkout ( success )
> >     Logs:
> >         - stdio: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/2/logs/stdio
> > 
> > - 3: autoreconf ( success )
> >     Logs:
> >         - stdio: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/3/logs/stdio
> > 
> > - 4: configure ( success )
> >     Logs:
> >         - stdio: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/4/logs/stdio
> > 
> > - 5: get version ( success )
> >     Logs:
> >         - stdio: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/5/logs/stdio
> >         - property changes: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/5/logs/property_changes
> > 
> > - 6: make ( failure )
> >     Logs:
> >         - stdio: 
> > https://builder.sourceware.org/buildbot/#builders/201/builds/47/steps/6/logs/stdio
> > 
> 
> Mark, do you know what can be wrong about the configure test:
> 
> elf_compress.c: In function ‘__libelf_compress_zstd’:
> elf_compress.c:205:3: error: unknown type name ‘ZSTD_EndDirective’;
> did you mean ‘ZSTD_DDict’?
>    ZSTD_EndDirective mode = ZSTD_e_continue;
>    ^~~~~~~~~~~~~~~~~
>    ZSTD_DDict
> elf_compress.c:205:28: error: ‘ZSTD_e_continue’ undeclared (first use
> in this function)
>    ZSTD_EndDirective mode = ZSTD_e_continue;
>                             ^~~~~~~~~~~~~~~
> elf_compress.c:205:28: note: each undeclared identifier is reported
> only once for each function it appears in
> elf_compress.c:237:9: error: ‘ZSTD_e_end’ undeclared (first use in
> this function); did you mean ‘ZSTD_DDict’?
>   mode = ZSTD_e_end;
>          ^~~~~~~~~~
>          ZSTD_DDict
> elf_compress.c:243:17: error: implicit declaration of function
> ‘ZSTD_compressStream2’; did you mean ‘ZSTD_compressStream’? [-
> Werror=implicit-function-declaration]
>     size_t ret = ZSTD_compressStream2 (cctx, &ob, &ib, mode);
>                  ^~~~~~~~~~~~~~~~~~~~
>                  ZSTD_compressStream

Heay cool, the try buildbot worked.

The configure test seems to succeed (from the configure log):
checking for library containing ZSTD_compressStream2... -lzstd

Unfortunately buildbot itself doesn't show the config.log.
Trying to get that...

configure:6961: checking for library containing ZSTD_compressStream2
configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
-fsanitize=undefined 
-fno-sanitize-recover -fsanitize=address -fno-sanitize-
recover   conftest.c -llz
ma -lbz2  >&5
/usr/bin/ld: /tmp/ccKZnuYA.o: in function `main':
/var/lib/buildbot/workers/wildebeest/elfutils-try-debian-
armhf/build/conftest.c:
30: undefined reference to `ZSTD_compressStream2'
collect2: error: ld returned 1 exit status
configure:6992: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "elfutils"
| #define PACKAGE_TARNAME "elfutils"
| #define PACKAGE_VERSION "0.188"
| #define PACKAGE_STRING "elfutils 0.188"
| #define PACKAGE_BUGREPORT "https://sourceware.org/bugzilla"
| #define PACKAGE_URL "http://elfutils.org/"
| #define PACKAGE "elfutils"
| #define VERSION "0.188"
| #define DEFAULT_AR_DETERMINISTIC false
| #define HAVE_VISIBILITY 1
| #define HAVE_STDATOMIC_H 1
| #define _FILE_OFFSET_BITS 64
| #define CHECK_UNDEFINED 1
| #define USE_ZLIB 1
| #define USE_BZLIB 1
| #define USE_LZMA 1
| /* end confdefs.h.  */
| 
| /* Override any GCC internal prototype to avoid an error.
|    Use char because int might match the return type of a GCC
|    builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char ZSTD_compressStream2 ();
| int
| main ()
| {
| return ZSTD_compressStream2 ();
|   ;
|   return 0;
| }
configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
-fsanitize=undefined 
-fno-sanitize-recover -fsanitize=address -fno-sanitize-
recover   conftest.c -lzs
td  -llzma -lbz2  >&5
configure:6992: $? = 0
configure:7009: result: -lzstd

ehe? So it fails and then succeeds anyway?

Strangely the same happens for the other compression library checks...
lzma_auto_decoder and BZ2_bzdopen

I don't fully understand this. Oddly the same seems to happen locally
on my x86_64 box. So maybe the configure macros we have been using have
always been bad/broken?

Cheers,

Mark

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 14:42   ` Mark Wielaard
@ 2022-12-21 14:54     ` Frank Ch. Eigler
  2022-12-21 15:21     ` Mark Wielaard
  1 sibling, 0 replies; 8+ messages in thread
From: Frank Ch. Eigler @ 2022-12-21 14:54 UTC (permalink / raw)
  To: Mark Wielaard; +Cc: Martin Liška, elfutils-devel

Hi -

> [...]
> Unfortunately buildbot itself doesn't show the config.log.
> Trying to get that...

Ah darn.  It would have been saved in bunsen, but the buildbot
configuration means that a make-stage failure means it won't even try.

- FChE


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 14:42   ` Mark Wielaard
  2022-12-21 14:54     ` Frank Ch. Eigler
@ 2022-12-21 15:21     ` Mark Wielaard
  2022-12-21 18:29       ` Martin Liška
  1 sibling, 1 reply; 8+ messages in thread
From: Mark Wielaard @ 2022-12-21 15:21 UTC (permalink / raw)
  To: Martin Liška; +Cc: elfutils-devel

Hi,

On Wed, 2022-12-21 at 15:42 +0100, Mark Wielaard wrote:
> 
> Unfortunately buildbot itself doesn't show the config.log.
> Trying to get that...
> 
> configure:6961: checking for library containing ZSTD_compressStream2
> configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
> -fsanitize=undefined 
> -fno-sanitize-recover -fsanitize=address -fno-sanitize-
> recover   conftest.c -llz
> ma -lbz2  >&5
> /usr/bin/ld: /tmp/ccKZnuYA.o: in function `main':
> /var/lib/buildbot/workers/wildebeest/elfutils-try-debian-
> armhf/build/conftest.c:
> 30: undefined reference to `ZSTD_compressStream2'
> collect2: error: ld returned 1 exit status
> configure:6992: $? = 1
> configure: failed program was:
> > /* confdefs.h */
> > #define PACKAGE_NAME "elfutils"
> > #define PACKAGE_TARNAME "elfutils"
> > #define PACKAGE_VERSION "0.188"
> > #define PACKAGE_STRING "elfutils 0.188"
> > #define PACKAGE_BUGREPORT "https://sourceware.org/bugzilla"
> > #define PACKAGE_URL "http://elfutils.org/"
> > #define PACKAGE "elfutils"
> > #define VERSION "0.188"
> > #define DEFAULT_AR_DETERMINISTIC false
> > #define HAVE_VISIBILITY 1
> > #define HAVE_STDATOMIC_H 1
> > #define _FILE_OFFSET_BITS 64
> > #define CHECK_UNDEFINED 1
> > #define USE_ZLIB 1
> > #define USE_BZLIB 1
> > #define USE_LZMA 1
> > /* end confdefs.h.  */
> > 
> > /* Override any GCC internal prototype to avoid an error.
> >    Use char because int might match the return type of a GCC
> >    builtin and then its argument prototype would still apply.  */
> > #ifdef __cplusplus
> > extern "C"
> > #endif
> > char ZSTD_compressStream2 ();
> > int
> > main ()
> > {
> > return ZSTD_compressStream2 ();
> >   ;
> >   return 0;
> > }
> 
> configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
> -fsanitize=undefined 
> -fno-sanitize-recover -fsanitize=address -fno-sanitize-
> recover   conftest.c -lzs
> td  -llzma -lbz2  >&5
> configure:6992: $? = 0
> configure:7009: result: -lzstd
> 
> ehe? So it fails and then succeeds anyway?
> 
> Strangely the same happens for the other compression library
> checks...
> lzma_auto_decoder and BZ2_bzdopen
> 
> I don't fully understand this. Oddly the same seems to happen locally
> on my x86_64 box. So maybe the configure macros we have been using
> have
> always been bad/broken?

No, this is "normal". It first tries without the -l library (which is
the failing case) and then with the -l library (which succeeds).

What goes wrong is that this debian old stable arm setup has libzstd
containing a ZSTD_compressStream2 symbol...

    40: 0000d349   264 FUNC    GLOBAL DEFAULT   11 ZSTD_compressStream2

But the zstd.h header doesn't expose it...

/*******************************************************************************
*********
 *   ADVANCED AND EXPERIMENTAL FUNCTIONS
 *******************************************************************************
*********
 * The definitions in the following section are considered experimental.
 * They are provided for advanced scenarios.
 * They should never be used with a dynamic library, as prototypes may change in
 the future.
 * Use them only in association with static linking.
 * *****************************************************************************
**********/

#if defined(ZSTD_STATIC_LINKING_ONLY) && !defined(ZSTD_H_ZSTD_STATIC_LINKING_ONLY)

...

grmbl.

This is somewhat unfortunate. We need a smarter configure check for this...

Cheers,

Mark

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 15:21     ` Mark Wielaard
@ 2022-12-21 18:29       ` Martin Liška
  2022-12-21 23:38         ` Mark Wielaard
  0 siblings, 1 reply; 8+ messages in thread
From: Martin Liška @ 2022-12-21 18:29 UTC (permalink / raw)
  To: Mark Wielaard; +Cc: elfutils-devel

On 12/21/22 16:21, Mark Wielaard wrote:
> Hi,
> 
> On Wed, 2022-12-21 at 15:42 +0100, Mark Wielaard wrote:
>>
>> Unfortunately buildbot itself doesn't show the config.log.
>> Trying to get that...
>>
>> configure:6961: checking for library containing ZSTD_compressStream2
>> configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
>> -fsanitize=undefined 
>> -fno-sanitize-recover -fsanitize=address -fno-sanitize-
>> recover   conftest.c -llz
>> ma -lbz2  >&5
>> /usr/bin/ld: /tmp/ccKZnuYA.o: in function `main':
>> /var/lib/buildbot/workers/wildebeest/elfutils-try-debian-
>> armhf/build/conftest.c:
>> 30: undefined reference to `ZSTD_compressStream2'
>> collect2: error: ld returned 1 exit status
>> configure:6992: $? = 1
>> configure: failed program was:
>>> /* confdefs.h */
>>> #define PACKAGE_NAME "elfutils"
>>> #define PACKAGE_TARNAME "elfutils"
>>> #define PACKAGE_VERSION "0.188"
>>> #define PACKAGE_STRING "elfutils 0.188"
>>> #define PACKAGE_BUGREPORT "https://sourceware.org/bugzilla"
>>> #define PACKAGE_URL "http://elfutils.org/"
>>> #define PACKAGE "elfutils"
>>> #define VERSION "0.188"
>>> #define DEFAULT_AR_DETERMINISTIC false
>>> #define HAVE_VISIBILITY 1
>>> #define HAVE_STDATOMIC_H 1
>>> #define _FILE_OFFSET_BITS 64
>>> #define CHECK_UNDEFINED 1
>>> #define USE_ZLIB 1
>>> #define USE_BZLIB 1
>>> #define USE_LZMA 1
>>> /* end confdefs.h.  */
>>>
>>> /* Override any GCC internal prototype to avoid an error.
>>>    Use char because int might match the return type of a GCC
>>>    builtin and then its argument prototype would still apply.  */
>>> #ifdef __cplusplus
>>> extern "C"
>>> #endif
>>> char ZSTD_compressStream2 ();
>>> int
>>> main ()
>>> {
>>> return ZSTD_compressStream2 ();
>>>   ;
>>>   return 0;
>>> }
>>
>> configure:6992: gcc -o conftest -D_FORTIFY_SOURCE=3 -g -O2
>> -fsanitize=undefined 
>> -fno-sanitize-recover -fsanitize=address -fno-sanitize-
>> recover   conftest.c -lzs
>> td  -llzma -lbz2  >&5
>> configure:6992: $? = 0
>> configure:7009: result: -lzstd
>>
>> ehe? So it fails and then succeeds anyway?
>>
>> Strangely the same happens for the other compression library
>> checks...
>> lzma_auto_decoder and BZ2_bzdopen
>>
>> I don't fully understand this. Oddly the same seems to happen locally
>> on my x86_64 box. So maybe the configure macros we have been using
>> have
>> always been bad/broken?
> 

Hi.

Good!

> No, this is "normal". It first tries without the -l library (which is
> the failing case) and then with the -l library (which succeeds).

Ah, makes sense.

> 
> What goes wrong is that this debian old stable arm setup has libzstd
> containing a ZSTD_compressStream2 symbol...
> 
>     40: 0000d349   264 FUNC    GLOBAL DEFAULT   11 ZSTD_compressStream2
> 
> But the zstd.h header doesn't expose it...

Yeah, that's their way of how to move an API from "staging" into a stable state.
It was changed in:
https://github.com/facebook/zstd/commit/d7d89513d6a21

and so it's present in zstd since 1.4.0. Can we somehow specify library version
in configure.ac?

Thanks,
Martin

> 
> /*******************************************************************************
> *********
>  *   ADVANCED AND EXPERIMENTAL FUNCTIONS
>  *******************************************************************************
> *********
>  * The definitions in the following section are considered experimental.
>  * They are provided for advanced scenarios.
>  * They should never be used with a dynamic library, as prototypes may change in
>  the future.
>  * Use them only in association with static linking.
>  * *****************************************************************************
> **********/
> 
> #if defined(ZSTD_STATIC_LINKING_ONLY) && !defined(ZSTD_H_ZSTD_STATIC_LINKING_ONLY)
> 
> ...
> 
> grmbl.
> 
> This is somewhat unfortunate. We need a smarter configure check for this...
> 
> Cheers,
> 
> Mark


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 18:29       ` Martin Liška
@ 2022-12-21 23:38         ` Mark Wielaard
  2022-12-22  9:09           ` Martin Liška
  0 siblings, 1 reply; 8+ messages in thread
From: Mark Wielaard @ 2022-12-21 23:38 UTC (permalink / raw)
  To: Martin Liška; +Cc: Mark Wielaard, elfutils-devel

Hi Martin,

On Wed, Dec 21, 2022 at 07:29:19PM +0100, Martin Liška wrote:
> > What goes wrong is that this debian old stable arm setup has libzstd
> > containing a ZSTD_compressStream2 symbol...
> > 
> >     40: 0000d349   264 FUNC    GLOBAL DEFAULT   11 ZSTD_compressStream2
> > 
> > But the zstd.h header doesn't expose it...
> 
> Yeah, that's their way of how to move an API from "staging" into a stable state.
> It was changed in:
> https://github.com/facebook/zstd/commit/d7d89513d6a21
> 
> and so it's present in zstd since 1.4.0. Can we somehow specify library version
> in configure.ac?

It isn't as nice, but since there seems to be a pkgconfig libzstd.pc
you could use PKG_CHECK_MODULES. See for example how we check for
libcurl.

Cheers,

Mark

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-21 23:38         ` Mark Wielaard
@ 2022-12-22  9:09           ` Martin Liška
  2022-12-22 18:17             ` Mark Wielaard
  0 siblings, 1 reply; 8+ messages in thread
From: Martin Liška @ 2022-12-22  9:09 UTC (permalink / raw)
  To: Mark Wielaard; +Cc: Mark Wielaard, elfutils-devel

On 12/22/22 00:38, Mark Wielaard wrote:
> Hi Martin,
> 
> On Wed, Dec 21, 2022 at 07:29:19PM +0100, Martin Liška wrote:
>>> What goes wrong is that this debian old stable arm setup has libzstd
>>> containing a ZSTD_compressStream2 symbol...
>>>
>>>     40: 0000d349   264 FUNC    GLOBAL DEFAULT   11 ZSTD_compressStream2
>>>
>>> But the zstd.h header doesn't expose it...
>>
>> Yeah, that's their way of how to move an API from "staging" into a stable state.
>> It was changed in:
>> https://github.com/facebook/zstd/commit/d7d89513d6a21
>>
>> and so it's present in zstd since 1.4.0. Can we somehow specify library version
>> in configure.ac?
> 

Hi Mark.

> It isn't as nice, but since there seems to be a pkgconfig libzstd.pc
> you could use PKG_CHECK_MODULES. See for example how we check for
> libcurl.

I can see it, however, it not easily addable to eu_ZIPLIB function :/ I must confess,
this autoconf machinery is always something that makes me crazy :)

Do you have an elegant way how to handle it?

Thanks,
Martin

> 
> Cheers,
> 
> Mark


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2)
  2022-12-22  9:09           ` Martin Liška
@ 2022-12-22 18:17             ` Mark Wielaard
  0 siblings, 0 replies; 8+ messages in thread
From: Mark Wielaard @ 2022-12-22 18:17 UTC (permalink / raw)
  To: Martin Liška; +Cc: elfutils-devel

[-- Attachment #1: Type: text/plain, Size: 1554 bytes --]

On Thu, 2022-12-22 at 10:09 +0100, Martin Liška wrote:
> On 12/22/22 00:38, Mark Wielaard wrote:
> > Hi Martin,
> > 
> > On Wed, Dec 21, 2022 at 07:29:19PM +0100, Martin Liška wrote:
> > > > What goes wrong is that this debian old stable arm setup has
> > > > libzstd
> > > > containing a ZSTD_compressStream2 symbol...
> > > > 
> > > >     40: 0000d349   264 FUNC    GLOBAL DEFAULT   11
> > > > ZSTD_compressStream2
> > > > 
> > > > But the zstd.h header doesn't expose it...
> > > 
> > > Yeah, that's their way of how to move an API from "staging" into
> > > a stable state.
> > > It was changed in:
> > > https://github.com/facebook/zstd/commit/d7d89513d6a21
> > > 
> > > and so it's present in zstd since 1.4.0. Can we somehow specify
> > > library version
> > > in configure.ac?
> 
> Hi Mark.
> 
> > It isn't as nice, but since there seems to be a pkgconfig
> > libzstd.pc
> > you could use PKG_CHECK_MODULES. See for example how we check for
> > libcurl.
> 
> I can see it, however, it not easily addable to eu_ZIPLIB function :/
> I must confess,
> this autoconf machinery is always something that makes me crazy :)
> 
> Do you have an elegant way how to handle it?

Maybe we should simply split it from the "decompress" support. Trying
to make eu_ZIPLIB handle it means we also suddenly require a newer
libzstd for the simpler dwfl zstd file decompression support.

So lets keep the "normal" zstd detection in place and add something
like the attached (on top of your latest patch).

Cheers,

Mark

[-- Attachment #2: Type: text/x-patch, Size: 4132 bytes --]

diff --git a/configure.ac b/configure.ac
index ef16f79e..b2597f8b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -417,7 +417,7 @@ AC_SUBST([BZ2_LIB])
 eu_ZIPLIB(lzma,LZMA,lzma,lzma_auto_decoder,[LZMA (xz)])
 AS_IF([test "x$with_lzma" = xyes], [LIBLZMA="liblzma"], [LIBLZMA=""])
 AC_SUBST([LIBLZMA])
-eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_compressStream2,[ZSTD (zst)])
+eu_ZIPLIB(zstd,ZSTD,zstd,ZSTD_decompress,[ZSTD (zst)])
 AS_IF([test "x$with_zstd" = xyes], [LIBZSTD="libzstd"], [LIBLZSTD=""])
 AC_SUBST([LIBZSTD])
 zstd_LIBS="$LIBS"
@@ -426,6 +426,16 @@ zip_LIBS="$LIBS"
 LIBS="$save_LIBS"
 AC_SUBST([zip_LIBS])
 
+dnl zstd compression support requires libzstd 1.4.0+
+AS_IF([test "x$with_zstd" = xyes], [
+      PKG_PROG_PKG_CONFIG
+      PKG_CHECK_MODULES([ZSTD_COMPRESS],[libzstd >= 1.4.0],
+                        [with_zstd_compress="yes"],[with_zstd_compress="no"])],
+      [with_zstd_compress="no"])
+AM_CONDITIONAL(USE_ZSTD_COMPRESS, test "x$with_zstd_compress" = "xyes")
+AS_IF([test "x$with_zstd_compress" = "xyes"],
+      [AC_DEFINE([USE_ZSTD_COMPRESS], [1], [zstd compression support])])
+
 AC_CHECK_DECLS([memrchr, rawmemchr],[],[],
                [#define _GNU_SOURCE
                 #include <string.h>])
@@ -831,6 +841,7 @@ AC_MSG_NOTICE([
     bzip2 support                      : ${with_bzlib}
     lzma/xz support                    : ${with_lzma}
     zstd support                       : ${with_zstd}
+    zstd compression support           : ${with_zstd_compress}
     libstdc++ demangle support         : ${enable_demangler}
     File textrel check                 : ${enable_textrelcheck}
     Symbol versioning                  : ${enable_symbol_versioning}
diff --git a/libelf/elf_compress.c b/libelf/elf_compress.c
index deb585b7..5a9d3da1 100644
--- a/libelf/elf_compress.c
+++ b/libelf/elf_compress.c
@@ -170,7 +170,7 @@ __libelf_compress_zlib (Elf_Scn *scn, size_t hsize, int ei_data,
   return out_buf;
 }
 
-#ifdef USE_ZSTD
+#ifdef USE_ZSTD_COMPRESS
 /* Cleanup and return result.  Don't leak memory.  */
 static void *
 do_zstd_cleanup (void *result, ZSTD_CCtx * const cctx, void *out_buf,
@@ -333,7 +333,7 @@ __libelf_compress (Elf_Scn *scn, size_t hsize, int ei_data,
 
   if (use_zstd)
     {
-#ifdef USE_ZSTD
+#ifdef USE_ZSTD_COMPRESS
       return __libelf_compress_zstd (scn, hsize, ei_data, orig_size,
 				   orig_addralign, new_size, force,
 				   data, next_data, out_buf, out_size,
@@ -406,8 +406,7 @@ __libelf_decompress_zlib (void *buf_in, size_t size_in, size_t size_out)
 }
 
 #ifdef USE_ZSTD
-void *
-internal_function
+static void *
 __libelf_decompress_zstd (void *buf_in, size_t size_in, size_t size_out)
 {
   /* Malloc might return NULL when requesting zero size.  This is highly
@@ -425,7 +424,7 @@ __libelf_decompress_zstd (void *buf_in, size_t size_in, size_t size_out)
   if (ZSTD_isError (ret))
     {
       free (buf_out);
-      __libelf_seterrno (ELF_E_UNKNOWN_COMPRESSION_TYPE);
+      __libelf_seterrno (ELF_E_DECOMPRESS_ERROR);
       return NULL;
     }
   else
@@ -444,7 +443,7 @@ __libelf_decompress (int chtype, void *buf_in, size_t size_in, size_t size_out)
 #ifdef USE_ZSTD
     return __libelf_decompress_zstd (buf_in, size_in, size_out);
 #else
-    __libelf_seterrno (ELF_E_DECOMPRESS_ERROR);
+    __libelf_seterrno (ELF_E_UNKNOWN_COMPRESSION_TYPE);
     return NULL;
 #endif
     }
diff --git a/src/elfcompress.c b/src/elfcompress.c
index bfdac2b4..1f32331c 100644
--- a/src/elfcompress.c
+++ b/src/elfcompress.c
@@ -230,7 +230,8 @@ compress_section (Elf_Scn *scn, size_t orig_size, const char *name,
     res = elf_compress (scn, dchtype, flags);
 
   if (res < 0)
-    error (0, 0, "Couldn't decompress section [%zd] %s: %s",
+    error (0, 0, "Couldn't %s section [%zd] %s: %s",
+	   compress ? "compress" : "decompress",
 	   ndx, name, elf_errmsg (-1));
   else
     {
diff --git a/tests/Makefile.am b/tests/Makefile.am
index cb70229e..71b19601 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -218,6 +218,9 @@ endif
 
 if HAVE_ZSTD
 TESTS += run-readelf-compressed-zstd.sh
+endif
+
+if USE_ZSTD_COMPRESS
 export ELFUTILS_ZSTD = 1
 endif
 

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2022-12-22 18:17 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20221221111547.74C4C385B50B@sourceware.org>
2022-12-21 14:20 ` ☠ Buildbot (GNU Toolchain): elfutils-try-debian-armhf - failed compile (failure) (users/marxin/try-zstd-support-v2) Martin Liška
2022-12-21 14:42   ` Mark Wielaard
2022-12-21 14:54     ` Frank Ch. Eigler
2022-12-21 15:21     ` Mark Wielaard
2022-12-21 18:29       ` Martin Liška
2022-12-21 23:38         ` Mark Wielaard
2022-12-22  9:09           ` Martin Liška
2022-12-22 18:17             ` Mark Wielaard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).