* [PATCH v3] binutils, gdb: support zstd compressed debug sections @ 2022-09-23 4:08 Fangrui Song 2022-09-23 14:32 ` Simon Marchi ` (4 more replies) 0 siblings, 5 replies; 25+ messages in thread From: Fangrui Song @ 2022-09-23 4:08 UTC (permalink / raw) To: Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi Cc: binutils, gdb-patches, Fangrui Song PR29397 PR29563: The new configure option --with-zstd defaults to auto. If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd compressed debug sections for most tools. * bfd: for addr2line, objdump --dwarf, gdb, etc * gas: support --compress-debug-sections=zstd * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd * objcopy: support ELFCOMPRESS_ZSTD input for --decompress-debug-sections and --compress-debug-sections=zstd * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd symbols, so gdb has to link against -lzstd in this patch. If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this is too heavyweight, so don't do it for now. ``` % ld/ld-new a.o ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... % ld/ld-new a.o --compress-debug-sections=zstd ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support % binutils/objcopy --compress-debug-sections=zstd a.o b.o binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support % binutils/objcopy b.o --decompress-debug-sections binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... ``` --- Changes from v1: * use PKG_CHECK_MODULES to check libzstd.pc Changes from v2: * Improve PKG_CHECK_MODULES, autoreconf -vf * objcopy: check --compress-debug-sections=zstd in a !HAVE_ZSTD build --- bfd/Makefile.am | 4 +- bfd/Makefile.in | 13 +- bfd/aclocal.m4 | 2 + bfd/bfd-in.h | 3 +- bfd/bfd-in2.h | 11 +- bfd/bfd.c | 26 +- bfd/compress.c | 72 +++- bfd/config.in | 3 + bfd/configure | 268 ++++++++++++- bfd/configure.ac | 3 +- bfd/elf.c | 12 + bfd/elfxx-target.h | 6 +- bfd/section.c | 3 +- binutils/Makefile.in | 5 +- binutils/aclocal.m4 | 1 + binutils/config.in | 3 + binutils/configure | 135 ++++++- binutils/configure.ac | 3 +- binutils/doc/binutils.texi | 16 +- binutils/objcopy.c | 19 +- binutils/testsuite/binutils-all/compress.exp | 44 +++ config/zstd.m4 | 24 ++ configure | 10 + configure.ac | 3 + gas/Makefile.am | 4 +- gas/Makefile.in | 13 +- gas/aclocal.m4 | 2 + gas/as.c | 13 +- gas/compress-debug.c | 60 ++- gas/compress-debug.h | 10 +- gas/config.in | 3 + gas/configure | 268 ++++++++++++- gas/configure.ac | 3 +- gas/doc/as.texi | 11 +- gas/write.c | 36 +- gdb/Makefile.in | 8 +- gdb/acinclude.m4 | 3 +- gdb/config.in | 3 + gdb/configure | 136 ++++++- gdb/configure.ac | 4 +- ld/Makefile.am | 5 +- ld/Makefile.in | 11 +- ld/aclocal.m4 | 1 + ld/config.in | 3 + ld/configure | 389 ++++++++++++------- ld/configure.ac | 7 +- ld/emultempl/elf.em | 9 + ld/ld.texi | 5 + ld/ldmain.c | 8 +- ld/lexsup.c | 4 +- ld/testsuite/ld-elf/compress.exp | 17 + 51 files changed, 1470 insertions(+), 255 deletions(-) create mode 100644 config/zstd.m4 diff --git a/bfd/Makefile.am b/bfd/Makefile.am index c23dff6cac3..cfe903c63f0 100644 --- a/bfd/Makefile.am +++ b/bfd/Makefile.am @@ -57,7 +57,7 @@ ZLIBINC = @zlibinc@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' @LARGEFILE_CPPFLAGS@ if PLUGINS bfdinclude_HEADERS += $(INCDIR)/plugin-api.h @@ -776,7 +776,7 @@ ofiles: stamp-ofiles ; @true libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) libbfd_la_LDFLAGS += -release `cat libtool-soversion` @SHARED_LDFLAGS@ # libtool will build .libs/libbfd.a. We create libbfd.a in the build diff --git a/bfd/Makefile.in b/bfd/Makefile.in index 82843d2d61d..6be41a72a59 100644 --- a/bfd/Makefile.in +++ b/bfd/Makefile.in @@ -122,10 +122,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../config/acx.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/bfd.m4 $(top_srcdir)/warning.m4 \ @@ -399,6 +401,9 @@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ PKGVERSION = @PKGVERSION@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ REPORT_BUGS_TEXI = @REPORT_BUGS_TEXI@ @@ -416,6 +421,8 @@ WARN_CFLAGS = @WARN_CFLAGS@ WARN_CFLAGS_FOR_BUILD = @WARN_CFLAGS_FOR_BUILD@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -520,7 +527,7 @@ libbfd_la_LDFLAGS = $(am__append_1) -release `cat libtool-soversion` \ # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' \ @LARGEFILE_CPPFLAGS@ @HDEFINES@ @COREFLAG@ @TDEFINES@ \ $(CSEARCH) $(CSWITCHES) $(HAVEVECS) @INCINTL@ @@ -1199,7 +1206,7 @@ OFILES = $(BFD_BACKENDS) $(BFD_MACHINES) @COREFILE@ @bfd64_libs@ libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) # libtool will build .libs/libbfd.a. We create libbfd.a in the build # directory so that we don't have to convert all the programs that use diff --git a/bfd/aclocal.m4 b/bfd/aclocal.m4 index 0f8aa1d4518..09b849dfc87 100644 --- a/bfd/aclocal.m4 +++ b/bfd/aclocal.m4 @@ -1176,10 +1176,12 @@ m4_include([../config/largefile.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/bfd/bfd-in.h b/bfd/bfd-in.h index 8605056aefe..4765ea80536 100644 --- a/bfd/bfd-in.h +++ b/bfd/bfd-in.h @@ -342,7 +342,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 79fcc4eb912..5c80956c79c 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -349,7 +349,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections @@ -962,7 +963,8 @@ typedef struct bfd_section unsigned int compress_status : 2; #define COMPRESS_SECTION_NONE 0 #define COMPRESS_SECTION_DONE 1 -#define DECOMPRESS_SECTION_SIZED 2 +#define DECOMPRESS_SECTION_ZLIB 2 +#define DECOMPRESS_SECTION_ZSTD 3 /* The following flags are used by the ELF linker. */ @@ -6637,12 +6639,14 @@ struct bfd #define BFD_ARCHIVE_FULL_PATH 0x100000 #define BFD_CLOSED_BY_CACHE 0x200000 + /* Compress sections in this BFD with SHF_COMPRESSED zstd. */ +#define BFD_COMPRESS_ZSTD 0x400000 /* Flags bits to be saved in bfd_preserve_save. */ #define BFD_FLAGS_SAVED \ (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ - | BFD_USE_ELF_STT_COMMON) + | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) /* Flags bits which are for BFD use only. */ #define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -7271,6 +7275,7 @@ void bfd_update_compression_header bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); diff --git a/bfd/bfd.c b/bfd/bfd.c index 0a21db11fd6..5f2033bee7a 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -177,12 +177,15 @@ CODE_FRAGMENT .#define BFD_ARCHIVE_FULL_PATH 0x100000 . .#define BFD_CLOSED_BY_CACHE 0x200000 + +. {* Compress sections in this BFD with SHF_COMPRESSED zstd. *} +.#define BFD_COMPRESS_ZSTD 0x400000 . . {* Flags bits to be saved in bfd_preserve_save. *} .#define BFD_FLAGS_SAVED \ . (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ . | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ -. | BFD_USE_ELF_STT_COMMON) +. | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) . . {* Flags bits which are for BFD use only. *} .#define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -2500,6 +2503,9 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, { const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data * esd = elf_section_data (sec); + const unsigned int ch_type = abfd->flags & BFD_COMPRESS_ZSTD + ? ELFCOMPRESS_ZSTD + : ELFCOMPRESS_ZLIB; /* Set the SHF_COMPRESSED bit. */ elf_section_flags (sec) |= SHF_COMPRESSED; @@ -2507,7 +2513,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, if (bed->s->elfclass == ELFCLASS32) { Elf32_External_Chdr *echdr = (Elf32_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, sec->size, &echdr->ch_size); bfd_put_32 (abfd, 1u << sec->alignment_power, &echdr->ch_addralign); @@ -2518,7 +2524,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, else { Elf64_External_Chdr *echdr = (Elf64_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, 0, &echdr->ch_reserved); bfd_put_64 (abfd, sec->size, &echdr->ch_size); bfd_put_64 (abfd, UINT64_C (1) << sec->alignment_power, @@ -2553,14 +2559,15 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, SYNOPSIS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); DESCRIPTION - Check the compression header at CONTENTS of SEC in ABFD and - store the uncompressed size in UNCOMPRESSED_SIZE and the - uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER - if the compression header is valid. + Check the compression header at CONTENTS of SEC in ABFD and store the + ch_type in CH_TYPE, uncompressed size in UNCOMPRESSED_SIZE, and the + uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER if the + compression header is valid. RETURNS Return TRUE if the compression header is valid. @@ -2569,6 +2576,7 @@ RETURNS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power) { @@ -2591,7 +2599,9 @@ bfd_check_compression_header (bfd *abfd, bfd_byte *contents, chdr.ch_size = bfd_get_64 (abfd, &echdr->ch_size); chdr.ch_addralign = bfd_get_64 (abfd, &echdr->ch_addralign); } - if (chdr.ch_type == ELFCOMPRESS_ZLIB + *ch_type = chdr.ch_type; + if ((chdr.ch_type == ELFCOMPRESS_ZLIB + || chdr.ch_type == ELFCOMPRESS_ZSTD) && chdr.ch_addralign == (chdr.ch_addralign & -chdr.ch_addralign)) { *uncompressed_size = chdr.ch_size; diff --git a/bfd/compress.c b/bfd/compress.c index b2e39826e38..95c1f4c42d1 100644 --- a/bfd/compress.c +++ b/bfd/compress.c @@ -20,18 +20,31 @@ #include "sysdep.h" #include <zlib.h> +#ifdef HAVE_ZSTD +#include <zstd.h> +#endif #include "bfd.h" +#include "elf-bfd.h" #include "libbfd.h" #include "safe-ctype.h" #define MAX_COMPRESSION_HEADER_SIZE 24 static bool -decompress_contents (bfd_byte *compressed_buffer, +decompress_contents (bool is_zstd, bfd_byte *compressed_buffer, bfd_size_type compressed_size, bfd_byte *uncompressed_buffer, bfd_size_type uncompressed_size) { + if (is_zstd) + { +#ifdef HAVE_ZSTD + size_t ret = ZSTD_decompress (uncompressed_buffer, uncompressed_size, + compressed_buffer, compressed_size); + return !ZSTD_isError (ret); +#endif + } + z_stream strm; int rc; @@ -69,7 +82,7 @@ decompress_contents (bfd_byte *compressed_buffer, } /* Compress data of the size specified in @var{uncompressed_size} - and pointed to by @var{uncompressed_buffer} using zlib and store + and pointed to by @var{uncompressed_buffer} using zlib/zstd and store as the contents field. This function assumes the contents field was allocated using bfd_malloc() or equivalent. @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, sec->size = orig_uncompressed_size; if (decompress) { - if (!decompress_contents (uncompressed_buffer - + orig_compression_header_size, - zlib_size, buffer, buffer_size)) + if (!decompress_contents ( + sec->compress_status == DECOMPRESS_SECTION_ZSTD, + uncompressed_buffer + orig_compression_header_size, + zlib_size, buffer, buffer_size)) { bfd_set_error (bfd_error_bad_value); bfd_release (abfd, buffer); @@ -175,10 +189,23 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, } else { - if (compress ((Bytef*) buffer + header_size, - &compressed_size, - (const Bytef*) uncompressed_buffer, - uncompressed_size) != Z_OK) + if (abfd->flags & BFD_COMPRESS_ZSTD) + { +#if HAVE_ZSTD + compressed_size = ZSTD_compress ( + buffer + header_size, compressed_size, uncompressed_buffer, + uncompressed_size, ZSTD_CLEVEL_DEFAULT); + if (ZSTD_isError (compressed_size)) + { + bfd_release (abfd, buffer); + bfd_set_error (bfd_error_bad_value); + return 0; + } +#endif + } + else if (compress ((Bytef *)buffer + header_size, &compressed_size, + (const Bytef *)uncompressed_buffer, uncompressed_size) + != Z_OK) { bfd_release (abfd, buffer); bfd_set_error (bfd_error_bad_value); @@ -237,6 +264,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) bfd_size_type save_rawsize; bfd_byte *compressed_buffer; unsigned int compression_header_size; + const unsigned int compress_status = sec->compress_status; if (abfd->direction != write_direction && sec->rawsize != 0) sz = sec->rawsize; @@ -248,7 +276,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) return true; } - switch (sec->compress_status) + switch (compress_status) { case COMPRESS_SECTION_NONE: if (p == NULL) @@ -298,7 +326,8 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) *ptr = p; return true; - case DECOMPRESS_SECTION_SIZED: + case DECOMPRESS_SECTION_ZLIB: + case DECOMPRESS_SECTION_ZSTD: /* Read in the full compressed section contents. */ compressed_buffer = (bfd_byte *) bfd_malloc (sec->compressed_size); if (compressed_buffer == NULL) @@ -316,7 +345,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Restore rawsize and size. */ sec->rawsize = save_rawsize; sec->size = save_size; - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = compress_status; if (!ret) goto fail_compressed; @@ -330,8 +359,10 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Set header size to the zlib header size if it is a SHF_COMPRESSED section. */ compression_header_size = 12; - if (!decompress_contents (compressed_buffer + compression_header_size, - sec->compressed_size - compression_header_size, p, sz)) + if (!decompress_contents (compress_status == DECOMPRESS_SECTION_ZSTD, + compressed_buffer + compression_header_size, + sec->compressed_size - compression_header_size, + p, sz)) { bfd_set_error (bfd_error_bad_value); if (p != *ptr) @@ -381,7 +412,8 @@ DESCRIPTION void bfd_cache_section_contents (asection *sec, void *contents) { - if (sec->compress_status == DECOMPRESS_SECTION_SIZED) + if (sec->compress_status == DECOMPRESS_SECTION_ZLIB + || sec->compress_status == DECOMPRESS_SECTION_ZSTD) sec->compress_status = COMPRESS_SECTION_DONE; sec->contents = contents; sec->flags |= SEC_IN_MEMORY; @@ -418,6 +450,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, int compression_header_size; int header_size; unsigned int saved = sec->compress_status; + unsigned int ch_type; bool compressed; *uncompressed_align_pow_p = 0; @@ -448,7 +481,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, { if (compression_header_size != 0) { - if (!bfd_check_compression_header (abfd, header, sec, + if (!bfd_check_compression_header (abfd, header, sec, &ch_type, uncompressed_size_p, uncompressed_align_pow_p)) compression_header_size = -1; @@ -507,7 +540,7 @@ SYNOPSIS DESCRIPTION Record compressed section size, update section size with decompressed size and set compress_status to - DECOMPRESS_SECTION_SIZED. + DECOMPRESS_SECTION_{ZLIB,ZSTD}. Return @code{FALSE} if the section is not a valid compressed section. Otherwise, return @code{TRUE}. @@ -521,6 +554,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) int header_size; bfd_size_type uncompressed_size; unsigned int uncompressed_alignment_power = 0; + unsigned int ch_type; z_stream strm; compression_header_size = bfd_get_compression_header_size (abfd, sec); @@ -550,6 +584,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) uncompressed_size = bfd_getb64 (header + 4); } else if (!bfd_check_compression_header (abfd, header, sec, + &ch_type, &uncompressed_size, &uncompressed_alignment_power)) { @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) sec->compressed_size = sec->size; sec->size = uncompressed_size; bfd_set_section_alignment (sec, uncompressed_alignment_power); - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD + : DECOMPRESS_SECTION_ZLIB; return true; } diff --git a/bfd/config.in b/bfd/config.in index f54a3cacbea..a59304e0a66 100644 --- a/bfd/config.in +++ b/bfd/config.in @@ -232,6 +232,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/bfd/configure b/bfd/configure index 075d2ee0a1b..d905a7bf7a8 100755 --- a/bfd/configure +++ b/bfd/configure @@ -652,6 +652,11 @@ TDEFINES SHARED_LIBADD SHARED_LDFLAGS LIBM +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir EXEEXT_FOR_BUILD @@ -839,6 +844,7 @@ enable_maintainer_mode enable_install_libbfd enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -848,7 +854,12 @@ CFLAGS LDFLAGS LIBS CPPFLAGS -CPP' +CPP +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1511,6 +1522,8 @@ Optional Packages: Binutils" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1521,6 +1534,13 @@ Some influential environment variables: CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> CPP C preprocessor + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -11086,7 +11106,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11089 "configure" +#line 11109 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11192,7 +11212,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11195 "configure" +#line 11215 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -12995,7 +13015,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. # Use the system's zlib library. @@ -13015,6 +13035,246 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" { $as_echo "$as_me:${as_lineno-$LINENO}: checking compiler support for hidden visibility" >&5 diff --git a/bfd/configure.ac b/bfd/configure.ac index 28f3d1afce6..d1b2f76b31a 100644 --- a/bfd/configure.ac +++ b/bfd/configure.ac @@ -230,9 +230,10 @@ AC_CHECK_DECLS([basename, ffs, stpcpy, asprintf, vasprintf, strnlen]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. AM_ZLIB +AM_ZSTD save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" diff --git a/bfd/elf.c b/bfd/elf.c index 16cea4f8aeb..e52fa458a5a 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -1260,6 +1260,18 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, abfd, name); return false; } +#ifndef HAVE_ZSTD + if (newsect->compress_status == DECOMPRESS_SECTION_ZSTD) + { + _bfd_error_handler + /* xgettext:c-format */ + (_ ("%pB: section %s is compressed with zstd, but BFD " + "is not built with zstd support"), + abfd, name); + newsect->compress_status = COMPRESS_SECTION_NONE; + return false; + } +#endif } if (abfd->is_linker_input) diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h index feaba84bf2e..ca600bb5ddf 100644 --- a/bfd/elfxx-target.h +++ b/bfd/elfxx-target.h @@ -989,7 +989,8 @@ const bfd_target TARGET_BIG_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY @@ -1093,7 +1094,8 @@ const bfd_target TARGET_LITTLE_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY diff --git a/bfd/section.c b/bfd/section.c index c7a02d729f2..614570e976e 100644 --- a/bfd/section.c +++ b/bfd/section.c @@ -392,7 +392,8 @@ CODE_FRAGMENT . unsigned int compress_status : 2; .#define COMPRESS_SECTION_NONE 0 .#define COMPRESS_SECTION_DONE 1 -.#define DECOMPRESS_SECTION_SIZED 2 +.#define DECOMPRESS_SECTION_ZLIB 2 +.#define DECOMPRESS_SECTION_ZSTD 3 . . {* The following flags are used by the ELF linker. *} . diff --git a/binutils/Makefile.in b/binutils/Makefile.in index 78d32b350e3..6de4e239408 100644 --- a/binutils/Makefile.in +++ b/binutils/Makefile.in @@ -156,7 +156,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 \ @@ -575,6 +576,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ diff --git a/binutils/aclocal.m4 b/binutils/aclocal.m4 index a877fa7f873..28271f56279 100644 --- a/binutils/aclocal.m4 +++ b/binutils/aclocal.m4 @@ -1205,6 +1205,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/binutils/config.in b/binutils/config.in index c5fb919aa95..bee8c07e2f7 100644 --- a/binutils/config.in +++ b/binutils/config.in @@ -157,6 +157,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define as const if the declaration of iconv() needs const. */ #undef ICONV_CONST diff --git a/binutils/configure b/binutils/configure index 1c518227f57..54d0f184da1 100755 --- a/binutils/configure +++ b/binutils/configure @@ -650,6 +650,8 @@ LTLIBICONV LIBICONV MSGPACK_LIBS MSGPACK_CFLAGS +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir DEMANGLER_NAME @@ -832,6 +834,7 @@ enable_build_warnings enable_nls enable_maintainer_mode with_system_zlib +with_zstd with_msgpack enable_rpath with_libiconv_prefix @@ -853,6 +856,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS MSGPACK_CFLAGS MSGPACK_LIBS' @@ -1517,6 +1522,8 @@ Optional Packages: --with-debuginfod Enable debuginfo lookups with debuginfod (auto/yes/no) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-msgpack Enable msgpack support (auto/yes/no) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib @@ -1547,6 +1554,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config MSGPACK_CFLAGS C compiler flags for MSGPACK, overriding pkg-config MSGPACK_LIBS @@ -10804,7 +10813,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10807 "configure" +#line 10816 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10910,7 +10919,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10913 "configure" +#line 10922 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13468,7 +13477,7 @@ cat >>confdefs.h <<_ACEOF _ACEOF -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). @@ -13490,6 +13499,126 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + + case "${host}" in *-*-msdos* | *-*-go32* | *-*-mingw32* | *-*-cygwin* | *-*-windows*) diff --git a/binutils/configure.ac b/binutils/configure.ac index ec002d3f88f..c152b7f6733 100644 --- a/binutils/configure.ac +++ b/binutils/configure.ac @@ -265,10 +265,11 @@ fi AC_CHECK_DECLS([asprintf, environ, getc_unlocked, stpcpy, strnlen]) -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). AM_ZLIB +AM_ZSTD BFD_BINARY_FOPEN diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi index 1499db5728c..34a4164eb94 100644 --- a/binutils/doc/binutils.texi +++ b/binutils/doc/binutils.texi @@ -2159,21 +2159,23 @@ ELF ABI. Note - if compression would actually make a section @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd For ELF files, these options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--decompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug} instead of @samp{.debug}. Note - if compression would -actually make a section @emph{larger}, then it is not compressed nor -renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @item --decompress-debug-sections -Decompress DWARF debug sections using zlib. The original section -names of the compressed sections are restored. +Decompress DWARF debug sections. For a @samp{.zdebug} section, the original +name is restored. @item --elf-stt-common=yes @itemx --elf-stt-common=no diff --git a/binutils/objcopy.c b/binutils/objcopy.c index 43261756a42..fc668f00bbc 100644 --- a/binutils/objcopy.c +++ b/binutils/objcopy.c @@ -232,7 +232,8 @@ static enum compress_zlib = compress | 1 << 1, compress_gnu_zlib = compress | 1 << 2, compress_gabi_zlib = compress | 1 << 3, - decompress = 1 << 4 + compress_zstd = compress | 1 << 4, + decompress = 1 << 5 } do_debug_sections = nothing; /* Whether to generate ELF common symbols with the STT_COMMON type. */ @@ -678,8 +679,8 @@ copy_usage (FILE *stream, int exit_status) <commit>\n\ --subsystem <name>[:<version>]\n\ Set PE subsystem to <name> [& <version>]\n\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ - Compress DWARF debug sections using zlib\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ + Compress DWARF debug sections\n\ --decompress-debug-sections Decompress DWARF debug sections using zlib\n\ --elf-stt-common=[yes|no] Generate ELF common symbols with STT_COMMON\n\ type\n\ @@ -2659,7 +2660,8 @@ copy_object (bfd *ibfd, bfd *obfd, const bfd_arch_info_type *input_arch) if ((do_debug_sections & compress) != 0 && do_debug_sections != compress) { - non_fatal (_("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi] is unsupported on `%s'"), + non_fatal (_ ("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi|" + "zstd] is unsupported on `%s'"), bfd_get_archive_filename (ibfd)); return false; } @@ -3807,6 +3809,13 @@ copy_file (const char *input_filename, const char *output_filename, int ofd, if (do_debug_sections != compress_gnu_zlib) ibfd->flags |= BFD_COMPRESS_GABI; break; + case compress_zstd: + ibfd->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; +#ifndef HAVE_ZSTD + fatal (_ ("--compress-debug-sections=zstd: binutils is not built with " + "zstd support")); +#endif + break; case decompress: ibfd->flags |= BFD_DECOMPRESS; break; @@ -5469,6 +5478,8 @@ copy_main (int argc, char *argv[]) do_debug_sections = compress_gnu_zlib; else if (strcasecmp (optarg, "zlib-gabi") == 0) do_debug_sections = compress_gabi_zlib; + else if (strcasecmp (optarg, "zstd") == 0) + do_debug_sections = compress_zstd; else fatal (_("unrecognized --compress-debug-sections type `%s'"), optarg); diff --git a/binutils/testsuite/binutils-all/compress.exp b/binutils/testsuite/binutils-all/compress.exp index c88da74a987..4e74c824664 100644 --- a/binutils/testsuite/binutils-all/compress.exp +++ b/binutils/testsuite/binutils-all/compress.exp @@ -576,6 +576,50 @@ if { [regexp_diff objdump.out $srcdir/$subdir/dw2-3gabi.W] } then { pass "$testname" } +if { [binutils_assemble_flags $srcdir/$subdir/dw2-1.S ${compressedfile}zstd.o --compress-debug-sections=zstd] } then { + set testname "objcopy compress debug sections with zstd" + set got [binutils_run $OBJCOPY "--compress-debug-sections=zstd ${testfile}.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${compressedfile}zstd.o ${copyfile}zstd.o\n" + verbose "cmp ${compressedfile}zstd.o ${copyfile}zstd.o" + set src1 ${compressedfile}zstd.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } + + set testname "objcopy decompress compressed debug sections with zstd" + set got [binutils_run $OBJCOPY "--decompress-debug-sections ${compressedfile}zstd.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${testfile}.o ${copyfile}zstd.o\n" + verbose "cmp ${testfile}.o ${copyfile}zstd.o" + set src1 ${testfile}.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } +} + proc convert_test { testname as_flags objcop_flags } { global srcdir global subdir diff --git a/config/zstd.m4 b/config/zstd.m4 new file mode 100644 index 00000000000..93ae17b89d3 --- /dev/null +++ b/config/zstd.m4 @@ -0,0 +1,24 @@ +dnl Copyright (C) 2022 Free Software Foundation, Inc. +dnl This file is free software, distributed under the terms of the GNU +dnl General Public License. As a special exception to the GNU General +dnl Public License, this file may be distributed as part of a program +dnl that contains a configuration script generated by Autoconf, under +dnl the same distribution terms as the rest of that program. + +dnl Enable features using the zstd library. +AC_DEFUN([AM_ZSTD], +[ +AC_ARG_WITH(zstd, + [AS_HELP_STRING([--with-zstd], [support zstd compressed debug sections (default=auto)])], + [], [with_zstd=auto]) + +if test "$with_zstd" != no; then + PKG_CHECK_MODULES(ZSTD, [libzstd], [ + AC_DEFINE(HAVE_ZSTD, 1, [Define to 1 if zstd is enabled.]) + ], [ + if test "$with_zstd" = yes; then + AC_MSG_ERROR([--with-zstd was given, but pkgconfig/libzstd.pc is not found]) + fi + ]) +fi +]) diff --git a/configure b/configure index d75f47a1e95..f14e0efd675 100755 --- a/configure +++ b/configure @@ -785,6 +785,7 @@ ac_user_opts=' enable_option_checking with_build_libsubdir with_system_zlib +with_zstd enable_as_accelerator_for enable_offload_targets enable_gold @@ -1567,6 +1568,8 @@ Optional Packages: --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-build-libsubdir=DIR Directory where to find libraries for build system --with-system-zlib use installed libz + --with-zstd Support zstd compressed debug sections + (default=auto) --with-mpc=PATH specify prefix directory for installed MPC package. Equivalent to --with-mpc-include=PATH/include plus --with-mpc-lib=PATH/lib @@ -2925,6 +2928,13 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +fi + + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/configure.ac b/configure.ac index ae18d436aca..0152c69292e 100644 --- a/configure.ac +++ b/configure.ac @@ -246,6 +246,9 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi +AC_ARG_WITH(zstd, +[AS_HELP_STRING([--with-zstd], [Support zstd compressed debug sections (default=auto)])]) + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/gas/Makefile.am b/gas/Makefile.am index bd597398671..5f0f24abf8d 100644 --- a/gas/Makefile.am +++ b/gas/Makefile.am @@ -42,7 +42,7 @@ am__skipyacc = WARN_CFLAGS = @WARN_CFLAGS@ @WARN_WRITE_STRINGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c @@ -407,7 +407,7 @@ STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) EXTRA_as_new_SOURCES = $(CFILES) $(HFILES) $(TARGET_CPU_CFILES) \ diff --git a/gas/Makefile.in b/gas/Makefile.in index c57d78f82c4..5a4dd702252 100644 --- a/gas/Makefile.in +++ b/gas/Makefile.in @@ -140,10 +140,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/../bfd/version.m4 \ @@ -429,6 +431,9 @@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ SED = @SED@ @@ -443,6 +448,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ] ; then echo ../bison/bison -y -L../bison/bison ; else echo @YACC@ ; fi` YFLAGS = @YFLAGS@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -524,7 +531,7 @@ ZLIBINC = @zlibinc@ # maintainer mode is disabled. Avoid this. am__skiplex = am__skipyacc = -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c TARG_CPU_O = config/tc-@target_cpu_type@.@OBJEXT@ @@ -874,7 +881,7 @@ GASLIBS = @OPCODES_LIB@ ../bfd/libbfd.la ../libiberty/libiberty.a STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 index 70183124da7..722030c776f 100644 --- a/gas/aclocal.m4 +++ b/gas/aclocal.m4 @@ -1196,10 +1196,12 @@ m4_include([../config/lcmessage.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/gas/as.c b/gas/as.c index 6268779cf90..35ad6b3ab3b 100644 --- a/gas/as.c +++ b/gas/as.c @@ -252,14 +252,14 @@ Options:\n\ --alternate initially turn on alternate macro syntax\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib [default]\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ don't compress DWARF debug sections\n")); #else fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ @@ -736,6 +736,15 @@ This program has absolutely no warranty.\n")); flag_compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + flag_compress_debug = COMPRESS_DEBUG_ZSTD; +#else + as_fatal (_ ("--compress-debug-sections=zstd: gas is not " + "built with zstd support")); +#endif + } else as_fatal (_("Invalid --compress-debug-sections option: `%s'"), optarg); diff --git a/gas/compress-debug.c b/gas/compress-debug.c index c80dbeec4e7..3cd175f6e57 100644 --- a/gas/compress-debug.c +++ b/gas/compress-debug.c @@ -21,14 +21,23 @@ #include "config.h" #include <stdio.h> #include <zlib.h> +#if HAVE_ZSTD +#include <zstd.h> +#endif #include "ansidecl.h" #include "compress-debug.h" /* Initialize the compression engine. */ -struct z_stream_s * -compress_init (void) +void * +compress_init (bool use_zstd) { + if (use_zstd) { +#if HAVE_ZSTD + return ZSTD_createCCtx (); +#endif + } + static struct z_stream_s strm; strm.zalloc = NULL; @@ -42,22 +51,37 @@ compress_init (void) from the engine goes into the current frag on the obstack. */ int -compress_data (struct z_stream_s *strm, const char **next_in, - int *avail_in, char **next_out, int *avail_out) +compress_data (bool use_zstd, void *ctx, const char **next_in, int *avail_in, + char **next_out, int *avail_out) { - int out_size = 0; - int x; + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { *next_in, *avail_in, 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_continue); + *next_in += ib.pos; + *avail_in -= ib.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + return (int)ob.pos; +#endif + } + + struct z_stream_s *strm = ctx; strm->next_in = (Bytef *) (*next_in); strm->avail_in = *avail_in; strm->next_out = (Bytef *) (*next_out); strm->avail_out = *avail_out; - x = deflate (strm, Z_NO_FLUSH); + int x = deflate (strm, Z_NO_FLUSH); if (x != Z_OK) return -1; - out_size = *avail_out - strm->avail_out; + int out_size = *avail_out - strm->avail_out; *next_in = (char *) (strm->next_in); *avail_in = strm->avail_in; *next_out = (char *) (strm->next_out); @@ -71,10 +95,28 @@ compress_data (struct z_stream_s *strm, const char **next_in, needed. */ int -compress_finish (struct z_stream_s *strm, char **next_out, +compress_finish (bool use_zstd, void *ctx, char **next_out, int *avail_out, int *out_size) { + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_end); + *out_size = ob.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + if (ret == 0) + ZSTD_freeCCtx (ctx); + return ret ? 1 : 0; +#endif + } + int x; + struct z_stream_s *strm = ctx; strm->avail_in = 0; strm->next_out = (Bytef *) (*next_out); diff --git a/gas/compress-debug.h b/gas/compress-debug.h index 0183e269fef..87de0f8444e 100644 --- a/gas/compress-debug.h +++ b/gas/compress-debug.h @@ -21,19 +21,19 @@ #ifndef COMPRESS_DEBUG_H #define COMPRESS_DEBUG_H +#include <stdbool.h> + struct z_stream_s; /* Initialize the compression engine. */ -extern struct z_stream_s * -compress_init (void); +extern void *compress_init (bool); /* Stream the contents of a frag to the compression engine. Output from the engine goes into the current frag on the obstack. */ -extern int -compress_data (struct z_stream_s *, const char **, int *, char **, int *); +extern int compress_data (bool, void *, const char **, int *, char **, int *); /* Finish the compression and consume the remaining compressed output. */ extern int -compress_finish (struct z_stream_s *, char **, int *, int *); +compress_finish (bool, void *, char **, int *, int *); #endif /* COMPRESS_DEBUG_H */ diff --git a/gas/config.in b/gas/config.in index e243fd277ee..0d1668a3eac 100644 --- a/gas/config.in +++ b/gas/config.in @@ -134,6 +134,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Using i386 COFF? */ #undef I386COFF diff --git a/gas/configure b/gas/configure index d0449a1d7ab..1833adffaa8 100755 --- a/gas/configure +++ b/gas/configure @@ -633,6 +633,11 @@ ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS LIBOBJS +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir LIBM @@ -817,6 +822,7 @@ with_cpu enable_nls enable_maintainer_mode with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -828,7 +834,12 @@ LIBS CPPFLAGS CPP YACC -YFLAGS' +YFLAGS +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1493,6 +1504,8 @@ Optional Packages: --with-cpu=CPU default cpu variant is CPU (currently only supported on ARC) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1509,6 +1522,13 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -10702,7 +10722,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10705 "configure" +#line 10725 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10808,7 +10828,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10811 "configure" +#line 10831 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13945,7 +13965,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -13964,6 +13984,246 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + # Support for VMS timestamps via cross compile if test "$ac_cv_header_time_h" = yes; then diff --git a/gas/configure.ac b/gas/configure.ac index 21795a82bdd..8750ea21381 100644 --- a/gas/configure.ac +++ b/gas/configure.ac @@ -1004,8 +1004,9 @@ AC_CHECK_DECLS([asprintf, mempcpy, stpcpy]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. AM_ZLIB +AM_ZSTD # Support for VMS timestamps via cross compile diff --git a/gas/doc/as.texi b/gas/doc/as.texi index e915893da86..01f49434021 100644 --- a/gas/doc/as.texi +++ b/gas/doc/as.texi @@ -711,16 +711,19 @@ given section @emph{larger} then it is not compressed. @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd These options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--nocompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug}. Note if compression would make a given section -@emph{larger} then it is not compressed nor renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @end ifset diff --git a/gas/write.c b/gas/write.c index f76bbdb706e..0e49df7c03f 100644 --- a/gas/write.c +++ b/gas/write.c @@ -1413,7 +1413,7 @@ write_relocs (bfd *abfd ATTRIBUTE_UNUSED, asection *sec, } static int -compress_frag (struct z_stream_s *strm, const char *contents, int in_size, +compress_frag (bool use_zstd, void *ctx, const char *contents, int in_size, fragS **last_newf, struct obstack *ob) { int out_size; @@ -1442,10 +1442,10 @@ compress_frag (struct z_stream_s *strm, const char *contents, int in_size, as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - out_size = compress_data (strm, &contents, &in_size, - &next_out, &avail_out); + out_size = compress_data (use_zstd, ctx, &contents, &in_size, &next_out, + &avail_out); if (out_size < 0) - return -1; + return -1; f->fr_fix += out_size; total_out_size += out_size; @@ -1471,7 +1471,6 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) const char *section_name; char *compressed_name; char *header; - struct z_stream_s *strm; int x; flagword flags = bfd_section_flags (sec); unsigned int header_size, compression_header_size; @@ -1485,21 +1484,22 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) if (!startswith (section_name, ".debug_")) return; - strm = compress_init (); - if (strm == NULL) + bool use_zstd = abfd->flags & BFD_COMPRESS_ZSTD; + void *ctx = compress_init (use_zstd); + if (ctx == NULL) return; - if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) + if (flag_compress_debug == COMPRESS_DEBUG_GNU_ZLIB) + { + compression_header_size = 0; + header_size = 12; + } + else { compression_header_size = bfd_get_compression_header_size (stdoutput, NULL); header_size = compression_header_size; } - else - { - compression_header_size = 0; - header_size = 12; - } /* Create a new frag to contain the compression header. */ first_newf = frag_alloc (ob); @@ -1531,7 +1531,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) gas_assert (f->fr_type == rs_fill); if (f->fr_fix) { - out_size = compress_frag (strm, f->fr_literal, f->fr_fix, + out_size = compress_frag (use_zstd, ctx, f->fr_literal, f->fr_fix, &last_newf, ob); if (out_size < 0) return; @@ -1545,8 +1545,8 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) { while (count--) { - out_size = compress_frag (strm, fill_literal, (int) fill_size, - &last_newf, ob); + out_size = compress_frag (use_zstd, ctx, fill_literal, + (int)fill_size, &last_newf, ob); if (out_size < 0) return; compressed_size += out_size; @@ -1579,7 +1579,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - x = compress_finish (strm, &next_out, &avail_out, &out_size); + x = compress_finish (use_zstd, ctx, &next_out, &avail_out, &out_size); if (x < 0) return; @@ -2540,6 +2540,8 @@ write_object_file (void) { if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI; + else if (flag_compress_debug == COMPRESS_DEBUG_ZSTD) + stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; else stdoutput->flags |= BFD_COMPRESS; bfd_map_over_sections (stdoutput, compress_debug, (char *) 0); diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 2598b81d205..c528ee5aa80 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -173,6 +173,9 @@ BFD_CFLAGS = -I$(BFD_DIR) -I$(BFD_SRC) ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ + # Where is the decnumber library? Typically in ../libdecnumber. LIBDECNUMBER_DIR = ../libdecnumber LIBDECNUMBER = $(LIBDECNUMBER_DIR)/libdecnumber.a @@ -625,7 +628,7 @@ INTERNAL_CPPFLAGS = $(CPPFLAGS) @GUILE_CPPFLAGS@ @PYTHON_CPPFLAGS@ \ INTERNAL_CFLAGS_BASE = \ $(GLOBAL_CFLAGS) $(PROFILE_CFLAGS) \ $(GDB_CFLAGS) $(OPCODES_CFLAGS) $(READLINE_CFLAGS) $(ZLIBINC) \ - $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ + $(ZSTD_CFLAGS) $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ $(INTL_CFLAGS) $(INCGNU) $(INCSUPPORT) $(LIBBACKTRACE_INC) \ $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS) $(SRCHIGH_CFLAGS) \ $(TOP_CFLAGS) $(PTHREAD_CFLAGS) $(DEBUGINFOD_CFLAGS) @@ -647,7 +650,7 @@ INTERNAL_LDFLAGS = \ # Libraries and corresponding dependencies for compiling gdb. # XM_CLIBS, defined in *config files, have host-dependent libs. # LIBIBERTY appears twice on purpose. -CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \ +CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) $(ZSTD_LIBS) \ $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \ $(XM_CLIBS) $(GDBTKLIBS) $(LIBBACKTRACE_LIB) \ @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \ @@ -2298,6 +2301,7 @@ aclocal_m4_deps = \ ../config/lcmessage.m4 \ ../config/codeset.m4 \ ../config/zlib.m4 \ + ../config/zstd.m4 \ ../config/ax_pthread.m4 $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 index 95ff2b6f35e..28846119dcb 100644 --- a/gdb/acinclude.m4 +++ b/gdb/acinclude.m4 @@ -43,6 +43,7 @@ m4_include([../config/lib-link.m4]) m4_include([../config/iconv.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../gdbsupport/common.m4]) @@ -233,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" AC_CACHE_CHECK( diff --git a/gdb/config.in b/gdb/config.in index a4975c68aad..e13a409ec2d 100644 --- a/gdb/config.in +++ b/gdb/config.in @@ -589,6 +589,9 @@ /* Define to 1 if you have the `XML_StopParser' function. */ #undef HAVE_XML_STOPPARSER +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to 1 if your system has the _etext variable. */ #undef HAVE__ETEXT diff --git a/gdb/configure b/gdb/configure index 4dbd0c3b13c..57cd165088a 100755 --- a/gdb/configure +++ b/gdb/configure @@ -747,6 +747,8 @@ READLINE_DEPS READLINE LTLIBICONV LIBICONV +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir MIG @@ -893,6 +895,7 @@ enable_codesign with_pkgversion with_bugurl with_system_zlib +with_zstd with_gnu_ld enable_rpath with_libiconv_prefix @@ -961,6 +964,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS XMKMF' ac_subdirs_all='testsuite gdbtk' @@ -1641,6 +1646,8 @@ Optional Packages: --with-pkgversion=PKG Use PKG in the version string in place of "GDB" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib --without-libiconv-prefix don't search for libiconv in includedir and libdir @@ -1721,6 +1728,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config XMKMF Path to xmkmf, Makefile generator for X Window System Use these variables to override the choices made by `configure' or to help @@ -8242,7 +8251,8 @@ if test "$ac_res" != no; then : fi -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -8262,6 +8272,126 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + + if test "X$prefix" = "XNONE"; then acl_final_prefix="$ac_default_prefix" else @@ -17281,7 +17411,7 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 @@ -17396,7 +17526,7 @@ fi # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 diff --git a/gdb/configure.ac b/gdb/configure.ac index 215a91c5b27..e8cf808d280 100644 --- a/gdb/configure.ac +++ b/gdb/configure.ac @@ -460,8 +460,10 @@ AC_SEARCH_LIBS(gethostbyname, nsl) # Some systems (e.g. Solaris) have `socketpair' in libsocket. AC_SEARCH_LIBS(socketpair, socket) -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. AM_ZLIB +AM_ZSTD AM_ICONV diff --git a/ld/Makefile.am b/ld/Makefile.am index d31021c13e2..bbe25f3aca2 100644 --- a/ld/Makefile.am +++ b/ld/Makefile.am @@ -45,7 +45,7 @@ ELF_CLFAGS=-DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -959,7 +959,8 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ldbuildid.c ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) # Dependency tracking for the generated emulation files. EXTRA_ld_new_SOURCES += $(ALL_EMULATION_SOURCES) $(ALL_64_EMULATION_SOURCES) diff --git a/ld/Makefile.in b/ld/Makefile.in index ee0c98f65b0..400eed2717e 100644 --- a/ld/Makefile.in +++ b/ld/Makefile.in @@ -126,7 +126,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 $(top_srcdir)/configure.ac @@ -477,6 +478,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -564,7 +567,7 @@ ELF_CLFAGS = -DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -1011,7 +1014,9 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) + # # # Build a dummy plugin using libtool. diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 index d20c542eed5..893e973e4f2 100644 --- a/ld/aclocal.m4 +++ b/ld/aclocal.m4 @@ -1203,6 +1203,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/ld/config.in b/ld/config.in index 0ccd79d59cd..3916740eee4 100644 --- a/ld/config.in +++ b/ld/config.in @@ -162,6 +162,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/ld/configure b/ld/configure index a1a07005400..175dd995b2b 100755 --- a/ld/configure +++ b/ld/configure @@ -646,6 +646,8 @@ elf_plt_unwind_list_options elf_shlib_list_options elf_list_options STRINGIFY +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir NATIVE_LIB_DIRS @@ -679,9 +681,6 @@ WARN_CFLAGS_FOR_BUILD WARN_CFLAGS JANSSON_LIBS JANSSON_CFLAGS -PKG_CONFIG_LIBDIR -PKG_CONFIG_PATH -PKG_CONFIG enable_libctf ENABLE_LIBCTF_FALSE ENABLE_LIBCTF_TRUE @@ -711,6 +710,9 @@ LD FGREP SED LIBTOOL +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG EGREP CPP GREP @@ -855,6 +857,7 @@ enable_werror enable_build_warnings enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -868,14 +871,16 @@ CXX CXXFLAGS CCC CPP -CXXCPP PKG_CONFIG PKG_CONFIG_PATH PKG_CONFIG_LIBDIR +CXXCPP JANSSON_CFLAGS JANSSON_LIBS YACC -YFLAGS' +YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1552,6 +1557,8 @@ Optional Packages: --with-lib-path=dir1:dir2... set default LIB_PATH --with-sysroot=DIR Search for usr/lib et al within DIR. --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1564,12 +1571,12 @@ Some influential environment variables: CXX C++ compiler command CXXFLAGS C++ compiler flags CPP C preprocessor - CXXCPP C++ preprocessor PKG_CONFIG path to pkg-config utility PKG_CONFIG_PATH directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path + CXXCPP C++ preprocessor JANSSON_CFLAGS C compiler flags for JANSSON, overriding pkg-config JANSSON_LIBS @@ -1580,6 +1587,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -5388,6 +5397,126 @@ $as_echo "$ac_cv_safe_to_define___extensions__" >&6; } + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + case `pwd` in *\ * | *\ *) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 @@ -11491,7 +11620,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11494 "configure" +#line 11623 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11597,7 +11726,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11600 "configure" +#line 11729 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -15586,126 +15715,6 @@ fi if test "x$enable_jansson" != "xno"; then - - - - - - - -if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. -set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -PKG_CONFIG=$ac_cv_path_PKG_CONFIG -if test -n "$PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 -$as_echo "$PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_path_PKG_CONFIG"; then - ac_pt_PKG_CONFIG=$PKG_CONFIG - # Extract the first word of "pkg-config", so it can be a program name with args. -set dummy pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $ac_pt_PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG -if test -n "$ac_pt_PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 -$as_echo "$ac_pt_PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_pt_PKG_CONFIG" = x; then - PKG_CONFIG="" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - PKG_CONFIG=$ac_pt_PKG_CONFIG - fi -else - PKG_CONFIG="$ac_cv_path_PKG_CONFIG" -fi - -fi -if test -n "$PKG_CONFIG"; then - _pkg_min_version=0.9.0 - { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 -$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } - if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - PKG_CONFIG="" - fi -fi if test -n "$PKG_CONFIG"; then : @@ -17052,8 +17061,8 @@ $as_echo "#define HAVE_DECL_GETOPT 1" >>confdefs.h fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -17072,6 +17081,126 @@ fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string # concatenation, or ostring.sed otherwise. This is to support the diff --git a/ld/configure.ac b/ld/configure.ac index eb55904c090..a491f9f1116 100644 --- a/ld/configure.ac +++ b/ld/configure.ac @@ -34,6 +34,7 @@ AC_PROG_GREP AC_GNU_SOURCE AC_USE_SYSTEM_EXTENSIONS AC_PROG_INSTALL +PKG_PROG_PKG_CONFIG LT_INIT ACX_LARGEFILE @@ -297,7 +298,6 @@ AC_ARG_ENABLE([jansson], [enable_jansson="no"]) if test "x$enable_jansson" != "xno"; then - PKG_PROG_PKG_CONFIG AS_IF([test -n "$PKG_CONFIG"], [ PKG_CHECK_MODULES(JANSSON, [jansson], @@ -386,9 +386,10 @@ if test $ld_cv_decl_getopt_unistd_h = yes; then [Is the prototype for getopt in <unistd.h> in the expected format?]) fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. AM_ZLIB +AM_ZSTD # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string diff --git a/ld/emultempl/elf.em b/ld/emultempl/elf.em index c52484f35d3..acd66f907d1 100644 --- a/ld/emultempl/elf.em +++ b/ld/emultempl/elf.em @@ -668,6 +668,15 @@ gld${EMULATION_NAME}_handle_option (int optc) link_info.compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + link_info.compress_debug = COMPRESS_DEBUG_ZSTD; +#else + einfo (_ ("%F%P: --compress-debug-sections=zstd: ld is not built " + "with zstd support\n")); +#endif + } else einfo (_("%F%P: invalid --compress-debug-sections option: \`%s'\n"), optarg); diff --git a/ld/ld.texi b/ld/ld.texi index eabbec8faa9..9daed2e7e9f 100644 --- a/ld/ld.texi +++ b/ld/ld.texi @@ -2863,10 +2863,12 @@ but for most Linux based systems it will be @code{both}. @kindex --compress-debug-sections=zlib @kindex --compress-debug-sections=zlib-gnu @kindex --compress-debug-sections=zlib-gabi +@kindex --compress-debug-sections=zstd @item --compress-debug-sections=none @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd On ELF platforms, these options control how DWARF debug sections are compressed using zlib. @@ -2880,6 +2882,9 @@ sets the SHF_COMPRESSED flag in the sections' headers. The @option{--compress-debug-sections=zlib} option is an alias for @option{--compress-debug-sections=zlib-gabi}. +@option{--compress-debug-sections=zstd} compresses DWARF debug sections using +zstd. + Note that this option overrides any compression in input debug sections, so if a binary is linked with @option{--compress-debug-sections=none} for example, then any compressed debug sections in input files will be diff --git a/ld/ldmain.c b/ld/ldmain.c index 1bbddaaad32..71af5935b74 100644 --- a/ld/ldmain.c +++ b/ld/ldmain.c @@ -506,8 +506,12 @@ main (int argc, char **argv) if ((link_info.compress_debug & COMPRESS_DEBUG)) { link_info.output_bfd->flags |= BFD_COMPRESS; - if (link_info.compress_debug == COMPRESS_DEBUG_GABI_ZLIB) - link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug != COMPRESS_DEBUG_GNU_ZLIB) + { + link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug == COMPRESS_DEBUG_ZSTD) + link_info.output_bfd->flags |= BFD_COMPRESS_ZSTD; + } } ldwrite (); diff --git a/ld/lexsup.c b/ld/lexsup.c index 9225f71b3ce..299371fb775 100644 --- a/ld/lexsup.c +++ b/ld/lexsup.c @@ -2146,8 +2146,8 @@ elf_static_list_options (FILE *file) fprintf (file, _("\ --package-metadata[=JSON] Generate package metadata note\n")); fprintf (file, _("\ - --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ - Compress DWARF debug sections using zlib\n")); + --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi|zstd]\n\ + Compress DWARF debug sections\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (file, _("\ Default: zlib-gabi\n")); diff --git a/ld/testsuite/ld-elf/compress.exp b/ld/testsuite/ld-elf/compress.exp index cfdd767b8ab..7b8a31cc41d 100644 --- a/ld/testsuite/ld-elf/compress.exp +++ b/ld/testsuite/ld-elf/compress.exp @@ -254,3 +254,20 @@ if { [regexp_diff tmpdir/$test.out $srcdir/$subdir/$test.rt] } then { } else { pass "$test_name" } + +if { ![ld_assemble $as "--compress-debug-sections=zstd $srcdir/$subdir/empty.s" tmpdir/emptyzstd.o ] } { + return +} +set build_tests { + {"Build libzstdfoo.so with zstd compressed debug sections" + "-shared" "-fPIC -g -Wa,--compress-debug-sections=zstd -Wl,--compress-debug-sections=zstd" + {foo.c} {} "libzstdfoo.so"} +} +set run_tests { + {"Run zstdnormal with libzstdfoo.so with zstd compressed debug sections" + "tmpdir/begin.o tmpdir/libzstdfoo.so tmpdir/end.o -Wl,--compress-debug-sections=zstd" "" + {main.c} "zstdnormal" "normal.out" "-Wa,--compress-debug-sections=zstd"} +} + +run_cc_link_tests $build_tests +run_ld_link_exec_tests $run_tests -- 2.37.3.998.g577e59143f-goog ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song @ 2022-09-23 14:32 ` Simon Marchi 2022-09-26 5:12 ` Alan Modra 2022-09-23 15:45 ` Nick Clifton ` (3 subsequent siblings) 4 siblings, 1 reply; 25+ messages in thread From: Simon Marchi @ 2022-09-23 14:32 UTC (permalink / raw) To: Fangrui Song, Alan Modra, Jan Beulich, Nick Clifton; +Cc: binutils, gdb-patches On 2022-09-23 00:08, Fangrui Song wrote: > PR29397 PR29563: The new configure option --with-zstd defaults to auto. > If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd > compressed debug sections for most tools. > > * bfd: for addr2line, objdump --dwarf, gdb, etc > * gas: support --compress-debug-sections=zstd > * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd > * objcopy: support ELFCOMPRESS_ZSTD input for > --decompress-debug-sections and --compress-debug-sections=zstd > * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd > symbols, so gdb has to link against -lzstd in this patch. > > If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We > can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this > is too heavyweight, so don't do it for now. > > ``` > % ld/ld-new a.o > ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support > ... > > % ld/ld-new a.o --compress-debug-sections=zstd > ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support > > % binutils/objcopy --compress-debug-sections=zstd a.o b.o > binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support > > % binutils/objcopy b.o --decompress-debug-sections > binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support > ... > ``` > --- > Changes from v1: > * use PKG_CHECK_MODULES to check libzstd.pc > > Changes from v2: > * Improve PKG_CHECK_MODULES, autoreconf -vf > * objcopy: check --compress-debug-sections=zstd in a !HAVE_ZSTD build Just one question: you moved PKG_PROG_PKG_CONFIG up in ld/configure.ac, which I think is ok. But what about the other configure.ac files, don't they need PKG_PROG_PKG_CONFIG too? gdb/configure.ac, for instance, uses pkg-config for debuginfod. So if the user passes --without-debuginfod --with-zstd, I expect things to fail, as the pkg-config will be skipped. binutils/configure.ac probably has the same problem, since it uses PKG_CHECK_MODULES for msgpack. I would suggest using PKG_PROG_PKG_CONFIG at the top of all configure.ac that use AM_ZSTD, out of precaution. Simon ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 14:32 ` Simon Marchi @ 2022-09-26 5:12 ` Alan Modra 2022-09-26 7:20 ` Fangrui Song 2022-09-26 14:08 ` Simon Marchi 0 siblings, 2 replies; 25+ messages in thread From: Alan Modra @ 2022-09-26 5:12 UTC (permalink / raw) To: Simon Marchi Cc: Fangrui Song, Jan Beulich, Nick Clifton, binutils, gdb-patches On Fri, Sep 23, 2022 at 10:32:43AM -0400, Simon Marchi wrote: > Just one question: you moved PKG_PROG_PKG_CONFIG up in ld/configure.ac, > which I think is ok. I'm going to fix this one another way. > But what about the other configure.ac files, don't > they need PKG_PROG_PKG_CONFIG too? gdb/configure.ac, for instance, uses > pkg-config for debuginfod. So if the user passes --without-debuginfod > --with-zstd, I expect things to fail, as the pkg-config will be skipped. No, that will be fine, I think The reason being that PKG_CHECK_MODULES AC_REQUIREs PKG_PROG_PKG_CONFIG. So the expansion of AC_DEBUGINFOD will emit an expansion of PKG_PROG_PKG_CONFIG if that has not already occurred, before the entire AC_DEBUGINFOD expansion. > binutils/configure.ac probably has the same problem, since it uses > PKG_CHECK_MODULES for msgpack. I'll fix this one even though it isn't a problem now because AC_DEBUGINFOD and AM_ZSTD preceed it. We should use AS_IF more often, particularly when autoconf macros are invoked. > I would suggest using PKG_PROG_PKG_CONFIG at the top of all configure.ac > that use AM_ZSTD, out of precaution. Better to avoid the need.. binutils/ * configure.ac (msgpack): Use "AS_IF" rather than "if". * configure: Regenerate. ld/ * configure.ac (jansson): Use "AS_IF" rather than "if". * configure: Regenerate. diff --git a/binutils/configure b/binutils/configure index 1c518227f57..4c0c391e9d5 100755 --- a/binutils/configure +++ b/binutils/configure @@ -13507,8 +13507,7 @@ else fi - -if test "$with_msgpack" != no; then +if test "$with_msgpack" != no; then : pkg_failed=no { $as_echo "$as_me:${as_lineno-$LINENO}: checking for msgpack" >&5 @@ -13592,29 +13591,23 @@ fi # Put the nasty error message in config.log where it belongs echo "$MSGPACK_PKG_ERRORS" >&5 - - if test "$with_msgpack" = yes; then - as_fn_error $? "--with-msgpack was given, but msgpack is missing or unusable." "$LINENO" 5 - fi - + if test "$with_msgpack" = yes; then : + as_fn_error $? "--with-msgpack was given, but msgpack is missing or unusable." "$LINENO" 5 +fi elif test $pkg_failed = untried; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } - - if test "$with_msgpack" = yes; then - as_fn_error $? "--with-msgpack was given, but msgpack is missing or unusable." "$LINENO" 5 - fi - + if test "$with_msgpack" = yes; then : + as_fn_error $? "--with-msgpack was given, but msgpack is missing or unusable." "$LINENO" 5 +fi else MSGPACK_CFLAGS=$pkg_cv_MSGPACK_CFLAGS MSGPACK_LIBS=$pkg_cv_MSGPACK_LIBS { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } - $as_echo "#define HAVE_MSGPACK 1" >>confdefs.h - fi fi diff --git a/binutils/configure.ac b/binutils/configure.ac index ec002d3f88f..0798d84f4d1 100644 --- a/binutils/configure.ac +++ b/binutils/configure.ac @@ -278,16 +278,11 @@ AC_ARG_WITH([msgpack], [], [with_msgpack=auto]) - -if test "$with_msgpack" != no; then - PKG_CHECK_MODULES(MSGPACK, msgpack, [ - AC_DEFINE([HAVE_MSGPACK], [1], [Define to 1 if msgpack is available.]) - ], [ - if test "$with_msgpack" = yes; then - AC_MSG_ERROR([--with-msgpack was given, but msgpack is missing or unusable.]) - fi - ]) -fi +AS_IF([test "$with_msgpack" != no], + [PKG_CHECK_MODULES(MSGPACK, msgpack, + [AC_DEFINE([HAVE_MSGPACK], [1], [Define to 1 if msgpack is available.])], + [AS_IF([test "$with_msgpack" = yes], + [AC_MSG_ERROR([--with-msgpack was given, but msgpack is missing or unusable.])])])]) # target-specific stuff: diff --git a/ld/configure b/ld/configure index a1a07005400..4efe3ef5dfc 100755 --- a/ld/configure +++ b/ld/configure @@ -15585,7 +15585,6 @@ else fi -if test "x$enable_jansson" != "xno"; then @@ -15706,8 +15705,7 @@ $as_echo "no" >&6; } PKG_CONFIG="" fi fi - if test -n "$PKG_CONFIG"; then : - +if test "x$enable_jansson" != "xno"; then : pkg_failed=no { $as_echo "$as_me:${as_lineno-$LINENO}: checking for jansson" >&5 @@ -15791,15 +15789,11 @@ fi # Put the nasty error message in config.log where it belongs echo "$JANSSON_PKG_ERRORS" >&5 - - as_fn_error $? "Cannot find jansson library" "$LINENO" 5 - + as_fn_error $? "Cannot find jansson library" "$LINENO" 5 elif test $pkg_failed = untried; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } - - as_fn_error $? "Cannot find jansson library" "$LINENO" 5 - + as_fn_error $? "Cannot find jansson library" "$LINENO" 5 else JANSSON_CFLAGS=$pkg_cv_JANSSON_CFLAGS JANSSON_LIBS=$pkg_cv_JANSSON_LIBS @@ -15814,11 +15808,6 @@ $as_echo "#define HAVE_JANSSON 1" >>confdefs.h fi -else - - as_fn_error $? "Cannot find pkg-config" "$LINENO" 5 - -fi fi diff --git a/ld/configure.ac b/ld/configure.ac index eb55904c090..8a8bfc9a14e 100644 --- a/ld/configure.ac +++ b/ld/configure.ac @@ -296,24 +296,15 @@ AC_ARG_ENABLE([jansson], [enable_jansson=$enableval], [enable_jansson="no"]) -if test "x$enable_jansson" != "xno"; then - PKG_PROG_PKG_CONFIG - AS_IF([test -n "$PKG_CONFIG"], +AS_IF([test "x$enable_jansson" != "xno"], + [PKG_CHECK_MODULES(JANSSON, [jansson], [ - PKG_CHECK_MODULES(JANSSON, [jansson], - [ - AC_DEFINE(HAVE_JANSSON, 1, [The jansson library is to be used]) - AC_SUBST([JANSSON_CFLAGS]) - AC_SUBST([JANSSON_LIBS]) - ], - [ - AC_MSG_ERROR([Cannot find jansson library]) - ]) + AC_DEFINE(HAVE_JANSSON, 1, [The jansson library is to be used]) + AC_SUBST([JANSSON_CFLAGS]) + AC_SUBST([JANSSON_LIBS]) ], - [ - AC_MSG_ERROR([Cannot find pkg-config]) - ]) -fi + [AC_MSG_ERROR([Cannot find jansson library])]) + ]) AM_BINUTILS_WARNINGS -- Alan Modra Australia Development Lab, IBM ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-26 5:12 ` Alan Modra @ 2022-09-26 7:20 ` Fangrui Song 2022-09-26 13:30 ` Alan Modra 2022-09-26 14:08 ` Simon Marchi 1 sibling, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-26 7:20 UTC (permalink / raw) To: Alan Modra; +Cc: Simon Marchi, Jan Beulich, Nick Clifton, binutils, gdb-patches [-- Attachment #1: Type: text/plain, Size: 1586 bytes --] On 2022-09-26, Alan Modra wrote: >On Fri, Sep 23, 2022 at 10:32:43AM -0400, Simon Marchi wrote: >> Just one question: you moved PKG_PROG_PKG_CONFIG up in ld/configure.ac, >> which I think is ok. > >I'm going to fix this one another way. > >> But what about the other configure.ac files, don't >> they need PKG_PROG_PKG_CONFIG too? gdb/configure.ac, for instance, uses >> pkg-config for debuginfod. So if the user passes --without-debuginfod >> --with-zstd, I expect things to fail, as the pkg-config will be skipped. > >No, that will be fine, I think The reason being that >PKG_CHECK_MODULES AC_REQUIREs PKG_PROG_PKG_CONFIG. So the expansion >of AC_DEBUGINFOD will emit an expansion of PKG_PROG_PKG_CONFIG if that >has not already occurred, before the entire AC_DEBUGINFOD expansion. > >> binutils/configure.ac probably has the same problem, since it uses >> PKG_CHECK_MODULES for msgpack. > >I'll fix this one even though it isn't a problem now because >AC_DEBUGINFOD and AM_ZSTD preceed it. We should use AS_IF more often, >particularly when autoconf macros are invoked. > >> I would suggest using PKG_PROG_PKG_CONFIG at the top of all configure.ac >> that use AM_ZSTD, out of precaution. > >Better to avoid the need.. > >binutils/ > * configure.ac (msgpack): Use "AS_IF" rather than "if". > * configure: Regenerate. >ld/ > * configure.ac (jansson): Use "AS_IF" rather than "if". > * configure: Regenerate. > Thanks. I've changed the zstd patch to use AS_IF, too. BTW: I think the `!= xno` trick for ancient shells is not needed. `!= no` is used many times in binutils-gdb. [-- Attachment #2: v4-0001-binutils-gdb-support-zstd-compressed-debug-sectio.patch --] [-- Type: text/x-diff, Size: 109885 bytes --] From 1c70c807039cecab5fff2e7c09e0086d093d4042 Mon Sep 17 00:00:00 2001 From: Fangrui Song <maskray@google.com> Date: Mon, 26 Sep 2022 00:15:19 -0700 Subject: [PATCH v4] binutils, gdb: support zstd compressed debug sections PR29397 PR29563: The new configure option --with-zstd defaults to auto. If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd compressed debug sections for most tools. * bfd: for addr2line, objdump --dwarf, gdb, etc * gas: support --compress-debug-sections=zstd * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd * objcopy: support ELFCOMPRESS_ZSTD input for --decompress-debug-sections and --compress-debug-sections=zstd * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd symbols, so gdb has to link against -lzstd in this patch. If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this is too heavyweight, so don't do it for now. ``` % ld/ld-new a.o ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... % ld/ld-new a.o --compress-debug-sections=zstd ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support % binutils/objcopy --compress-debug-sections=zstd a.o b.o binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support % binutils/objcopy b.o --decompress-debug-sections binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... ``` --- bfd/Makefile.am | 4 +- bfd/Makefile.in | 13 +- bfd/aclocal.m4 | 2 + bfd/bfd-in.h | 3 +- bfd/bfd-in2.h | 11 +- bfd/bfd.c | 26 +- bfd/compress.c | 72 +++- bfd/config.in | 3 + bfd/configure | 268 ++++++++++++- bfd/configure.ac | 3 +- bfd/elf.c | 12 + bfd/elfxx-target.h | 6 +- bfd/section.c | 3 +- binutils/Makefile.in | 5 +- binutils/NEWS | 6 + binutils/aclocal.m4 | 1 + binutils/config.in | 3 + binutils/configure | 136 ++++++- binutils/configure.ac | 3 +- binutils/doc/binutils.texi | 16 +- binutils/objcopy.c | 19 +- binutils/testsuite/binutils-all/compress.exp | 44 +++ config/zstd.m4 | 23 ++ configure | 10 + configure.ac | 3 + gas/Makefile.am | 4 +- gas/Makefile.in | 13 +- gas/NEWS | 3 + gas/aclocal.m4 | 2 + gas/as.c | 13 +- gas/compress-debug.c | 60 ++- gas/compress-debug.h | 10 +- gas/config.in | 3 + gas/configure | 269 ++++++++++++- gas/configure.ac | 3 +- gas/doc/as.texi | 11 +- gas/write.c | 36 +- gdb/Makefile.in | 8 +- gdb/NEWS | 2 + gdb/acinclude.m4 | 3 +- gdb/config.in | 3 + gdb/configure | 137 ++++++- gdb/configure.ac | 4 +- ld/Makefile.am | 5 +- ld/Makefile.in | 11 +- ld/NEWS | 3 + ld/aclocal.m4 | 1 + ld/config.in | 3 + ld/configure | 390 ++++++++++++------- ld/configure.ac | 6 +- ld/emultempl/elf.em | 9 + ld/ld.texi | 5 + ld/ldmain.c | 8 +- ld/lexsup.c | 4 +- ld/testsuite/ld-elf/compress.exp | 17 + 55 files changed, 1487 insertions(+), 254 deletions(-) create mode 100644 config/zstd.m4 diff --git a/bfd/Makefile.am b/bfd/Makefile.am index a5652b2a9c7..98a487b7fcd 100644 --- a/bfd/Makefile.am +++ b/bfd/Makefile.am @@ -57,7 +57,7 @@ ZLIBINC = @zlibinc@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' @LARGEFILE_CPPFLAGS@ if PLUGINS bfdinclude_HEADERS += $(INCDIR)/plugin-api.h @@ -776,7 +776,7 @@ ofiles: stamp-ofiles ; @true libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) libbfd_la_LDFLAGS += -release `cat libtool-soversion` @SHARED_LDFLAGS@ # libtool will build .libs/libbfd.a. We create libbfd.a in the build diff --git a/bfd/Makefile.in b/bfd/Makefile.in index 83d686529a0..3c9ebf38f83 100644 --- a/bfd/Makefile.in +++ b/bfd/Makefile.in @@ -122,10 +122,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../config/acx.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/bfd.m4 $(top_srcdir)/warning.m4 \ @@ -399,6 +401,9 @@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ PKGVERSION = @PKGVERSION@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ REPORT_BUGS_TEXI = @REPORT_BUGS_TEXI@ @@ -416,6 +421,8 @@ WARN_CFLAGS = @WARN_CFLAGS@ WARN_CFLAGS_FOR_BUILD = @WARN_CFLAGS_FOR_BUILD@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -520,7 +527,7 @@ libbfd_la_LDFLAGS = $(am__append_1) -release `cat libtool-soversion` \ # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' \ @LARGEFILE_CPPFLAGS@ @HDEFINES@ @COREFLAG@ @TDEFINES@ \ $(CSEARCH) $(CSWITCHES) $(HAVEVECS) @INCINTL@ @@ -1199,7 +1206,7 @@ OFILES = $(BFD_BACKENDS) $(BFD_MACHINES) @COREFILE@ @bfd64_libs@ libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) # libtool will build .libs/libbfd.a. We create libbfd.a in the build # directory so that we don't have to convert all the programs that use diff --git a/bfd/aclocal.m4 b/bfd/aclocal.m4 index 0f8aa1d4518..09b849dfc87 100644 --- a/bfd/aclocal.m4 +++ b/bfd/aclocal.m4 @@ -1176,10 +1176,12 @@ m4_include([../config/largefile.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/bfd/bfd-in.h b/bfd/bfd-in.h index 8605056aefe..4765ea80536 100644 --- a/bfd/bfd-in.h +++ b/bfd/bfd-in.h @@ -342,7 +342,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 79fcc4eb912..5c80956c79c 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -349,7 +349,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections @@ -962,7 +963,8 @@ typedef struct bfd_section unsigned int compress_status : 2; #define COMPRESS_SECTION_NONE 0 #define COMPRESS_SECTION_DONE 1 -#define DECOMPRESS_SECTION_SIZED 2 +#define DECOMPRESS_SECTION_ZLIB 2 +#define DECOMPRESS_SECTION_ZSTD 3 /* The following flags are used by the ELF linker. */ @@ -6637,12 +6639,14 @@ struct bfd #define BFD_ARCHIVE_FULL_PATH 0x100000 #define BFD_CLOSED_BY_CACHE 0x200000 + /* Compress sections in this BFD with SHF_COMPRESSED zstd. */ +#define BFD_COMPRESS_ZSTD 0x400000 /* Flags bits to be saved in bfd_preserve_save. */ #define BFD_FLAGS_SAVED \ (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ - | BFD_USE_ELF_STT_COMMON) + | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) /* Flags bits which are for BFD use only. */ #define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -7271,6 +7275,7 @@ void bfd_update_compression_header bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); diff --git a/bfd/bfd.c b/bfd/bfd.c index 0a21db11fd6..5f2033bee7a 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -177,12 +177,15 @@ CODE_FRAGMENT .#define BFD_ARCHIVE_FULL_PATH 0x100000 . .#define BFD_CLOSED_BY_CACHE 0x200000 + +. {* Compress sections in this BFD with SHF_COMPRESSED zstd. *} +.#define BFD_COMPRESS_ZSTD 0x400000 . . {* Flags bits to be saved in bfd_preserve_save. *} .#define BFD_FLAGS_SAVED \ . (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ . | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ -. | BFD_USE_ELF_STT_COMMON) +. | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) . . {* Flags bits which are for BFD use only. *} .#define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -2500,6 +2503,9 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, { const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data * esd = elf_section_data (sec); + const unsigned int ch_type = abfd->flags & BFD_COMPRESS_ZSTD + ? ELFCOMPRESS_ZSTD + : ELFCOMPRESS_ZLIB; /* Set the SHF_COMPRESSED bit. */ elf_section_flags (sec) |= SHF_COMPRESSED; @@ -2507,7 +2513,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, if (bed->s->elfclass == ELFCLASS32) { Elf32_External_Chdr *echdr = (Elf32_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, sec->size, &echdr->ch_size); bfd_put_32 (abfd, 1u << sec->alignment_power, &echdr->ch_addralign); @@ -2518,7 +2524,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, else { Elf64_External_Chdr *echdr = (Elf64_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, 0, &echdr->ch_reserved); bfd_put_64 (abfd, sec->size, &echdr->ch_size); bfd_put_64 (abfd, UINT64_C (1) << sec->alignment_power, @@ -2553,14 +2559,15 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, SYNOPSIS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); DESCRIPTION - Check the compression header at CONTENTS of SEC in ABFD and - store the uncompressed size in UNCOMPRESSED_SIZE and the - uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER - if the compression header is valid. + Check the compression header at CONTENTS of SEC in ABFD and store the + ch_type in CH_TYPE, uncompressed size in UNCOMPRESSED_SIZE, and the + uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER if the + compression header is valid. RETURNS Return TRUE if the compression header is valid. @@ -2569,6 +2576,7 @@ RETURNS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power) { @@ -2591,7 +2599,9 @@ bfd_check_compression_header (bfd *abfd, bfd_byte *contents, chdr.ch_size = bfd_get_64 (abfd, &echdr->ch_size); chdr.ch_addralign = bfd_get_64 (abfd, &echdr->ch_addralign); } - if (chdr.ch_type == ELFCOMPRESS_ZLIB + *ch_type = chdr.ch_type; + if ((chdr.ch_type == ELFCOMPRESS_ZLIB + || chdr.ch_type == ELFCOMPRESS_ZSTD) && chdr.ch_addralign == (chdr.ch_addralign & -chdr.ch_addralign)) { *uncompressed_size = chdr.ch_size; diff --git a/bfd/compress.c b/bfd/compress.c index b2e39826e38..95c1f4c42d1 100644 --- a/bfd/compress.c +++ b/bfd/compress.c @@ -20,18 +20,31 @@ #include "sysdep.h" #include <zlib.h> +#ifdef HAVE_ZSTD +#include <zstd.h> +#endif #include "bfd.h" +#include "elf-bfd.h" #include "libbfd.h" #include "safe-ctype.h" #define MAX_COMPRESSION_HEADER_SIZE 24 static bool -decompress_contents (bfd_byte *compressed_buffer, +decompress_contents (bool is_zstd, bfd_byte *compressed_buffer, bfd_size_type compressed_size, bfd_byte *uncompressed_buffer, bfd_size_type uncompressed_size) { + if (is_zstd) + { +#ifdef HAVE_ZSTD + size_t ret = ZSTD_decompress (uncompressed_buffer, uncompressed_size, + compressed_buffer, compressed_size); + return !ZSTD_isError (ret); +#endif + } + z_stream strm; int rc; @@ -69,7 +82,7 @@ decompress_contents (bfd_byte *compressed_buffer, } /* Compress data of the size specified in @var{uncompressed_size} - and pointed to by @var{uncompressed_buffer} using zlib and store + and pointed to by @var{uncompressed_buffer} using zlib/zstd and store as the contents field. This function assumes the contents field was allocated using bfd_malloc() or equivalent. @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, sec->size = orig_uncompressed_size; if (decompress) { - if (!decompress_contents (uncompressed_buffer - + orig_compression_header_size, - zlib_size, buffer, buffer_size)) + if (!decompress_contents ( + sec->compress_status == DECOMPRESS_SECTION_ZSTD, + uncompressed_buffer + orig_compression_header_size, + zlib_size, buffer, buffer_size)) { bfd_set_error (bfd_error_bad_value); bfd_release (abfd, buffer); @@ -175,10 +189,23 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, } else { - if (compress ((Bytef*) buffer + header_size, - &compressed_size, - (const Bytef*) uncompressed_buffer, - uncompressed_size) != Z_OK) + if (abfd->flags & BFD_COMPRESS_ZSTD) + { +#if HAVE_ZSTD + compressed_size = ZSTD_compress ( + buffer + header_size, compressed_size, uncompressed_buffer, + uncompressed_size, ZSTD_CLEVEL_DEFAULT); + if (ZSTD_isError (compressed_size)) + { + bfd_release (abfd, buffer); + bfd_set_error (bfd_error_bad_value); + return 0; + } +#endif + } + else if (compress ((Bytef *)buffer + header_size, &compressed_size, + (const Bytef *)uncompressed_buffer, uncompressed_size) + != Z_OK) { bfd_release (abfd, buffer); bfd_set_error (bfd_error_bad_value); @@ -237,6 +264,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) bfd_size_type save_rawsize; bfd_byte *compressed_buffer; unsigned int compression_header_size; + const unsigned int compress_status = sec->compress_status; if (abfd->direction != write_direction && sec->rawsize != 0) sz = sec->rawsize; @@ -248,7 +276,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) return true; } - switch (sec->compress_status) + switch (compress_status) { case COMPRESS_SECTION_NONE: if (p == NULL) @@ -298,7 +326,8 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) *ptr = p; return true; - case DECOMPRESS_SECTION_SIZED: + case DECOMPRESS_SECTION_ZLIB: + case DECOMPRESS_SECTION_ZSTD: /* Read in the full compressed section contents. */ compressed_buffer = (bfd_byte *) bfd_malloc (sec->compressed_size); if (compressed_buffer == NULL) @@ -316,7 +345,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Restore rawsize and size. */ sec->rawsize = save_rawsize; sec->size = save_size; - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = compress_status; if (!ret) goto fail_compressed; @@ -330,8 +359,10 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Set header size to the zlib header size if it is a SHF_COMPRESSED section. */ compression_header_size = 12; - if (!decompress_contents (compressed_buffer + compression_header_size, - sec->compressed_size - compression_header_size, p, sz)) + if (!decompress_contents (compress_status == DECOMPRESS_SECTION_ZSTD, + compressed_buffer + compression_header_size, + sec->compressed_size - compression_header_size, + p, sz)) { bfd_set_error (bfd_error_bad_value); if (p != *ptr) @@ -381,7 +412,8 @@ DESCRIPTION void bfd_cache_section_contents (asection *sec, void *contents) { - if (sec->compress_status == DECOMPRESS_SECTION_SIZED) + if (sec->compress_status == DECOMPRESS_SECTION_ZLIB + || sec->compress_status == DECOMPRESS_SECTION_ZSTD) sec->compress_status = COMPRESS_SECTION_DONE; sec->contents = contents; sec->flags |= SEC_IN_MEMORY; @@ -418,6 +450,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, int compression_header_size; int header_size; unsigned int saved = sec->compress_status; + unsigned int ch_type; bool compressed; *uncompressed_align_pow_p = 0; @@ -448,7 +481,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, { if (compression_header_size != 0) { - if (!bfd_check_compression_header (abfd, header, sec, + if (!bfd_check_compression_header (abfd, header, sec, &ch_type, uncompressed_size_p, uncompressed_align_pow_p)) compression_header_size = -1; @@ -507,7 +540,7 @@ SYNOPSIS DESCRIPTION Record compressed section size, update section size with decompressed size and set compress_status to - DECOMPRESS_SECTION_SIZED. + DECOMPRESS_SECTION_{ZLIB,ZSTD}. Return @code{FALSE} if the section is not a valid compressed section. Otherwise, return @code{TRUE}. @@ -521,6 +554,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) int header_size; bfd_size_type uncompressed_size; unsigned int uncompressed_alignment_power = 0; + unsigned int ch_type; z_stream strm; compression_header_size = bfd_get_compression_header_size (abfd, sec); @@ -550,6 +584,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) uncompressed_size = bfd_getb64 (header + 4); } else if (!bfd_check_compression_header (abfd, header, sec, + &ch_type, &uncompressed_size, &uncompressed_alignment_power)) { @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) sec->compressed_size = sec->size; sec->size = uncompressed_size; bfd_set_section_alignment (sec, uncompressed_alignment_power); - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD + : DECOMPRESS_SECTION_ZLIB; return true; } diff --git a/bfd/config.in b/bfd/config.in index f54a3cacbea..a59304e0a66 100644 --- a/bfd/config.in +++ b/bfd/config.in @@ -232,6 +232,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/bfd/configure b/bfd/configure index 075d2ee0a1b..d905a7bf7a8 100755 --- a/bfd/configure +++ b/bfd/configure @@ -652,6 +652,11 @@ TDEFINES SHARED_LIBADD SHARED_LDFLAGS LIBM +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir EXEEXT_FOR_BUILD @@ -839,6 +844,7 @@ enable_maintainer_mode enable_install_libbfd enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -848,7 +854,12 @@ CFLAGS LDFLAGS LIBS CPPFLAGS -CPP' +CPP +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1511,6 +1522,8 @@ Optional Packages: Binutils" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1521,6 +1534,13 @@ Some influential environment variables: CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> CPP C preprocessor + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -11086,7 +11106,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11089 "configure" +#line 11109 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11192,7 +11212,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11195 "configure" +#line 11215 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -12995,7 +13015,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. # Use the system's zlib library. @@ -13015,6 +13035,246 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" { $as_echo "$as_me:${as_lineno-$LINENO}: checking compiler support for hidden visibility" >&5 diff --git a/bfd/configure.ac b/bfd/configure.ac index 28f3d1afce6..d1b2f76b31a 100644 --- a/bfd/configure.ac +++ b/bfd/configure.ac @@ -230,9 +230,10 @@ AC_CHECK_DECLS([basename, ffs, stpcpy, asprintf, vasprintf, strnlen]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. AM_ZLIB +AM_ZSTD save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" diff --git a/bfd/elf.c b/bfd/elf.c index 9a0bc1d2740..9d34781a97c 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -1260,6 +1260,18 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, abfd, name); return false; } +#ifndef HAVE_ZSTD + if (newsect->compress_status == DECOMPRESS_SECTION_ZSTD) + { + _bfd_error_handler + /* xgettext:c-format */ + (_ ("%pB: section %s is compressed with zstd, but BFD " + "is not built with zstd support"), + abfd, name); + newsect->compress_status = COMPRESS_SECTION_NONE; + return false; + } +#endif } if (abfd->is_linker_input) diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h index feaba84bf2e..ca600bb5ddf 100644 --- a/bfd/elfxx-target.h +++ b/bfd/elfxx-target.h @@ -989,7 +989,8 @@ const bfd_target TARGET_BIG_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY @@ -1093,7 +1094,8 @@ const bfd_target TARGET_LITTLE_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY diff --git a/bfd/section.c b/bfd/section.c index c7a02d729f2..614570e976e 100644 --- a/bfd/section.c +++ b/bfd/section.c @@ -392,7 +392,8 @@ CODE_FRAGMENT . unsigned int compress_status : 2; .#define COMPRESS_SECTION_NONE 0 .#define COMPRESS_SECTION_DONE 1 -.#define DECOMPRESS_SECTION_SIZED 2 +.#define DECOMPRESS_SECTION_ZLIB 2 +.#define DECOMPRESS_SECTION_ZSTD 3 . . {* The following flags are used by the ELF linker. *} . diff --git a/binutils/Makefile.in b/binutils/Makefile.in index 78d32b350e3..6de4e239408 100644 --- a/binutils/Makefile.in +++ b/binutils/Makefile.in @@ -156,7 +156,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 \ @@ -575,6 +576,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ diff --git a/binutils/NEWS b/binutils/NEWS index 8c2c416c17e..83c73d0660a 100644 --- a/binutils/NEWS +++ b/binutils/NEWS @@ -1,5 +1,11 @@ -*- text -*- +* objcopy --decompress-debug-sections now supports zstd compressed debug + sections. The new option --compress-debug-sections=zstd compresses debug + sections with zstd. + +* addr2line and objdump --dwarf now support zstd compressed debug sections. + * The dlltool program now accepts --deterministic-libraries and --non-deterministic-libraries as command line options to control whether or not it generates deterministic output libraries. If neither of these options diff --git a/binutils/aclocal.m4 b/binutils/aclocal.m4 index a877fa7f873..28271f56279 100644 --- a/binutils/aclocal.m4 +++ b/binutils/aclocal.m4 @@ -1205,6 +1205,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/binutils/config.in b/binutils/config.in index c5fb919aa95..bee8c07e2f7 100644 --- a/binutils/config.in +++ b/binutils/config.in @@ -157,6 +157,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define as const if the declaration of iconv() needs const. */ #undef ICONV_CONST diff --git a/binutils/configure b/binutils/configure index 4c0c391e9d5..5f68fed3f94 100755 --- a/binutils/configure +++ b/binutils/configure @@ -650,6 +650,8 @@ LTLIBICONV LIBICONV MSGPACK_LIBS MSGPACK_CFLAGS +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir DEMANGLER_NAME @@ -832,6 +834,7 @@ enable_build_warnings enable_nls enable_maintainer_mode with_system_zlib +with_zstd with_msgpack enable_rpath with_libiconv_prefix @@ -853,6 +856,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS MSGPACK_CFLAGS MSGPACK_LIBS' @@ -1517,6 +1522,8 @@ Optional Packages: --with-debuginfod Enable debuginfo lookups with debuginfod (auto/yes/no) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-msgpack Enable msgpack support (auto/yes/no) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib @@ -1547,6 +1554,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config MSGPACK_CFLAGS C compiler flags for MSGPACK, overriding pkg-config MSGPACK_LIBS @@ -10804,7 +10813,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10807 "configure" +#line 10816 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10910,7 +10919,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10913 "configure" +#line 10922 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13468,7 +13477,7 @@ cat >>confdefs.h <<_ACEOF _ACEOF -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). @@ -13490,6 +13499,127 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if "$with_zstd" != no; then : + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi + +fi + + + case "${host}" in *-*-msdos* | *-*-go32* | *-*-mingw32* | *-*-cygwin* | *-*-windows*) diff --git a/binutils/configure.ac b/binutils/configure.ac index 0798d84f4d1..01cf8882be4 100644 --- a/binutils/configure.ac +++ b/binutils/configure.ac @@ -265,10 +265,11 @@ fi AC_CHECK_DECLS([asprintf, environ, getc_unlocked, stpcpy, strnlen]) -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). AM_ZLIB +AM_ZSTD BFD_BINARY_FOPEN diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi index 1499db5728c..34a4164eb94 100644 --- a/binutils/doc/binutils.texi +++ b/binutils/doc/binutils.texi @@ -2159,21 +2159,23 @@ ELF ABI. Note - if compression would actually make a section @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd For ELF files, these options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--decompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug} instead of @samp{.debug}. Note - if compression would -actually make a section @emph{larger}, then it is not compressed nor -renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @item --decompress-debug-sections -Decompress DWARF debug sections using zlib. The original section -names of the compressed sections are restored. +Decompress DWARF debug sections. For a @samp{.zdebug} section, the original +name is restored. @item --elf-stt-common=yes @itemx --elf-stt-common=no diff --git a/binutils/objcopy.c b/binutils/objcopy.c index 43261756a42..fc668f00bbc 100644 --- a/binutils/objcopy.c +++ b/binutils/objcopy.c @@ -232,7 +232,8 @@ static enum compress_zlib = compress | 1 << 1, compress_gnu_zlib = compress | 1 << 2, compress_gabi_zlib = compress | 1 << 3, - decompress = 1 << 4 + compress_zstd = compress | 1 << 4, + decompress = 1 << 5 } do_debug_sections = nothing; /* Whether to generate ELF common symbols with the STT_COMMON type. */ @@ -678,8 +679,8 @@ copy_usage (FILE *stream, int exit_status) <commit>\n\ --subsystem <name>[:<version>]\n\ Set PE subsystem to <name> [& <version>]\n\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ - Compress DWARF debug sections using zlib\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ + Compress DWARF debug sections\n\ --decompress-debug-sections Decompress DWARF debug sections using zlib\n\ --elf-stt-common=[yes|no] Generate ELF common symbols with STT_COMMON\n\ type\n\ @@ -2659,7 +2660,8 @@ copy_object (bfd *ibfd, bfd *obfd, const bfd_arch_info_type *input_arch) if ((do_debug_sections & compress) != 0 && do_debug_sections != compress) { - non_fatal (_("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi] is unsupported on `%s'"), + non_fatal (_ ("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi|" + "zstd] is unsupported on `%s'"), bfd_get_archive_filename (ibfd)); return false; } @@ -3807,6 +3809,13 @@ copy_file (const char *input_filename, const char *output_filename, int ofd, if (do_debug_sections != compress_gnu_zlib) ibfd->flags |= BFD_COMPRESS_GABI; break; + case compress_zstd: + ibfd->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; +#ifndef HAVE_ZSTD + fatal (_ ("--compress-debug-sections=zstd: binutils is not built with " + "zstd support")); +#endif + break; case decompress: ibfd->flags |= BFD_DECOMPRESS; break; @@ -5469,6 +5478,8 @@ copy_main (int argc, char *argv[]) do_debug_sections = compress_gnu_zlib; else if (strcasecmp (optarg, "zlib-gabi") == 0) do_debug_sections = compress_gabi_zlib; + else if (strcasecmp (optarg, "zstd") == 0) + do_debug_sections = compress_zstd; else fatal (_("unrecognized --compress-debug-sections type `%s'"), optarg); diff --git a/binutils/testsuite/binutils-all/compress.exp b/binutils/testsuite/binutils-all/compress.exp index c88da74a987..4e74c824664 100644 --- a/binutils/testsuite/binutils-all/compress.exp +++ b/binutils/testsuite/binutils-all/compress.exp @@ -576,6 +576,50 @@ if { [regexp_diff objdump.out $srcdir/$subdir/dw2-3gabi.W] } then { pass "$testname" } +if { [binutils_assemble_flags $srcdir/$subdir/dw2-1.S ${compressedfile}zstd.o --compress-debug-sections=zstd] } then { + set testname "objcopy compress debug sections with zstd" + set got [binutils_run $OBJCOPY "--compress-debug-sections=zstd ${testfile}.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${compressedfile}zstd.o ${copyfile}zstd.o\n" + verbose "cmp ${compressedfile}zstd.o ${copyfile}zstd.o" + set src1 ${compressedfile}zstd.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } + + set testname "objcopy decompress compressed debug sections with zstd" + set got [binutils_run $OBJCOPY "--decompress-debug-sections ${compressedfile}zstd.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${testfile}.o ${copyfile}zstd.o\n" + verbose "cmp ${testfile}.o ${copyfile}zstd.o" + set src1 ${testfile}.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } +} + proc convert_test { testname as_flags objcop_flags } { global srcdir global subdir diff --git a/config/zstd.m4 b/config/zstd.m4 new file mode 100644 index 00000000000..4a20c49a2ef --- /dev/null +++ b/config/zstd.m4 @@ -0,0 +1,23 @@ +dnl Copyright (C) 2022 Free Software Foundation, Inc. +dnl This file is free software, distributed under the terms of the GNU +dnl General Public License. As a special exception to the GNU General +dnl Public License, this file may be distributed as part of a program +dnl that contains a configuration script generated by Autoconf, under +dnl the same distribution terms as the rest of that program. + +dnl Enable features using the zstd library. +AC_DEFUN([AM_ZSTD], [ +AC_ARG_WITH(zstd, + [AS_HELP_STRING([--with-zstd], [support zstd compressed debug sections (default=auto)])], + [], [with_zstd=auto]) + +AS_IF(["$with_zstd" != no], + [PKG_CHECK_MODULES(ZSTD, [libzstd], [ + AC_DEFINE(HAVE_ZSTD, 1, [Define to 1 if zstd is enabled.]) + ], [ + if test "$with_zstd" = yes; then + AC_MSG_ERROR([--with-zstd was given, but pkgconfig/libzstd.pc is not found]) + fi + ]) + ]) +]) diff --git a/configure b/configure index d75f47a1e95..f14e0efd675 100755 --- a/configure +++ b/configure @@ -785,6 +785,7 @@ ac_user_opts=' enable_option_checking with_build_libsubdir with_system_zlib +with_zstd enable_as_accelerator_for enable_offload_targets enable_gold @@ -1567,6 +1568,8 @@ Optional Packages: --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-build-libsubdir=DIR Directory where to find libraries for build system --with-system-zlib use installed libz + --with-zstd Support zstd compressed debug sections + (default=auto) --with-mpc=PATH specify prefix directory for installed MPC package. Equivalent to --with-mpc-include=PATH/include plus --with-mpc-lib=PATH/lib @@ -2925,6 +2928,13 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +fi + + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/configure.ac b/configure.ac index ae18d436aca..0152c69292e 100644 --- a/configure.ac +++ b/configure.ac @@ -246,6 +246,9 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi +AC_ARG_WITH(zstd, +[AS_HELP_STRING([--with-zstd], [Support zstd compressed debug sections (default=auto)])]) + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/gas/Makefile.am b/gas/Makefile.am index bd597398671..5f0f24abf8d 100644 --- a/gas/Makefile.am +++ b/gas/Makefile.am @@ -42,7 +42,7 @@ am__skipyacc = WARN_CFLAGS = @WARN_CFLAGS@ @WARN_WRITE_STRINGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c @@ -407,7 +407,7 @@ STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) EXTRA_as_new_SOURCES = $(CFILES) $(HFILES) $(TARGET_CPU_CFILES) \ diff --git a/gas/Makefile.in b/gas/Makefile.in index c57d78f82c4..5a4dd702252 100644 --- a/gas/Makefile.in +++ b/gas/Makefile.in @@ -140,10 +140,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/../bfd/version.m4 \ @@ -429,6 +431,9 @@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ SED = @SED@ @@ -443,6 +448,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ] ; then echo ../bison/bison -y -L../bison/bison ; else echo @YACC@ ; fi` YFLAGS = @YFLAGS@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -524,7 +531,7 @@ ZLIBINC = @zlibinc@ # maintainer mode is disabled. Avoid this. am__skiplex = am__skipyacc = -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c TARG_CPU_O = config/tc-@target_cpu_type@.@OBJEXT@ @@ -874,7 +881,7 @@ GASLIBS = @OPCODES_LIB@ ../bfd/libbfd.la ../libiberty/libiberty.a STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) diff --git a/gas/NEWS b/gas/NEWS index d61cdb9edd4..9a8b726b942 100644 --- a/gas/NEWS +++ b/gas/NEWS @@ -1,5 +1,8 @@ -*- text -*- +* gas now supports --compress-debug-sections=zstd to compress + debug sections with zstd. + Changes in 2.39: * Remove (rudimentary) support for the x86-64 sub-architectures Intel L1OM and diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 index 70183124da7..722030c776f 100644 --- a/gas/aclocal.m4 +++ b/gas/aclocal.m4 @@ -1196,10 +1196,12 @@ m4_include([../config/lcmessage.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/gas/as.c b/gas/as.c index 6268779cf90..35ad6b3ab3b 100644 --- a/gas/as.c +++ b/gas/as.c @@ -252,14 +252,14 @@ Options:\n\ --alternate initially turn on alternate macro syntax\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib [default]\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ don't compress DWARF debug sections\n")); #else fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ @@ -736,6 +736,15 @@ This program has absolutely no warranty.\n")); flag_compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + flag_compress_debug = COMPRESS_DEBUG_ZSTD; +#else + as_fatal (_ ("--compress-debug-sections=zstd: gas is not " + "built with zstd support")); +#endif + } else as_fatal (_("Invalid --compress-debug-sections option: `%s'"), optarg); diff --git a/gas/compress-debug.c b/gas/compress-debug.c index c80dbeec4e7..3cd175f6e57 100644 --- a/gas/compress-debug.c +++ b/gas/compress-debug.c @@ -21,14 +21,23 @@ #include "config.h" #include <stdio.h> #include <zlib.h> +#if HAVE_ZSTD +#include <zstd.h> +#endif #include "ansidecl.h" #include "compress-debug.h" /* Initialize the compression engine. */ -struct z_stream_s * -compress_init (void) +void * +compress_init (bool use_zstd) { + if (use_zstd) { +#if HAVE_ZSTD + return ZSTD_createCCtx (); +#endif + } + static struct z_stream_s strm; strm.zalloc = NULL; @@ -42,22 +51,37 @@ compress_init (void) from the engine goes into the current frag on the obstack. */ int -compress_data (struct z_stream_s *strm, const char **next_in, - int *avail_in, char **next_out, int *avail_out) +compress_data (bool use_zstd, void *ctx, const char **next_in, int *avail_in, + char **next_out, int *avail_out) { - int out_size = 0; - int x; + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { *next_in, *avail_in, 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_continue); + *next_in += ib.pos; + *avail_in -= ib.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + return (int)ob.pos; +#endif + } + + struct z_stream_s *strm = ctx; strm->next_in = (Bytef *) (*next_in); strm->avail_in = *avail_in; strm->next_out = (Bytef *) (*next_out); strm->avail_out = *avail_out; - x = deflate (strm, Z_NO_FLUSH); + int x = deflate (strm, Z_NO_FLUSH); if (x != Z_OK) return -1; - out_size = *avail_out - strm->avail_out; + int out_size = *avail_out - strm->avail_out; *next_in = (char *) (strm->next_in); *avail_in = strm->avail_in; *next_out = (char *) (strm->next_out); @@ -71,10 +95,28 @@ compress_data (struct z_stream_s *strm, const char **next_in, needed. */ int -compress_finish (struct z_stream_s *strm, char **next_out, +compress_finish (bool use_zstd, void *ctx, char **next_out, int *avail_out, int *out_size) { + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_end); + *out_size = ob.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + if (ret == 0) + ZSTD_freeCCtx (ctx); + return ret ? 1 : 0; +#endif + } + int x; + struct z_stream_s *strm = ctx; strm->avail_in = 0; strm->next_out = (Bytef *) (*next_out); diff --git a/gas/compress-debug.h b/gas/compress-debug.h index 0183e269fef..87de0f8444e 100644 --- a/gas/compress-debug.h +++ b/gas/compress-debug.h @@ -21,19 +21,19 @@ #ifndef COMPRESS_DEBUG_H #define COMPRESS_DEBUG_H +#include <stdbool.h> + struct z_stream_s; /* Initialize the compression engine. */ -extern struct z_stream_s * -compress_init (void); +extern void *compress_init (bool); /* Stream the contents of a frag to the compression engine. Output from the engine goes into the current frag on the obstack. */ -extern int -compress_data (struct z_stream_s *, const char **, int *, char **, int *); +extern int compress_data (bool, void *, const char **, int *, char **, int *); /* Finish the compression and consume the remaining compressed output. */ extern int -compress_finish (struct z_stream_s *, char **, int *, int *); +compress_finish (bool, void *, char **, int *, int *); #endif /* COMPRESS_DEBUG_H */ diff --git a/gas/config.in b/gas/config.in index e243fd277ee..0d1668a3eac 100644 --- a/gas/config.in +++ b/gas/config.in @@ -134,6 +134,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Using i386 COFF? */ #undef I386COFF diff --git a/gas/configure b/gas/configure index d0449a1d7ab..c4d5ba6f994 100755 --- a/gas/configure +++ b/gas/configure @@ -633,6 +633,11 @@ ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS LIBOBJS +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir LIBM @@ -817,6 +822,7 @@ with_cpu enable_nls enable_maintainer_mode with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -828,7 +834,12 @@ LIBS CPPFLAGS CPP YACC -YFLAGS' +YFLAGS +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1493,6 +1504,8 @@ Optional Packages: --with-cpu=CPU default cpu variant is CPU (currently only supported on ARC) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1509,6 +1522,13 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -10702,7 +10722,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10705 "configure" +#line 10725 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10808,7 +10828,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10811 "configure" +#line 10831 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13945,7 +13965,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -13964,6 +13984,247 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if "$with_zstd" != no; then : + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi + +fi + + # Support for VMS timestamps via cross compile if test "$ac_cv_header_time_h" = yes; then diff --git a/gas/configure.ac b/gas/configure.ac index 21795a82bdd..8750ea21381 100644 --- a/gas/configure.ac +++ b/gas/configure.ac @@ -1004,8 +1004,9 @@ AC_CHECK_DECLS([asprintf, mempcpy, stpcpy]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. AM_ZLIB +AM_ZSTD # Support for VMS timestamps via cross compile diff --git a/gas/doc/as.texi b/gas/doc/as.texi index e915893da86..01f49434021 100644 --- a/gas/doc/as.texi +++ b/gas/doc/as.texi @@ -711,16 +711,19 @@ given section @emph{larger} then it is not compressed. @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd These options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--nocompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug}. Note if compression would make a given section -@emph{larger} then it is not compressed nor renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @end ifset diff --git a/gas/write.c b/gas/write.c index f76bbdb706e..0e49df7c03f 100644 --- a/gas/write.c +++ b/gas/write.c @@ -1413,7 +1413,7 @@ write_relocs (bfd *abfd ATTRIBUTE_UNUSED, asection *sec, } static int -compress_frag (struct z_stream_s *strm, const char *contents, int in_size, +compress_frag (bool use_zstd, void *ctx, const char *contents, int in_size, fragS **last_newf, struct obstack *ob) { int out_size; @@ -1442,10 +1442,10 @@ compress_frag (struct z_stream_s *strm, const char *contents, int in_size, as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - out_size = compress_data (strm, &contents, &in_size, - &next_out, &avail_out); + out_size = compress_data (use_zstd, ctx, &contents, &in_size, &next_out, + &avail_out); if (out_size < 0) - return -1; + return -1; f->fr_fix += out_size; total_out_size += out_size; @@ -1471,7 +1471,6 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) const char *section_name; char *compressed_name; char *header; - struct z_stream_s *strm; int x; flagword flags = bfd_section_flags (sec); unsigned int header_size, compression_header_size; @@ -1485,21 +1484,22 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) if (!startswith (section_name, ".debug_")) return; - strm = compress_init (); - if (strm == NULL) + bool use_zstd = abfd->flags & BFD_COMPRESS_ZSTD; + void *ctx = compress_init (use_zstd); + if (ctx == NULL) return; - if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) + if (flag_compress_debug == COMPRESS_DEBUG_GNU_ZLIB) + { + compression_header_size = 0; + header_size = 12; + } + else { compression_header_size = bfd_get_compression_header_size (stdoutput, NULL); header_size = compression_header_size; } - else - { - compression_header_size = 0; - header_size = 12; - } /* Create a new frag to contain the compression header. */ first_newf = frag_alloc (ob); @@ -1531,7 +1531,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) gas_assert (f->fr_type == rs_fill); if (f->fr_fix) { - out_size = compress_frag (strm, f->fr_literal, f->fr_fix, + out_size = compress_frag (use_zstd, ctx, f->fr_literal, f->fr_fix, &last_newf, ob); if (out_size < 0) return; @@ -1545,8 +1545,8 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) { while (count--) { - out_size = compress_frag (strm, fill_literal, (int) fill_size, - &last_newf, ob); + out_size = compress_frag (use_zstd, ctx, fill_literal, + (int)fill_size, &last_newf, ob); if (out_size < 0) return; compressed_size += out_size; @@ -1579,7 +1579,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - x = compress_finish (strm, &next_out, &avail_out, &out_size); + x = compress_finish (use_zstd, ctx, &next_out, &avail_out, &out_size); if (x < 0) return; @@ -2540,6 +2540,8 @@ write_object_file (void) { if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI; + else if (flag_compress_debug == COMPRESS_DEBUG_ZSTD) + stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; else stdoutput->flags |= BFD_COMPRESS; bfd_map_over_sections (stdoutput, compress_debug, (char *) 0); diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 2598b81d205..c528ee5aa80 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -173,6 +173,9 @@ BFD_CFLAGS = -I$(BFD_DIR) -I$(BFD_SRC) ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ + # Where is the decnumber library? Typically in ../libdecnumber. LIBDECNUMBER_DIR = ../libdecnumber LIBDECNUMBER = $(LIBDECNUMBER_DIR)/libdecnumber.a @@ -625,7 +628,7 @@ INTERNAL_CPPFLAGS = $(CPPFLAGS) @GUILE_CPPFLAGS@ @PYTHON_CPPFLAGS@ \ INTERNAL_CFLAGS_BASE = \ $(GLOBAL_CFLAGS) $(PROFILE_CFLAGS) \ $(GDB_CFLAGS) $(OPCODES_CFLAGS) $(READLINE_CFLAGS) $(ZLIBINC) \ - $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ + $(ZSTD_CFLAGS) $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ $(INTL_CFLAGS) $(INCGNU) $(INCSUPPORT) $(LIBBACKTRACE_INC) \ $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS) $(SRCHIGH_CFLAGS) \ $(TOP_CFLAGS) $(PTHREAD_CFLAGS) $(DEBUGINFOD_CFLAGS) @@ -647,7 +650,7 @@ INTERNAL_LDFLAGS = \ # Libraries and corresponding dependencies for compiling gdb. # XM_CLIBS, defined in *config files, have host-dependent libs. # LIBIBERTY appears twice on purpose. -CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \ +CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) $(ZSTD_LIBS) \ $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \ $(XM_CLIBS) $(GDBTKLIBS) $(LIBBACKTRACE_LIB) \ @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \ @@ -2298,6 +2301,7 @@ aclocal_m4_deps = \ ../config/lcmessage.m4 \ ../config/codeset.m4 \ ../config/zlib.m4 \ + ../config/zstd.m4 \ ../config/ax_pthread.m4 $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) diff --git a/gdb/NEWS b/gdb/NEWS index 9619842bc03..1457c99ff04 100644 --- a/gdb/NEWS +++ b/gdb/NEWS @@ -57,6 +57,8 @@ * The Windows native target now supports target async. +* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. + * New commands maintenance set ignore-prologue-end-flag on|off diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 index 95ff2b6f35e..28846119dcb 100644 --- a/gdb/acinclude.m4 +++ b/gdb/acinclude.m4 @@ -43,6 +43,7 @@ m4_include([../config/lib-link.m4]) m4_include([../config/iconv.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../gdbsupport/common.m4]) @@ -233,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" AC_CACHE_CHECK( diff --git a/gdb/config.in b/gdb/config.in index a4975c68aad..e13a409ec2d 100644 --- a/gdb/config.in +++ b/gdb/config.in @@ -589,6 +589,9 @@ /* Define to 1 if you have the `XML_StopParser' function. */ #undef HAVE_XML_STOPPARSER +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to 1 if your system has the _etext variable. */ #undef HAVE__ETEXT diff --git a/gdb/configure b/gdb/configure index 4dbd0c3b13c..5d49b3eba5a 100755 --- a/gdb/configure +++ b/gdb/configure @@ -747,6 +747,8 @@ READLINE_DEPS READLINE LTLIBICONV LIBICONV +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir MIG @@ -893,6 +895,7 @@ enable_codesign with_pkgversion with_bugurl with_system_zlib +with_zstd with_gnu_ld enable_rpath with_libiconv_prefix @@ -961,6 +964,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS XMKMF' ac_subdirs_all='testsuite gdbtk' @@ -1641,6 +1646,8 @@ Optional Packages: --with-pkgversion=PKG Use PKG in the version string in place of "GDB" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib --without-libiconv-prefix don't search for libiconv in includedir and libdir @@ -1721,6 +1728,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config XMKMF Path to xmkmf, Makefile generator for X Window System Use these variables to override the choices made by `configure' or to help @@ -8242,7 +8251,8 @@ if test "$ac_res" != no; then : fi -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -8262,6 +8272,127 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if "$with_zstd" != no; then : + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi + +fi + + + if test "X$prefix" = "XNONE"; then acl_final_prefix="$ac_default_prefix" else @@ -17281,7 +17412,7 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 @@ -17396,7 +17527,7 @@ fi # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 diff --git a/gdb/configure.ac b/gdb/configure.ac index 215a91c5b27..e8cf808d280 100644 --- a/gdb/configure.ac +++ b/gdb/configure.ac @@ -460,8 +460,10 @@ AC_SEARCH_LIBS(gethostbyname, nsl) # Some systems (e.g. Solaris) have `socketpair' in libsocket. AC_SEARCH_LIBS(socketpair, socket) -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. AM_ZLIB +AM_ZSTD AM_ICONV diff --git a/ld/Makefile.am b/ld/Makefile.am index d31021c13e2..bbe25f3aca2 100644 --- a/ld/Makefile.am +++ b/ld/Makefile.am @@ -45,7 +45,7 @@ ELF_CLFAGS=-DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -959,7 +959,8 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ldbuildid.c ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) # Dependency tracking for the generated emulation files. EXTRA_ld_new_SOURCES += $(ALL_EMULATION_SOURCES) $(ALL_64_EMULATION_SOURCES) diff --git a/ld/Makefile.in b/ld/Makefile.in index ee0c98f65b0..400eed2717e 100644 --- a/ld/Makefile.in +++ b/ld/Makefile.in @@ -126,7 +126,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 $(top_srcdir)/configure.ac @@ -477,6 +478,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -564,7 +567,7 @@ ELF_CLFAGS = -DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -1011,7 +1014,9 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) + # # # Build a dummy plugin using libtool. diff --git a/ld/NEWS b/ld/NEWS index 355752e6b24..dfe2690d9f2 100644 --- a/ld/NEWS +++ b/ld/NEWS @@ -1,5 +1,8 @@ -*- text -*- +* ld now supports zstd compressed debug sections. The new option + --compress-debug-sections=zstd compresses debug sections with zstd. + Changes in 2.39: * The ELF linker will now generate a warning message if the stack is made diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 index d20c542eed5..893e973e4f2 100644 --- a/ld/aclocal.m4 +++ b/ld/aclocal.m4 @@ -1203,6 +1203,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/ld/config.in b/ld/config.in index 0ccd79d59cd..3916740eee4 100644 --- a/ld/config.in +++ b/ld/config.in @@ -162,6 +162,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/ld/configure b/ld/configure index 4efe3ef5dfc..80472f9342e 100755 --- a/ld/configure +++ b/ld/configure @@ -646,6 +646,8 @@ elf_plt_unwind_list_options elf_shlib_list_options elf_list_options STRINGIFY +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir NATIVE_LIB_DIRS @@ -679,9 +681,6 @@ WARN_CFLAGS_FOR_BUILD WARN_CFLAGS JANSSON_LIBS JANSSON_CFLAGS -PKG_CONFIG_LIBDIR -PKG_CONFIG_PATH -PKG_CONFIG enable_libctf ENABLE_LIBCTF_FALSE ENABLE_LIBCTF_TRUE @@ -711,6 +710,9 @@ LD FGREP SED LIBTOOL +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG EGREP CPP GREP @@ -855,6 +857,7 @@ enable_werror enable_build_warnings enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -868,14 +871,16 @@ CXX CXXFLAGS CCC CPP -CXXCPP PKG_CONFIG PKG_CONFIG_PATH PKG_CONFIG_LIBDIR +CXXCPP JANSSON_CFLAGS JANSSON_LIBS YACC -YFLAGS' +YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1552,6 +1557,8 @@ Optional Packages: --with-lib-path=dir1:dir2... set default LIB_PATH --with-sysroot=DIR Search for usr/lib et al within DIR. --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1564,12 +1571,12 @@ Some influential environment variables: CXX C++ compiler command CXXFLAGS C++ compiler flags CPP C preprocessor - CXXCPP C++ preprocessor PKG_CONFIG path to pkg-config utility PKG_CONFIG_PATH directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path + CXXCPP C++ preprocessor JANSSON_CFLAGS C compiler flags for JANSSON, overriding pkg-config JANSSON_LIBS @@ -1580,6 +1587,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -5388,6 +5397,126 @@ $as_echo "$ac_cv_safe_to_define___extensions__" >&6; } + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + case `pwd` in *\ * | *\ *) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 @@ -11491,7 +11620,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11494 "configure" +#line 11623 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11597,7 +11726,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11600 "configure" +#line 11729 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -15585,126 +15714,6 @@ else fi - - - - - - - -if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. -set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -PKG_CONFIG=$ac_cv_path_PKG_CONFIG -if test -n "$PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 -$as_echo "$PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_path_PKG_CONFIG"; then - ac_pt_PKG_CONFIG=$PKG_CONFIG - # Extract the first word of "pkg-config", so it can be a program name with args. -set dummy pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $ac_pt_PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG -if test -n "$ac_pt_PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 -$as_echo "$ac_pt_PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_pt_PKG_CONFIG" = x; then - PKG_CONFIG="" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - PKG_CONFIG=$ac_pt_PKG_CONFIG - fi -else - PKG_CONFIG="$ac_cv_path_PKG_CONFIG" -fi - -fi -if test -n "$PKG_CONFIG"; then - _pkg_min_version=0.9.0 - { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 -$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } - if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - PKG_CONFIG="" - fi -fi if test "x$enable_jansson" != "xno"; then : pkg_failed=no @@ -17041,8 +17050,8 @@ $as_echo "#define HAVE_DECL_GETOPT 1" >>confdefs.h fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -17061,6 +17070,127 @@ fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if "$with_zstd" != no; then : + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi + +fi + + # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string # concatenation, or ostring.sed otherwise. This is to support the diff --git a/ld/configure.ac b/ld/configure.ac index 8a8bfc9a14e..5a6e16764cc 100644 --- a/ld/configure.ac +++ b/ld/configure.ac @@ -34,6 +34,7 @@ AC_PROG_GREP AC_GNU_SOURCE AC_USE_SYSTEM_EXTENSIONS AC_PROG_INSTALL +PKG_PROG_PKG_CONFIG LT_INIT ACX_LARGEFILE @@ -377,9 +378,10 @@ if test $ld_cv_decl_getopt_unistd_h = yes; then [Is the prototype for getopt in <unistd.h> in the expected format?]) fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. AM_ZLIB +AM_ZSTD # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string diff --git a/ld/emultempl/elf.em b/ld/emultempl/elf.em index c52484f35d3..acd66f907d1 100644 --- a/ld/emultempl/elf.em +++ b/ld/emultempl/elf.em @@ -668,6 +668,15 @@ gld${EMULATION_NAME}_handle_option (int optc) link_info.compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + link_info.compress_debug = COMPRESS_DEBUG_ZSTD; +#else + einfo (_ ("%F%P: --compress-debug-sections=zstd: ld is not built " + "with zstd support\n")); +#endif + } else einfo (_("%F%P: invalid --compress-debug-sections option: \`%s'\n"), optarg); diff --git a/ld/ld.texi b/ld/ld.texi index eabbec8faa9..9daed2e7e9f 100644 --- a/ld/ld.texi +++ b/ld/ld.texi @@ -2863,10 +2863,12 @@ but for most Linux based systems it will be @code{both}. @kindex --compress-debug-sections=zlib @kindex --compress-debug-sections=zlib-gnu @kindex --compress-debug-sections=zlib-gabi +@kindex --compress-debug-sections=zstd @item --compress-debug-sections=none @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd On ELF platforms, these options control how DWARF debug sections are compressed using zlib. @@ -2880,6 +2882,9 @@ sets the SHF_COMPRESSED flag in the sections' headers. The @option{--compress-debug-sections=zlib} option is an alias for @option{--compress-debug-sections=zlib-gabi}. +@option{--compress-debug-sections=zstd} compresses DWARF debug sections using +zstd. + Note that this option overrides any compression in input debug sections, so if a binary is linked with @option{--compress-debug-sections=none} for example, then any compressed debug sections in input files will be diff --git a/ld/ldmain.c b/ld/ldmain.c index 1bbddaaad32..71af5935b74 100644 --- a/ld/ldmain.c +++ b/ld/ldmain.c @@ -506,8 +506,12 @@ main (int argc, char **argv) if ((link_info.compress_debug & COMPRESS_DEBUG)) { link_info.output_bfd->flags |= BFD_COMPRESS; - if (link_info.compress_debug == COMPRESS_DEBUG_GABI_ZLIB) - link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug != COMPRESS_DEBUG_GNU_ZLIB) + { + link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug == COMPRESS_DEBUG_ZSTD) + link_info.output_bfd->flags |= BFD_COMPRESS_ZSTD; + } } ldwrite (); diff --git a/ld/lexsup.c b/ld/lexsup.c index 9225f71b3ce..299371fb775 100644 --- a/ld/lexsup.c +++ b/ld/lexsup.c @@ -2146,8 +2146,8 @@ elf_static_list_options (FILE *file) fprintf (file, _("\ --package-metadata[=JSON] Generate package metadata note\n")); fprintf (file, _("\ - --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ - Compress DWARF debug sections using zlib\n")); + --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi|zstd]\n\ + Compress DWARF debug sections\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (file, _("\ Default: zlib-gabi\n")); diff --git a/ld/testsuite/ld-elf/compress.exp b/ld/testsuite/ld-elf/compress.exp index cfdd767b8ab..7b8a31cc41d 100644 --- a/ld/testsuite/ld-elf/compress.exp +++ b/ld/testsuite/ld-elf/compress.exp @@ -254,3 +254,20 @@ if { [regexp_diff tmpdir/$test.out $srcdir/$subdir/$test.rt] } then { } else { pass "$test_name" } + +if { ![ld_assemble $as "--compress-debug-sections=zstd $srcdir/$subdir/empty.s" tmpdir/emptyzstd.o ] } { + return +} +set build_tests { + {"Build libzstdfoo.so with zstd compressed debug sections" + "-shared" "-fPIC -g -Wa,--compress-debug-sections=zstd -Wl,--compress-debug-sections=zstd" + {foo.c} {} "libzstdfoo.so"} +} +set run_tests { + {"Run zstdnormal with libzstdfoo.so with zstd compressed debug sections" + "tmpdir/begin.o tmpdir/libzstdfoo.so tmpdir/end.o -Wl,--compress-debug-sections=zstd" "" + {main.c} "zstdnormal" "normal.out" "-Wa,--compress-debug-sections=zstd"} +} + +run_cc_link_tests $build_tests +run_ld_link_exec_tests $run_tests -- 2.37.3.998.g577e59143f-goog ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-26 7:20 ` Fangrui Song @ 2022-09-26 13:30 ` Alan Modra 0 siblings, 0 replies; 25+ messages in thread From: Alan Modra @ 2022-09-26 13:30 UTC (permalink / raw) To: Fangrui Song Cc: Simon Marchi, Jan Beulich, Nick Clifton, binutils, gdb-patches On Mon, Sep 26, 2022 at 12:20:27AM -0700, Fangrui Song wrote: > On 2022-09-26, Alan Modra wrote: > > Better to avoid the need.. > > > > binutils/ > > * configure.ac (msgpack): Use "AS_IF" rather than "if". > > * configure: Regenerate. > > ld/ > > * configure.ac (jansson): Use "AS_IF" rather than "if". > > * configure: Regenerate. > > > > Thanks. I've changed the zstd patch to use AS_IF, too. It probably wasn't necessary inside AC_DEFUN, but won't hurt either. > BTW: I think the `!= xno` trick for ancient shells is not needed. `!= > no` is used many times in binutils-gdb. Quite possibly true nowadays. I'm not going to be the one that removes them all. :-) Your patch looks OK to me, apart from a few formatting nits. Here are two examples: > @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, > sec->size = orig_uncompressed_size; > if (decompress) > { > - if (!decompress_contents (uncompressed_buffer > - + orig_compression_header_size, > - zlib_size, buffer, buffer_size)) > + if (!decompress_contents ( > + sec->compress_status == DECOMPRESS_SECTION_ZSTD, > + uncompressed_buffer + orig_compression_header_size, > + zlib_size, buffer, buffer_size)) > { > bfd_set_error (bfd_error_bad_value); > bfd_release (abfd, buffer); One of the formatting guidelines in https://www.gnu.org/prep/standards/html_node/Formatting.html#Formatting talks about emacs indenting of code. The above looks reasonable with your indenting by hand, but auto-indent will turn it into if (!decompress_contents ( sec->compress_status == DECOMPRESS_SECTION_ZSTD, uncompressed_buffer + orig_compression_header_size, zlib_size, buffer, buffer_size)) lining up function args with the open parenthesis. Inserting a couple of temporary vars is the nicest solution to keeping line length manageable. bool is_z = sec->compress_status == DECOMPRESS_SECTION_ZSTD; bfd_byte *p = uncompressed_buffer + orig_compression_header_size; if (!decompress_contents (is_z, p, zlib_size, buffer, buffer_size)) > @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) > sec->compressed_size = sec->size; > sec->size = uncompressed_size; > bfd_set_section_alignment (sec, uncompressed_alignment_power); > - sec->compress_status = DECOMPRESS_SECTION_SIZED; > + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD > + : DECOMPRESS_SECTION_ZLIB; > > return true; > } The same emacs auto-indent will ruin the above. Write sec->compress_status = (ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD : DECOMPRESS_SECTION_ZLIB); with unnecessary parentheses to guide indentation. OK to commit with those things fixed. -- Alan Modra Australia Development Lab, IBM ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-26 5:12 ` Alan Modra 2022-09-26 7:20 ` Fangrui Song @ 2022-09-26 14:08 ` Simon Marchi 2022-09-27 0:33 ` Alan Modra 1 sibling, 1 reply; 25+ messages in thread From: Simon Marchi @ 2022-09-26 14:08 UTC (permalink / raw) To: Alan Modra; +Cc: Fangrui Song, Jan Beulich, Nick Clifton, binutils, gdb-patches >> But what about the other configure.ac files, don't >> they need PKG_PROG_PKG_CONFIG too? gdb/configure.ac, for instance, uses >> pkg-config for debuginfod. So if the user passes --without-debuginfod >> --with-zstd, I expect things to fail, as the pkg-config will be skipped. > > No, that will be fine, I think The reason being that > PKG_CHECK_MODULES AC_REQUIREs PKG_PROG_PKG_CONFIG. So the expansion > of AC_DEBUGINFOD will emit an expansion of PKG_PROG_PKG_CONFIG if that > has not already occurred, before the entire AC_DEBUGINFOD expansion. Ah, makes sense. Thanks for the explanation Alan. > >> binutils/configure.ac probably has the same problem, since it uses >> PKG_CHECK_MODULES for msgpack. > > I'll fix this one even though it isn't a problem now because > AC_DEBUGINFOD and AM_ZSTD preceed it. We should use AS_IF more often, > particularly when autoconf macros are invoked. Silly question but... AM_ZSTD was clearly named after AM_ZLIB. But why does AM_ZLIB have the "AM" (automake, I suppose) prefix? It doesn't seem to have anything to do with automake. Ack about AS_IF, I will try to remember this for top-level configure code. Simon ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-26 14:08 ` Simon Marchi @ 2022-09-27 0:33 ` Alan Modra 0 siblings, 0 replies; 25+ messages in thread From: Alan Modra @ 2022-09-27 0:33 UTC (permalink / raw) To: Simon Marchi Cc: Fangrui Song, Jan Beulich, Nick Clifton, binutils, gdb-patches On Mon, Sep 26, 2022 at 10:08:30AM -0400, Simon Marchi wrote: > Silly question but... AM_ZSTD was clearly named after AM_ZLIB. But why > does AM_ZLIB have the "AM" (automake, I suppose) prefix? It doesn't > seem to have anything to do with automake. You'll have to ask Joel. The original patch is at https://sourceware.org/pipermail/binutils/2009-October/063904.html As far as I can see, we get to choose user-defined autoconf macro naming conventions ourselves. -- Alan Modra Australia Development Lab, IBM ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song 2022-09-23 14:32 ` Simon Marchi @ 2022-09-23 15:45 ` Nick Clifton 2022-09-23 15:58 ` Simon Marchi 2022-09-24 6:53 ` Enze Li ` (2 subsequent siblings) 4 siblings, 1 reply; 25+ messages in thread From: Nick Clifton @ 2022-09-23 15:45 UTC (permalink / raw) To: Fangrui Song, Alan Modra, Jan Beulich, Simon Marchi; +Cc: binutils, gdb-patches Hi Fangrui, > PR29397 PR29563: The new configure option --with-zstd defaults to auto. > If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd > compressed debug sections for most tools. One small, but important point. Please could you add entries to the gas/NEWS binutils/NEWS and ld/NEWS files giving a brief description of this new feature. Cheers Nick ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 15:45 ` Nick Clifton @ 2022-09-23 15:58 ` Simon Marchi 2022-09-23 18:20 ` Fangrui Song 0 siblings, 1 reply; 25+ messages in thread From: Simon Marchi @ 2022-09-23 15:58 UTC (permalink / raw) To: Nick Clifton, Fangrui Song, Alan Modra, Jan Beulich; +Cc: binutils, gdb-patches On 2022-09-23 11:45, Nick Clifton wrote: > Hi Fangrui, > >> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >> compressed debug sections for most tools. > > One small, but important point. Please could you add entries to the gas/NEWS > binutils/NEWS and ld/NEWS files giving a brief description of this new feature. I guess to gdb/NEWS, while at it :). Simon ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 15:58 ` Simon Marchi @ 2022-09-23 18:20 ` Fangrui Song 2022-09-23 18:57 ` Simon Marchi 0 siblings, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-23 18:20 UTC (permalink / raw) To: Simon Marchi, Nick Clifton; +Cc: Alan Modra, Jan Beulich, binutils, gdb-patches [-- Attachment #1: Type: text/plain, Size: 4302 bytes --] On 2022-09-23, Simon Marchi wrote: > > >On 2022-09-23 11:45, Nick Clifton wrote: >> Hi Fangrui, >> >>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>> compressed debug sections for most tools. Hi Simon and Nick, I have checked that `make all-gdb` works in a --enable-gdb --without-debuginfod build. I then inspected the 2020 commits adding AC_DEBUGINFOD to gdb/configure.ac and binutils/configure.ac. They come with no PKG_PROG_PKG_CONFIG and config/debuginfod.m4 does not have PKG_PROG_PKG_CONFIG, but they still work. So I think AC_ZSTD doesn't need PKG_PROG_PKG_CONFIG, either. Though I have no insight why it behaves that way:) >> One small, but important point. Please could you add entries to the gas/NEWS >> binutils/NEWS and ld/NEWS files giving a brief description of this new feature. > >I guess to gdb/NEWS, while at it :). > >Simon Here are the NEWS entries :) (As usual, it's available as the latest https://gitlab.com/MaskRay/binutils-gdb/-/commits/zstd) ---- bfd/Makefile.am | 4 +- @@ -58,2 +51,3 @@ binutils/Makefile.in | 5 +- + binutils/NEWS | 6 + binutils/aclocal.m4 | 1 + @@ -70,2 +64,3 @@ gas/Makefile.in | 13 +- + gas/NEWS | 3 + gas/aclocal.m4 | 2 + @@ -80,2 +75,3 @@ gdb/Makefile.in | 8 +- + gdb/NEWS | 2 + gdb/acinclude.m4 | 3 +- @@ -86,2 +82,3 @@ ld/Makefile.in | 11 +- + ld/NEWS | 3 + ld/aclocal.m4 | 1 + @@ -95,3 +92,3 @@ ld/testsuite/ld-elf/compress.exp | 17 + - 51 files changed, 1470 insertions(+), 255 deletions(-) + 55 files changed, 1484 insertions(+), 255 deletions(-) create mode 100644 config/zstd.m4 @@ -990,2 +987,18 @@ abs_top_builddir = @abs_top_builddir@ +diff --git a/binutils/NEWS b/binutils/NEWS +index 8c2c416c17e..073e9bc4647 100644 +--- a/binutils/NEWS ++++ b/binutils/NEWS +@@ -1,5 +1,11 @@ + -*- text -*- + ++* objcopy --decompress-debug-sections now supports zstd compressed debug ++ sections. The new option --compress-debug-sections=zstd compresses debug ++ sections with zstd. ++ ++* addr2line and objdump --dwarf now support zstd compressed debug sections. ++ + * The dlltool program now accepts --deterministic-libraries and + --non-deterministic-libraries as command line options to control whether or + not it generates deterministic output libraries. If neither of these options diff --git a/binutils/aclocal.m4 b/binutils/aclocal.m4 @@ -1539,2 +1552,15 @@ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) +diff --git a/gas/NEWS b/gas/NEWS +index d61cdb9edd4..9a8b726b942 100644 +--- a/gas/NEWS ++++ b/gas/NEWS +@@ -1,5 +1,8 @@ + -*- text -*- + ++* gas now supports --compress-debug-sections=zstd to compress ++ debug sections with zstd. ++ + Changes in 2.39: + + * Remove (rudimentary) support for the x86-64 sub-architectures Intel L1OM and diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 @@ -2262,2 +2288,15 @@ $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) +diff --git a/gdb/NEWS b/gdb/NEWS +index 9619842bc03..1457c99ff04 100644 +--- a/gdb/NEWS ++++ b/gdb/NEWS +@@ -57,6 +57,8 @@ + + * The Windows native target now supports target async. + ++* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. ++ + * New commands + + maintenance set ignore-prologue-end-flag on|off diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 @@ -2582,2 +2621,15 @@ # Build a dummy plugin using libtool. +diff --git a/ld/NEWS b/ld/NEWS +index 355752e6b24..dfe2690d9f2 100644 +--- a/ld/NEWS ++++ b/ld/NEWS +@@ -1,5 +1,8 @@ + -*- text -*- + ++* ld now supports zstd compressed debug sections. The new option ++ --compress-debug-sections=zstd compresses debug sections with zstd. ++ + Changes in 2.39: + + * The ELF linker will now generate a warning message if the stack is made diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 [-- Attachment #2: v4-0001-binutils-gdb-support-zstd-compressed-debug-sectio.patch --] [-- Type: text/x-diff, Size: 110131 bytes --] From 5e60b8fac5190059c0c253a8241ddf9a9763afa0 Mon Sep 17 00:00:00 2001 From: Fangrui Song <maskray@google.com> Date: Fri, 23 Sep 2022 11:18:36 -0700 Subject: [PATCH v4] binutils, gdb: support zstd compressed debug sections PR29397 PR29563: The new configure option --with-zstd defaults to auto. If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd compressed debug sections for most tools. * bfd: for addr2line, objdump --dwarf, gdb, etc * gas: support --compress-debug-sections=zstd * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd * objcopy: support ELFCOMPRESS_ZSTD input for --decompress-debug-sections and --compress-debug-sections=zstd * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd symbols, so gdb has to link against -lzstd in this patch. If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this is too heavyweight, so don't do it for now. ``` % ld/ld-new a.o ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... % ld/ld-new a.o --compress-debug-sections=zstd ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support % binutils/objcopy --compress-debug-sections=zstd a.o b.o binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support % binutils/objcopy b.o --decompress-debug-sections binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... ``` --- bfd/Makefile.am | 4 +- bfd/Makefile.in | 13 +- bfd/aclocal.m4 | 2 + bfd/bfd-in.h | 3 +- bfd/bfd-in2.h | 11 +- bfd/bfd.c | 26 +- bfd/compress.c | 72 +++- bfd/config.in | 3 + bfd/configure | 268 ++++++++++++- bfd/configure.ac | 3 +- bfd/elf.c | 12 + bfd/elfxx-target.h | 6 +- bfd/section.c | 3 +- binutils/Makefile.in | 5 +- binutils/NEWS | 6 + binutils/aclocal.m4 | 1 + binutils/config.in | 3 + binutils/configure | 135 ++++++- binutils/configure.ac | 3 +- binutils/doc/binutils.texi | 16 +- binutils/objcopy.c | 19 +- binutils/testsuite/binutils-all/compress.exp | 44 +++ config/zstd.m4 | 24 ++ configure | 10 + configure.ac | 3 + gas/Makefile.am | 4 +- gas/Makefile.in | 13 +- gas/NEWS | 3 + gas/aclocal.m4 | 2 + gas/as.c | 13 +- gas/compress-debug.c | 60 ++- gas/compress-debug.h | 10 +- gas/config.in | 3 + gas/configure | 268 ++++++++++++- gas/configure.ac | 3 +- gas/doc/as.texi | 11 +- gas/write.c | 36 +- gdb/Makefile.in | 8 +- gdb/NEWS | 2 + gdb/acinclude.m4 | 3 +- gdb/config.in | 3 + gdb/configure | 136 ++++++- gdb/configure.ac | 4 +- ld/Makefile.am | 5 +- ld/Makefile.in | 11 +- ld/NEWS | 3 + ld/aclocal.m4 | 1 + ld/config.in | 3 + ld/configure | 389 ++++++++++++------- ld/configure.ac | 7 +- ld/emultempl/elf.em | 9 + ld/ld.texi | 5 + ld/ldmain.c | 8 +- ld/lexsup.c | 4 +- ld/testsuite/ld-elf/compress.exp | 17 + 55 files changed, 1484 insertions(+), 255 deletions(-) create mode 100644 config/zstd.m4 diff --git a/bfd/Makefile.am b/bfd/Makefile.am index c23dff6cac3..cfe903c63f0 100644 --- a/bfd/Makefile.am +++ b/bfd/Makefile.am @@ -57,7 +57,7 @@ ZLIBINC = @zlibinc@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' @LARGEFILE_CPPFLAGS@ if PLUGINS bfdinclude_HEADERS += $(INCDIR)/plugin-api.h @@ -776,7 +776,7 @@ ofiles: stamp-ofiles ; @true libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) libbfd_la_LDFLAGS += -release `cat libtool-soversion` @SHARED_LDFLAGS@ # libtool will build .libs/libbfd.a. We create libbfd.a in the build diff --git a/bfd/Makefile.in b/bfd/Makefile.in index 82843d2d61d..6be41a72a59 100644 --- a/bfd/Makefile.in +++ b/bfd/Makefile.in @@ -122,10 +122,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../config/acx.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/bfd.m4 $(top_srcdir)/warning.m4 \ @@ -399,6 +401,9 @@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ PKGVERSION = @PKGVERSION@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ REPORT_BUGS_TEXI = @REPORT_BUGS_TEXI@ @@ -416,6 +421,8 @@ WARN_CFLAGS = @WARN_CFLAGS@ WARN_CFLAGS_FOR_BUILD = @WARN_CFLAGS_FOR_BUILD@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -520,7 +527,7 @@ libbfd_la_LDFLAGS = $(am__append_1) -release `cat libtool-soversion` \ # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' \ @LARGEFILE_CPPFLAGS@ @HDEFINES@ @COREFLAG@ @TDEFINES@ \ $(CSEARCH) $(CSWITCHES) $(HAVEVECS) @INCINTL@ @@ -1199,7 +1206,7 @@ OFILES = $(BFD_BACKENDS) $(BFD_MACHINES) @COREFILE@ @bfd64_libs@ libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) # libtool will build .libs/libbfd.a. We create libbfd.a in the build # directory so that we don't have to convert all the programs that use diff --git a/bfd/aclocal.m4 b/bfd/aclocal.m4 index 0f8aa1d4518..09b849dfc87 100644 --- a/bfd/aclocal.m4 +++ b/bfd/aclocal.m4 @@ -1176,10 +1176,12 @@ m4_include([../config/largefile.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/bfd/bfd-in.h b/bfd/bfd-in.h index 8605056aefe..4765ea80536 100644 --- a/bfd/bfd-in.h +++ b/bfd/bfd-in.h @@ -342,7 +342,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 79fcc4eb912..5c80956c79c 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -349,7 +349,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections @@ -962,7 +963,8 @@ typedef struct bfd_section unsigned int compress_status : 2; #define COMPRESS_SECTION_NONE 0 #define COMPRESS_SECTION_DONE 1 -#define DECOMPRESS_SECTION_SIZED 2 +#define DECOMPRESS_SECTION_ZLIB 2 +#define DECOMPRESS_SECTION_ZSTD 3 /* The following flags are used by the ELF linker. */ @@ -6637,12 +6639,14 @@ struct bfd #define BFD_ARCHIVE_FULL_PATH 0x100000 #define BFD_CLOSED_BY_CACHE 0x200000 + /* Compress sections in this BFD with SHF_COMPRESSED zstd. */ +#define BFD_COMPRESS_ZSTD 0x400000 /* Flags bits to be saved in bfd_preserve_save. */ #define BFD_FLAGS_SAVED \ (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ - | BFD_USE_ELF_STT_COMMON) + | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) /* Flags bits which are for BFD use only. */ #define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -7271,6 +7275,7 @@ void bfd_update_compression_header bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); diff --git a/bfd/bfd.c b/bfd/bfd.c index 0a21db11fd6..5f2033bee7a 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -177,12 +177,15 @@ CODE_FRAGMENT .#define BFD_ARCHIVE_FULL_PATH 0x100000 . .#define BFD_CLOSED_BY_CACHE 0x200000 + +. {* Compress sections in this BFD with SHF_COMPRESSED zstd. *} +.#define BFD_COMPRESS_ZSTD 0x400000 . . {* Flags bits to be saved in bfd_preserve_save. *} .#define BFD_FLAGS_SAVED \ . (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ . | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ -. | BFD_USE_ELF_STT_COMMON) +. | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) . . {* Flags bits which are for BFD use only. *} .#define BFD_FLAGS_FOR_BFD_USE_MASK \ @@ -2500,6 +2503,9 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, { const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data * esd = elf_section_data (sec); + const unsigned int ch_type = abfd->flags & BFD_COMPRESS_ZSTD + ? ELFCOMPRESS_ZSTD + : ELFCOMPRESS_ZLIB; /* Set the SHF_COMPRESSED bit. */ elf_section_flags (sec) |= SHF_COMPRESSED; @@ -2507,7 +2513,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, if (bed->s->elfclass == ELFCLASS32) { Elf32_External_Chdr *echdr = (Elf32_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, sec->size, &echdr->ch_size); bfd_put_32 (abfd, 1u << sec->alignment_power, &echdr->ch_addralign); @@ -2518,7 +2524,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, else { Elf64_External_Chdr *echdr = (Elf64_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, 0, &echdr->ch_reserved); bfd_put_64 (abfd, sec->size, &echdr->ch_size); bfd_put_64 (abfd, UINT64_C (1) << sec->alignment_power, @@ -2553,14 +2559,15 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, SYNOPSIS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); DESCRIPTION - Check the compression header at CONTENTS of SEC in ABFD and - store the uncompressed size in UNCOMPRESSED_SIZE and the - uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER - if the compression header is valid. + Check the compression header at CONTENTS of SEC in ABFD and store the + ch_type in CH_TYPE, uncompressed size in UNCOMPRESSED_SIZE, and the + uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER if the + compression header is valid. RETURNS Return TRUE if the compression header is valid. @@ -2569,6 +2576,7 @@ RETURNS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power) { @@ -2591,7 +2599,9 @@ bfd_check_compression_header (bfd *abfd, bfd_byte *contents, chdr.ch_size = bfd_get_64 (abfd, &echdr->ch_size); chdr.ch_addralign = bfd_get_64 (abfd, &echdr->ch_addralign); } - if (chdr.ch_type == ELFCOMPRESS_ZLIB + *ch_type = chdr.ch_type; + if ((chdr.ch_type == ELFCOMPRESS_ZLIB + || chdr.ch_type == ELFCOMPRESS_ZSTD) && chdr.ch_addralign == (chdr.ch_addralign & -chdr.ch_addralign)) { *uncompressed_size = chdr.ch_size; diff --git a/bfd/compress.c b/bfd/compress.c index b2e39826e38..95c1f4c42d1 100644 --- a/bfd/compress.c +++ b/bfd/compress.c @@ -20,18 +20,31 @@ #include "sysdep.h" #include <zlib.h> +#ifdef HAVE_ZSTD +#include <zstd.h> +#endif #include "bfd.h" +#include "elf-bfd.h" #include "libbfd.h" #include "safe-ctype.h" #define MAX_COMPRESSION_HEADER_SIZE 24 static bool -decompress_contents (bfd_byte *compressed_buffer, +decompress_contents (bool is_zstd, bfd_byte *compressed_buffer, bfd_size_type compressed_size, bfd_byte *uncompressed_buffer, bfd_size_type uncompressed_size) { + if (is_zstd) + { +#ifdef HAVE_ZSTD + size_t ret = ZSTD_decompress (uncompressed_buffer, uncompressed_size, + compressed_buffer, compressed_size); + return !ZSTD_isError (ret); +#endif + } + z_stream strm; int rc; @@ -69,7 +82,7 @@ decompress_contents (bfd_byte *compressed_buffer, } /* Compress data of the size specified in @var{uncompressed_size} - and pointed to by @var{uncompressed_buffer} using zlib and store + and pointed to by @var{uncompressed_buffer} using zlib/zstd and store as the contents field. This function assumes the contents field was allocated using bfd_malloc() or equivalent. @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, sec->size = orig_uncompressed_size; if (decompress) { - if (!decompress_contents (uncompressed_buffer - + orig_compression_header_size, - zlib_size, buffer, buffer_size)) + if (!decompress_contents ( + sec->compress_status == DECOMPRESS_SECTION_ZSTD, + uncompressed_buffer + orig_compression_header_size, + zlib_size, buffer, buffer_size)) { bfd_set_error (bfd_error_bad_value); bfd_release (abfd, buffer); @@ -175,10 +189,23 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, } else { - if (compress ((Bytef*) buffer + header_size, - &compressed_size, - (const Bytef*) uncompressed_buffer, - uncompressed_size) != Z_OK) + if (abfd->flags & BFD_COMPRESS_ZSTD) + { +#if HAVE_ZSTD + compressed_size = ZSTD_compress ( + buffer + header_size, compressed_size, uncompressed_buffer, + uncompressed_size, ZSTD_CLEVEL_DEFAULT); + if (ZSTD_isError (compressed_size)) + { + bfd_release (abfd, buffer); + bfd_set_error (bfd_error_bad_value); + return 0; + } +#endif + } + else if (compress ((Bytef *)buffer + header_size, &compressed_size, + (const Bytef *)uncompressed_buffer, uncompressed_size) + != Z_OK) { bfd_release (abfd, buffer); bfd_set_error (bfd_error_bad_value); @@ -237,6 +264,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) bfd_size_type save_rawsize; bfd_byte *compressed_buffer; unsigned int compression_header_size; + const unsigned int compress_status = sec->compress_status; if (abfd->direction != write_direction && sec->rawsize != 0) sz = sec->rawsize; @@ -248,7 +276,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) return true; } - switch (sec->compress_status) + switch (compress_status) { case COMPRESS_SECTION_NONE: if (p == NULL) @@ -298,7 +326,8 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) *ptr = p; return true; - case DECOMPRESS_SECTION_SIZED: + case DECOMPRESS_SECTION_ZLIB: + case DECOMPRESS_SECTION_ZSTD: /* Read in the full compressed section contents. */ compressed_buffer = (bfd_byte *) bfd_malloc (sec->compressed_size); if (compressed_buffer == NULL) @@ -316,7 +345,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Restore rawsize and size. */ sec->rawsize = save_rawsize; sec->size = save_size; - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = compress_status; if (!ret) goto fail_compressed; @@ -330,8 +359,10 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Set header size to the zlib header size if it is a SHF_COMPRESSED section. */ compression_header_size = 12; - if (!decompress_contents (compressed_buffer + compression_header_size, - sec->compressed_size - compression_header_size, p, sz)) + if (!decompress_contents (compress_status == DECOMPRESS_SECTION_ZSTD, + compressed_buffer + compression_header_size, + sec->compressed_size - compression_header_size, + p, sz)) { bfd_set_error (bfd_error_bad_value); if (p != *ptr) @@ -381,7 +412,8 @@ DESCRIPTION void bfd_cache_section_contents (asection *sec, void *contents) { - if (sec->compress_status == DECOMPRESS_SECTION_SIZED) + if (sec->compress_status == DECOMPRESS_SECTION_ZLIB + || sec->compress_status == DECOMPRESS_SECTION_ZSTD) sec->compress_status = COMPRESS_SECTION_DONE; sec->contents = contents; sec->flags |= SEC_IN_MEMORY; @@ -418,6 +450,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, int compression_header_size; int header_size; unsigned int saved = sec->compress_status; + unsigned int ch_type; bool compressed; *uncompressed_align_pow_p = 0; @@ -448,7 +481,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, { if (compression_header_size != 0) { - if (!bfd_check_compression_header (abfd, header, sec, + if (!bfd_check_compression_header (abfd, header, sec, &ch_type, uncompressed_size_p, uncompressed_align_pow_p)) compression_header_size = -1; @@ -507,7 +540,7 @@ SYNOPSIS DESCRIPTION Record compressed section size, update section size with decompressed size and set compress_status to - DECOMPRESS_SECTION_SIZED. + DECOMPRESS_SECTION_{ZLIB,ZSTD}. Return @code{FALSE} if the section is not a valid compressed section. Otherwise, return @code{TRUE}. @@ -521,6 +554,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) int header_size; bfd_size_type uncompressed_size; unsigned int uncompressed_alignment_power = 0; + unsigned int ch_type; z_stream strm; compression_header_size = bfd_get_compression_header_size (abfd, sec); @@ -550,6 +584,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) uncompressed_size = bfd_getb64 (header + 4); } else if (!bfd_check_compression_header (abfd, header, sec, + &ch_type, &uncompressed_size, &uncompressed_alignment_power)) { @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) sec->compressed_size = sec->size; sec->size = uncompressed_size; bfd_set_section_alignment (sec, uncompressed_alignment_power); - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD + : DECOMPRESS_SECTION_ZLIB; return true; } diff --git a/bfd/config.in b/bfd/config.in index f54a3cacbea..a59304e0a66 100644 --- a/bfd/config.in +++ b/bfd/config.in @@ -232,6 +232,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/bfd/configure b/bfd/configure index 075d2ee0a1b..d905a7bf7a8 100755 --- a/bfd/configure +++ b/bfd/configure @@ -652,6 +652,11 @@ TDEFINES SHARED_LIBADD SHARED_LDFLAGS LIBM +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir EXEEXT_FOR_BUILD @@ -839,6 +844,7 @@ enable_maintainer_mode enable_install_libbfd enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -848,7 +854,12 @@ CFLAGS LDFLAGS LIBS CPPFLAGS -CPP' +CPP +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1511,6 +1522,8 @@ Optional Packages: Binutils" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1521,6 +1534,13 @@ Some influential environment variables: CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if you have headers in a nonstandard directory <include dir> CPP C preprocessor + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -11086,7 +11106,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11089 "configure" +#line 11109 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11192,7 +11212,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11195 "configure" +#line 11215 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -12995,7 +13015,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. # Use the system's zlib library. @@ -13015,6 +13035,246 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" { $as_echo "$as_me:${as_lineno-$LINENO}: checking compiler support for hidden visibility" >&5 diff --git a/bfd/configure.ac b/bfd/configure.ac index 28f3d1afce6..d1b2f76b31a 100644 --- a/bfd/configure.ac +++ b/bfd/configure.ac @@ -230,9 +230,10 @@ AC_CHECK_DECLS([basename, ffs, stpcpy, asprintf, vasprintf, strnlen]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. AM_ZLIB +AM_ZSTD save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" diff --git a/bfd/elf.c b/bfd/elf.c index 16cea4f8aeb..e52fa458a5a 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -1260,6 +1260,18 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, abfd, name); return false; } +#ifndef HAVE_ZSTD + if (newsect->compress_status == DECOMPRESS_SECTION_ZSTD) + { + _bfd_error_handler + /* xgettext:c-format */ + (_ ("%pB: section %s is compressed with zstd, but BFD " + "is not built with zstd support"), + abfd, name); + newsect->compress_status = COMPRESS_SECTION_NONE; + return false; + } +#endif } if (abfd->is_linker_input) diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h index feaba84bf2e..ca600bb5ddf 100644 --- a/bfd/elfxx-target.h +++ b/bfd/elfxx-target.h @@ -989,7 +989,8 @@ const bfd_target TARGET_BIG_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY @@ -1093,7 +1094,8 @@ const bfd_target TARGET_LITTLE_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY diff --git a/bfd/section.c b/bfd/section.c index c7a02d729f2..614570e976e 100644 --- a/bfd/section.c +++ b/bfd/section.c @@ -392,7 +392,8 @@ CODE_FRAGMENT . unsigned int compress_status : 2; .#define COMPRESS_SECTION_NONE 0 .#define COMPRESS_SECTION_DONE 1 -.#define DECOMPRESS_SECTION_SIZED 2 +.#define DECOMPRESS_SECTION_ZLIB 2 +.#define DECOMPRESS_SECTION_ZSTD 3 . . {* The following flags are used by the ELF linker. *} . diff --git a/binutils/Makefile.in b/binutils/Makefile.in index 78d32b350e3..6de4e239408 100644 --- a/binutils/Makefile.in +++ b/binutils/Makefile.in @@ -156,7 +156,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 \ @@ -575,6 +576,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ diff --git a/binutils/NEWS b/binutils/NEWS index 8c2c416c17e..073e9bc4647 100644 --- a/binutils/NEWS +++ b/binutils/NEWS @@ -1,5 +1,11 @@ -*- text -*- +* objcopy --decompress-debug-sections now support zstd compressed debug + sections. The new option --compress-debug-sections=zstd compresses debug + sections with zstd. + +* addr2line and objdump --dwarf now support zstd compressed debug sections. + * The dlltool program now accepts --deterministic-libraries and --non-deterministic-libraries as command line options to control whether or not it generates deterministic output libraries. If neither of these options diff --git a/binutils/aclocal.m4 b/binutils/aclocal.m4 index a877fa7f873..28271f56279 100644 --- a/binutils/aclocal.m4 +++ b/binutils/aclocal.m4 @@ -1205,6 +1205,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/binutils/config.in b/binutils/config.in index c5fb919aa95..bee8c07e2f7 100644 --- a/binutils/config.in +++ b/binutils/config.in @@ -157,6 +157,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define as const if the declaration of iconv() needs const. */ #undef ICONV_CONST diff --git a/binutils/configure b/binutils/configure index 1c518227f57..54d0f184da1 100755 --- a/binutils/configure +++ b/binutils/configure @@ -650,6 +650,8 @@ LTLIBICONV LIBICONV MSGPACK_LIBS MSGPACK_CFLAGS +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir DEMANGLER_NAME @@ -832,6 +834,7 @@ enable_build_warnings enable_nls enable_maintainer_mode with_system_zlib +with_zstd with_msgpack enable_rpath with_libiconv_prefix @@ -853,6 +856,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS MSGPACK_CFLAGS MSGPACK_LIBS' @@ -1517,6 +1522,8 @@ Optional Packages: --with-debuginfod Enable debuginfo lookups with debuginfod (auto/yes/no) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-msgpack Enable msgpack support (auto/yes/no) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib @@ -1547,6 +1554,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config MSGPACK_CFLAGS C compiler flags for MSGPACK, overriding pkg-config MSGPACK_LIBS @@ -10804,7 +10813,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10807 "configure" +#line 10816 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10910,7 +10919,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10913 "configure" +#line 10922 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13468,7 +13477,7 @@ cat >>confdefs.h <<_ACEOF _ACEOF -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). @@ -13490,6 +13499,126 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + + case "${host}" in *-*-msdos* | *-*-go32* | *-*-mingw32* | *-*-cygwin* | *-*-windows*) diff --git a/binutils/configure.ac b/binutils/configure.ac index ec002d3f88f..c152b7f6733 100644 --- a/binutils/configure.ac +++ b/binutils/configure.ac @@ -265,10 +265,11 @@ fi AC_CHECK_DECLS([asprintf, environ, getc_unlocked, stpcpy, strnlen]) -# Link in zlib if we can. This allows us to read compressed debug +# Link in zlib/zstd if we can. This allows us to read compressed debug # sections. This is used only by readelf.c (objdump uses bfd for # reading compressed sections). AM_ZLIB +AM_ZSTD BFD_BINARY_FOPEN diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi index 1499db5728c..34a4164eb94 100644 --- a/binutils/doc/binutils.texi +++ b/binutils/doc/binutils.texi @@ -2159,21 +2159,23 @@ ELF ABI. Note - if compression would actually make a section @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd For ELF files, these options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--decompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug} instead of @samp{.debug}. Note - if compression would -actually make a section @emph{larger}, then it is not compressed nor -renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @item --decompress-debug-sections -Decompress DWARF debug sections using zlib. The original section -names of the compressed sections are restored. +Decompress DWARF debug sections. For a @samp{.zdebug} section, the original +name is restored. @item --elf-stt-common=yes @itemx --elf-stt-common=no diff --git a/binutils/objcopy.c b/binutils/objcopy.c index 43261756a42..fc668f00bbc 100644 --- a/binutils/objcopy.c +++ b/binutils/objcopy.c @@ -232,7 +232,8 @@ static enum compress_zlib = compress | 1 << 1, compress_gnu_zlib = compress | 1 << 2, compress_gabi_zlib = compress | 1 << 3, - decompress = 1 << 4 + compress_zstd = compress | 1 << 4, + decompress = 1 << 5 } do_debug_sections = nothing; /* Whether to generate ELF common symbols with the STT_COMMON type. */ @@ -678,8 +679,8 @@ copy_usage (FILE *stream, int exit_status) <commit>\n\ --subsystem <name>[:<version>]\n\ Set PE subsystem to <name> [& <version>]\n\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ - Compress DWARF debug sections using zlib\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ + Compress DWARF debug sections\n\ --decompress-debug-sections Decompress DWARF debug sections using zlib\n\ --elf-stt-common=[yes|no] Generate ELF common symbols with STT_COMMON\n\ type\n\ @@ -2659,7 +2660,8 @@ copy_object (bfd *ibfd, bfd *obfd, const bfd_arch_info_type *input_arch) if ((do_debug_sections & compress) != 0 && do_debug_sections != compress) { - non_fatal (_("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi] is unsupported on `%s'"), + non_fatal (_ ("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi|" + "zstd] is unsupported on `%s'"), bfd_get_archive_filename (ibfd)); return false; } @@ -3807,6 +3809,13 @@ copy_file (const char *input_filename, const char *output_filename, int ofd, if (do_debug_sections != compress_gnu_zlib) ibfd->flags |= BFD_COMPRESS_GABI; break; + case compress_zstd: + ibfd->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; +#ifndef HAVE_ZSTD + fatal (_ ("--compress-debug-sections=zstd: binutils is not built with " + "zstd support")); +#endif + break; case decompress: ibfd->flags |= BFD_DECOMPRESS; break; @@ -5469,6 +5478,8 @@ copy_main (int argc, char *argv[]) do_debug_sections = compress_gnu_zlib; else if (strcasecmp (optarg, "zlib-gabi") == 0) do_debug_sections = compress_gabi_zlib; + else if (strcasecmp (optarg, "zstd") == 0) + do_debug_sections = compress_zstd; else fatal (_("unrecognized --compress-debug-sections type `%s'"), optarg); diff --git a/binutils/testsuite/binutils-all/compress.exp b/binutils/testsuite/binutils-all/compress.exp index c88da74a987..4e74c824664 100644 --- a/binutils/testsuite/binutils-all/compress.exp +++ b/binutils/testsuite/binutils-all/compress.exp @@ -576,6 +576,50 @@ if { [regexp_diff objdump.out $srcdir/$subdir/dw2-3gabi.W] } then { pass "$testname" } +if { [binutils_assemble_flags $srcdir/$subdir/dw2-1.S ${compressedfile}zstd.o --compress-debug-sections=zstd] } then { + set testname "objcopy compress debug sections with zstd" + set got [binutils_run $OBJCOPY "--compress-debug-sections=zstd ${testfile}.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${compressedfile}zstd.o ${copyfile}zstd.o\n" + verbose "cmp ${compressedfile}zstd.o ${copyfile}zstd.o" + set src1 ${compressedfile}zstd.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } + + set testname "objcopy decompress compressed debug sections with zstd" + set got [binutils_run $OBJCOPY "--decompress-debug-sections ${compressedfile}zstd.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${testfile}.o ${copyfile}zstd.o\n" + verbose "cmp ${testfile}.o ${copyfile}zstd.o" + set src1 ${testfile}.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } +} + proc convert_test { testname as_flags objcop_flags } { global srcdir global subdir diff --git a/config/zstd.m4 b/config/zstd.m4 new file mode 100644 index 00000000000..93ae17b89d3 --- /dev/null +++ b/config/zstd.m4 @@ -0,0 +1,24 @@ +dnl Copyright (C) 2022 Free Software Foundation, Inc. +dnl This file is free software, distributed under the terms of the GNU +dnl General Public License. As a special exception to the GNU General +dnl Public License, this file may be distributed as part of a program +dnl that contains a configuration script generated by Autoconf, under +dnl the same distribution terms as the rest of that program. + +dnl Enable features using the zstd library. +AC_DEFUN([AM_ZSTD], +[ +AC_ARG_WITH(zstd, + [AS_HELP_STRING([--with-zstd], [support zstd compressed debug sections (default=auto)])], + [], [with_zstd=auto]) + +if test "$with_zstd" != no; then + PKG_CHECK_MODULES(ZSTD, [libzstd], [ + AC_DEFINE(HAVE_ZSTD, 1, [Define to 1 if zstd is enabled.]) + ], [ + if test "$with_zstd" = yes; then + AC_MSG_ERROR([--with-zstd was given, but pkgconfig/libzstd.pc is not found]) + fi + ]) +fi +]) diff --git a/configure b/configure index d75f47a1e95..f14e0efd675 100755 --- a/configure +++ b/configure @@ -785,6 +785,7 @@ ac_user_opts=' enable_option_checking with_build_libsubdir with_system_zlib +with_zstd enable_as_accelerator_for enable_offload_targets enable_gold @@ -1567,6 +1568,8 @@ Optional Packages: --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-build-libsubdir=DIR Directory where to find libraries for build system --with-system-zlib use installed libz + --with-zstd Support zstd compressed debug sections + (default=auto) --with-mpc=PATH specify prefix directory for installed MPC package. Equivalent to --with-mpc-include=PATH/include plus --with-mpc-lib=PATH/lib @@ -2925,6 +2928,13 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +fi + + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/configure.ac b/configure.ac index ae18d436aca..0152c69292e 100644 --- a/configure.ac +++ b/configure.ac @@ -246,6 +246,9 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi +AC_ARG_WITH(zstd, +[AS_HELP_STRING([--with-zstd], [Support zstd compressed debug sections (default=auto)])]) + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/gas/Makefile.am b/gas/Makefile.am index bd597398671..5f0f24abf8d 100644 --- a/gas/Makefile.am +++ b/gas/Makefile.am @@ -42,7 +42,7 @@ am__skipyacc = WARN_CFLAGS = @WARN_CFLAGS@ @WARN_WRITE_STRINGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c @@ -407,7 +407,7 @@ STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) EXTRA_as_new_SOURCES = $(CFILES) $(HFILES) $(TARGET_CPU_CFILES) \ diff --git a/gas/Makefile.in b/gas/Makefile.in index c57d78f82c4..5a4dd702252 100644 --- a/gas/Makefile.in +++ b/gas/Makefile.in @@ -140,10 +140,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/lead-dot.m4 \ $(top_srcdir)/../config/nls.m4 \ $(top_srcdir)/../config/override.m4 \ + $(top_srcdir)/../config/pkg.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/../bfd/version.m4 \ @@ -429,6 +431,9 @@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ POSUB = @POSUB@ RANLIB = @RANLIB@ SED = @SED@ @@ -443,6 +448,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ] ; then echo ../bison/bison -y -L../bison/bison ; else echo @YACC@ ; fi` YFLAGS = @YFLAGS@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -524,7 +531,7 @@ ZLIBINC = @zlibinc@ # maintainer mode is disabled. Avoid this. am__skiplex = am__skipyacc = -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) TARG_CPU = @target_cpu_type@ TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c TARG_CPU_O = config/tc-@target_cpu_type@.@OBJEXT@ @@ -874,7 +881,7 @@ GASLIBS = @OPCODES_LIB@ ../bfd/libbfd.la ../libiberty/libiberty.a STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) diff --git a/gas/NEWS b/gas/NEWS index d61cdb9edd4..9a8b726b942 100644 --- a/gas/NEWS +++ b/gas/NEWS @@ -1,5 +1,8 @@ -*- text -*- +* gas now supports --compress-debug-sections=zstd to compress + debug sections with zstd. + Changes in 2.39: * Remove (rudimentary) support for the x86-64 sub-architectures Intel L1OM and diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 index 70183124da7..722030c776f 100644 --- a/gas/aclocal.m4 +++ b/gas/aclocal.m4 @@ -1196,10 +1196,12 @@ m4_include([../config/lcmessage.m4]) m4_include([../config/lead-dot.m4]) m4_include([../config/nls.m4]) m4_include([../config/override.m4]) +m4_include([../config/pkg.m4]) m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/gas/as.c b/gas/as.c index 6268779cf90..35ad6b3ab3b 100644 --- a/gas/as.c +++ b/gas/as.c @@ -252,14 +252,14 @@ Options:\n\ --alternate initially turn on alternate macro syntax\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib [default]\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ don't compress DWARF debug sections\n")); #else fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ @@ -736,6 +736,15 @@ This program has absolutely no warranty.\n")); flag_compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + flag_compress_debug = COMPRESS_DEBUG_ZSTD; +#else + as_fatal (_ ("--compress-debug-sections=zstd: gas is not " + "built with zstd support")); +#endif + } else as_fatal (_("Invalid --compress-debug-sections option: `%s'"), optarg); diff --git a/gas/compress-debug.c b/gas/compress-debug.c index c80dbeec4e7..3cd175f6e57 100644 --- a/gas/compress-debug.c +++ b/gas/compress-debug.c @@ -21,14 +21,23 @@ #include "config.h" #include <stdio.h> #include <zlib.h> +#if HAVE_ZSTD +#include <zstd.h> +#endif #include "ansidecl.h" #include "compress-debug.h" /* Initialize the compression engine. */ -struct z_stream_s * -compress_init (void) +void * +compress_init (bool use_zstd) { + if (use_zstd) { +#if HAVE_ZSTD + return ZSTD_createCCtx (); +#endif + } + static struct z_stream_s strm; strm.zalloc = NULL; @@ -42,22 +51,37 @@ compress_init (void) from the engine goes into the current frag on the obstack. */ int -compress_data (struct z_stream_s *strm, const char **next_in, - int *avail_in, char **next_out, int *avail_out) +compress_data (bool use_zstd, void *ctx, const char **next_in, int *avail_in, + char **next_out, int *avail_out) { - int out_size = 0; - int x; + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { *next_in, *avail_in, 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_continue); + *next_in += ib.pos; + *avail_in -= ib.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + return (int)ob.pos; +#endif + } + + struct z_stream_s *strm = ctx; strm->next_in = (Bytef *) (*next_in); strm->avail_in = *avail_in; strm->next_out = (Bytef *) (*next_out); strm->avail_out = *avail_out; - x = deflate (strm, Z_NO_FLUSH); + int x = deflate (strm, Z_NO_FLUSH); if (x != Z_OK) return -1; - out_size = *avail_out - strm->avail_out; + int out_size = *avail_out - strm->avail_out; *next_in = (char *) (strm->next_in); *avail_in = strm->avail_in; *next_out = (char *) (strm->next_out); @@ -71,10 +95,28 @@ compress_data (struct z_stream_s *strm, const char **next_in, needed. */ int -compress_finish (struct z_stream_s *strm, char **next_out, +compress_finish (bool use_zstd, void *ctx, char **next_out, int *avail_out, int *out_size) { + if (use_zstd) + { +#if HAVE_ZSTD + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_end); + *out_size = ob.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + if (ret == 0) + ZSTD_freeCCtx (ctx); + return ret ? 1 : 0; +#endif + } + int x; + struct z_stream_s *strm = ctx; strm->avail_in = 0; strm->next_out = (Bytef *) (*next_out); diff --git a/gas/compress-debug.h b/gas/compress-debug.h index 0183e269fef..87de0f8444e 100644 --- a/gas/compress-debug.h +++ b/gas/compress-debug.h @@ -21,19 +21,19 @@ #ifndef COMPRESS_DEBUG_H #define COMPRESS_DEBUG_H +#include <stdbool.h> + struct z_stream_s; /* Initialize the compression engine. */ -extern struct z_stream_s * -compress_init (void); +extern void *compress_init (bool); /* Stream the contents of a frag to the compression engine. Output from the engine goes into the current frag on the obstack. */ -extern int -compress_data (struct z_stream_s *, const char **, int *, char **, int *); +extern int compress_data (bool, void *, const char **, int *, char **, int *); /* Finish the compression and consume the remaining compressed output. */ extern int -compress_finish (struct z_stream_s *, char **, int *, int *); +compress_finish (bool, void *, char **, int *, int *); #endif /* COMPRESS_DEBUG_H */ diff --git a/gas/config.in b/gas/config.in index e243fd277ee..0d1668a3eac 100644 --- a/gas/config.in +++ b/gas/config.in @@ -134,6 +134,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Using i386 COFF? */ #undef I386COFF diff --git a/gas/configure b/gas/configure index d0449a1d7ab..1833adffaa8 100755 --- a/gas/configure +++ b/gas/configure @@ -633,6 +633,11 @@ ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS LIBOBJS +ZSTD_LIBS +ZSTD_CFLAGS +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG zlibinc zlibdir LIBM @@ -817,6 +822,7 @@ with_cpu enable_nls enable_maintainer_mode with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -828,7 +834,12 @@ LIBS CPPFLAGS CPP YACC -YFLAGS' +YFLAGS +PKG_CONFIG +PKG_CONFIG_PATH +PKG_CONFIG_LIBDIR +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1493,6 +1504,8 @@ Optional Packages: --with-cpu=CPU default cpu variant is CPU (currently only supported on ARC) --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1509,6 +1522,13 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + PKG_CONFIG path to pkg-config utility + PKG_CONFIG_PATH + directories to add to pkg-config's search path + PKG_CONFIG_LIBDIR + path overriding pkg-config's built-in search path + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -10702,7 +10722,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10705 "configure" +#line 10725 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10808,7 +10828,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10811 "configure" +#line 10831 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13945,7 +13965,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -13964,6 +13984,246 @@ fi + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + # Support for VMS timestamps via cross compile if test "$ac_cv_header_time_h" = yes; then diff --git a/gas/configure.ac b/gas/configure.ac index 21795a82bdd..8750ea21381 100644 --- a/gas/configure.ac +++ b/gas/configure.ac @@ -1004,8 +1004,9 @@ AC_CHECK_DECLS([asprintf, mempcpy, stpcpy]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. AM_ZLIB +AM_ZSTD # Support for VMS timestamps via cross compile diff --git a/gas/doc/as.texi b/gas/doc/as.texi index e915893da86..01f49434021 100644 --- a/gas/doc/as.texi +++ b/gas/doc/as.texi @@ -711,16 +711,19 @@ given section @emph{larger} then it is not compressed. @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd These options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--nocompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug}. Note if compression would make a given section -@emph{larger} then it is not compressed nor renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @end ifset diff --git a/gas/write.c b/gas/write.c index f76bbdb706e..0e49df7c03f 100644 --- a/gas/write.c +++ b/gas/write.c @@ -1413,7 +1413,7 @@ write_relocs (bfd *abfd ATTRIBUTE_UNUSED, asection *sec, } static int -compress_frag (struct z_stream_s *strm, const char *contents, int in_size, +compress_frag (bool use_zstd, void *ctx, const char *contents, int in_size, fragS **last_newf, struct obstack *ob) { int out_size; @@ -1442,10 +1442,10 @@ compress_frag (struct z_stream_s *strm, const char *contents, int in_size, as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - out_size = compress_data (strm, &contents, &in_size, - &next_out, &avail_out); + out_size = compress_data (use_zstd, ctx, &contents, &in_size, &next_out, + &avail_out); if (out_size < 0) - return -1; + return -1; f->fr_fix += out_size; total_out_size += out_size; @@ -1471,7 +1471,6 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) const char *section_name; char *compressed_name; char *header; - struct z_stream_s *strm; int x; flagword flags = bfd_section_flags (sec); unsigned int header_size, compression_header_size; @@ -1485,21 +1484,22 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) if (!startswith (section_name, ".debug_")) return; - strm = compress_init (); - if (strm == NULL) + bool use_zstd = abfd->flags & BFD_COMPRESS_ZSTD; + void *ctx = compress_init (use_zstd); + if (ctx == NULL) return; - if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) + if (flag_compress_debug == COMPRESS_DEBUG_GNU_ZLIB) + { + compression_header_size = 0; + header_size = 12; + } + else { compression_header_size = bfd_get_compression_header_size (stdoutput, NULL); header_size = compression_header_size; } - else - { - compression_header_size = 0; - header_size = 12; - } /* Create a new frag to contain the compression header. */ first_newf = frag_alloc (ob); @@ -1531,7 +1531,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) gas_assert (f->fr_type == rs_fill); if (f->fr_fix) { - out_size = compress_frag (strm, f->fr_literal, f->fr_fix, + out_size = compress_frag (use_zstd, ctx, f->fr_literal, f->fr_fix, &last_newf, ob); if (out_size < 0) return; @@ -1545,8 +1545,8 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) { while (count--) { - out_size = compress_frag (strm, fill_literal, (int) fill_size, - &last_newf, ob); + out_size = compress_frag (use_zstd, ctx, fill_literal, + (int)fill_size, &last_newf, ob); if (out_size < 0) return; compressed_size += out_size; @@ -1579,7 +1579,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - x = compress_finish (strm, &next_out, &avail_out, &out_size); + x = compress_finish (use_zstd, ctx, &next_out, &avail_out, &out_size); if (x < 0) return; @@ -2540,6 +2540,8 @@ write_object_file (void) { if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI; + else if (flag_compress_debug == COMPRESS_DEBUG_ZSTD) + stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; else stdoutput->flags |= BFD_COMPRESS; bfd_map_over_sections (stdoutput, compress_debug, (char *) 0); diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 2598b81d205..c528ee5aa80 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -173,6 +173,9 @@ BFD_CFLAGS = -I$(BFD_DIR) -I$(BFD_SRC) ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ + # Where is the decnumber library? Typically in ../libdecnumber. LIBDECNUMBER_DIR = ../libdecnumber LIBDECNUMBER = $(LIBDECNUMBER_DIR)/libdecnumber.a @@ -625,7 +628,7 @@ INTERNAL_CPPFLAGS = $(CPPFLAGS) @GUILE_CPPFLAGS@ @PYTHON_CPPFLAGS@ \ INTERNAL_CFLAGS_BASE = \ $(GLOBAL_CFLAGS) $(PROFILE_CFLAGS) \ $(GDB_CFLAGS) $(OPCODES_CFLAGS) $(READLINE_CFLAGS) $(ZLIBINC) \ - $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ + $(ZSTD_CFLAGS) $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ $(INTL_CFLAGS) $(INCGNU) $(INCSUPPORT) $(LIBBACKTRACE_INC) \ $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS) $(SRCHIGH_CFLAGS) \ $(TOP_CFLAGS) $(PTHREAD_CFLAGS) $(DEBUGINFOD_CFLAGS) @@ -647,7 +650,7 @@ INTERNAL_LDFLAGS = \ # Libraries and corresponding dependencies for compiling gdb. # XM_CLIBS, defined in *config files, have host-dependent libs. # LIBIBERTY appears twice on purpose. -CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \ +CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) $(ZSTD_LIBS) \ $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \ $(XM_CLIBS) $(GDBTKLIBS) $(LIBBACKTRACE_LIB) \ @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \ @@ -2298,6 +2301,7 @@ aclocal_m4_deps = \ ../config/lcmessage.m4 \ ../config/codeset.m4 \ ../config/zlib.m4 \ + ../config/zstd.m4 \ ../config/ax_pthread.m4 $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) diff --git a/gdb/NEWS b/gdb/NEWS index 9619842bc03..1457c99ff04 100644 --- a/gdb/NEWS +++ b/gdb/NEWS @@ -57,6 +57,8 @@ * The Windows native target now supports target async. +* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. + * New commands maintenance set ignore-prologue-end-flag on|off diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 index 95ff2b6f35e..28846119dcb 100644 --- a/gdb/acinclude.m4 +++ b/gdb/acinclude.m4 @@ -43,6 +43,7 @@ m4_include([../config/lib-link.m4]) m4_include([../config/iconv.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../gdbsupport/common.m4]) @@ -233,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" AC_CACHE_CHECK( diff --git a/gdb/config.in b/gdb/config.in index a4975c68aad..e13a409ec2d 100644 --- a/gdb/config.in +++ b/gdb/config.in @@ -589,6 +589,9 @@ /* Define to 1 if you have the `XML_StopParser' function. */ #undef HAVE_XML_STOPPARSER +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to 1 if your system has the _etext variable. */ #undef HAVE__ETEXT diff --git a/gdb/configure b/gdb/configure index 4dbd0c3b13c..57cd165088a 100755 --- a/gdb/configure +++ b/gdb/configure @@ -747,6 +747,8 @@ READLINE_DEPS READLINE LTLIBICONV LIBICONV +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir MIG @@ -893,6 +895,7 @@ enable_codesign with_pkgversion with_bugurl with_system_zlib +with_zstd with_gnu_ld enable_rpath with_libiconv_prefix @@ -961,6 +964,8 @@ DEBUGINFOD_CFLAGS DEBUGINFOD_LIBS YACC YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS XMKMF' ac_subdirs_all='testsuite gdbtk' @@ -1641,6 +1646,8 @@ Optional Packages: --with-pkgversion=PKG Use PKG in the version string in place of "GDB" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib --without-libiconv-prefix don't search for libiconv in includedir and libdir @@ -1721,6 +1728,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config XMKMF Path to xmkmf, Makefile generator for X Window System Use these variables to override the choices made by `configure' or to help @@ -8242,7 +8251,8 @@ if test "$ac_res" != no; then : fi -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -8262,6 +8272,126 @@ fi +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + + if test "X$prefix" = "XNONE"; then acl_final_prefix="$ac_default_prefix" else @@ -17281,7 +17411,7 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 @@ -17396,7 +17526,7 @@ fi # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 diff --git a/gdb/configure.ac b/gdb/configure.ac index 215a91c5b27..e8cf808d280 100644 --- a/gdb/configure.ac +++ b/gdb/configure.ac @@ -460,8 +460,10 @@ AC_SEARCH_LIBS(gethostbyname, nsl) # Some systems (e.g. Solaris) have `socketpair' in libsocket. AC_SEARCH_LIBS(socketpair, socket) -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. AM_ZLIB +AM_ZSTD AM_ICONV diff --git a/ld/Makefile.am b/ld/Makefile.am index d31021c13e2..bbe25f3aca2 100644 --- a/ld/Makefile.am +++ b/ld/Makefile.am @@ -45,7 +45,7 @@ ELF_CLFAGS=-DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -959,7 +959,8 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ldbuildid.c ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) # Dependency tracking for the generated emulation files. EXTRA_ld_new_SOURCES += $(ALL_EMULATION_SOURCES) $(ALL_64_EMULATION_SOURCES) diff --git a/ld/Makefile.in b/ld/Makefile.in index ee0c98f65b0..400eed2717e 100644 --- a/ld/Makefile.in +++ b/ld/Makefile.in @@ -126,7 +126,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 $(top_srcdir)/configure.ac @@ -477,6 +478,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ XGETTEXT = @XGETTEXT@ YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` YFLAGS = -d +ZSTD_CFLAGS = @ZSTD_CFLAGS@ +ZSTD_LIBS = @ZSTD_LIBS@ abs_builddir = @abs_builddir@ abs_srcdir = @abs_srcdir@ abs_top_builddir = @abs_top_builddir@ @@ -564,7 +567,7 @@ ELF_CLFAGS = -DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) # We put the scripts in the directory $(scriptdir)/ldscripts. # We can't put the scripts in $(datadir) because the SEARCH_DIR @@ -1011,7 +1014,9 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) + # # # Build a dummy plugin using libtool. diff --git a/ld/NEWS b/ld/NEWS index 355752e6b24..dfe2690d9f2 100644 --- a/ld/NEWS +++ b/ld/NEWS @@ -1,5 +1,8 @@ -*- text -*- +* ld now supports zstd compressed debug sections. The new option + --compress-debug-sections=zstd compresses debug sections with zstd. + Changes in 2.39: * The ELF linker will now generate a warning message if the stack is made diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 index d20c542eed5..893e973e4f2 100644 --- a/ld/aclocal.m4 +++ b/ld/aclocal.m4 @@ -1203,6 +1203,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/ld/config.in b/ld/config.in index 0ccd79d59cd..3916740eee4 100644 --- a/ld/config.in +++ b/ld/config.in @@ -162,6 +162,9 @@ /* Define to 1 if you have the <windows.h> header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if zstd is enabled. */ +#undef HAVE_ZSTD + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/ld/configure b/ld/configure index a1a07005400..175dd995b2b 100755 --- a/ld/configure +++ b/ld/configure @@ -646,6 +646,8 @@ elf_plt_unwind_list_options elf_shlib_list_options elf_list_options STRINGIFY +ZSTD_LIBS +ZSTD_CFLAGS zlibinc zlibdir NATIVE_LIB_DIRS @@ -679,9 +681,6 @@ WARN_CFLAGS_FOR_BUILD WARN_CFLAGS JANSSON_LIBS JANSSON_CFLAGS -PKG_CONFIG_LIBDIR -PKG_CONFIG_PATH -PKG_CONFIG enable_libctf ENABLE_LIBCTF_FALSE ENABLE_LIBCTF_TRUE @@ -711,6 +710,9 @@ LD FGREP SED LIBTOOL +PKG_CONFIG_LIBDIR +PKG_CONFIG_PATH +PKG_CONFIG EGREP CPP GREP @@ -855,6 +857,7 @@ enable_werror enable_build_warnings enable_nls with_system_zlib +with_zstd ' ac_precious_vars='build_alias host_alias @@ -868,14 +871,16 @@ CXX CXXFLAGS CCC CPP -CXXCPP PKG_CONFIG PKG_CONFIG_PATH PKG_CONFIG_LIBDIR +CXXCPP JANSSON_CFLAGS JANSSON_LIBS YACC -YFLAGS' +YFLAGS +ZSTD_CFLAGS +ZSTD_LIBS' # Initialize some variables set by options. @@ -1552,6 +1557,8 @@ Optional Packages: --with-lib-path=dir1:dir2... set default LIB_PATH --with-sysroot=DIR Search for usr/lib et al within DIR. --with-system-zlib use installed libz + --with-zstd support zstd compressed debug sections + (default=auto) Some influential environment variables: CC C compiler command @@ -1564,12 +1571,12 @@ Some influential environment variables: CXX C++ compiler command CXXFLAGS C++ compiler flags CPP C preprocessor - CXXCPP C++ preprocessor PKG_CONFIG path to pkg-config utility PKG_CONFIG_PATH directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path + CXXCPP C++ preprocessor JANSSON_CFLAGS C compiler flags for JANSSON, overriding pkg-config JANSSON_LIBS @@ -1580,6 +1587,8 @@ Some influential environment variables: YFLAGS The list of arguments that will be passed by default to $YACC. This script will default YFLAGS to the empty string to avoid a default value of `-d' given by some make applications. + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -5388,6 +5397,126 @@ $as_echo "$ac_cv_safe_to_define___extensions__" >&6; } + + + + + + +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then + if test -n "$ac_tool_prefix"; then + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +PKG_CONFIG=$ac_cv_path_PKG_CONFIG +if test -n "$PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 +$as_echo "$PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +fi +if test -z "$ac_cv_path_PKG_CONFIG"; then + ac_pt_PKG_CONFIG=$PKG_CONFIG + # Extract the first word of "pkg-config", so it can be a program name with args. +set dummy pkg-config; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $ac_pt_PKG_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG +if test -n "$ac_pt_PKG_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 +$as_echo "$ac_pt_PKG_CONFIG" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + if test "x$ac_pt_PKG_CONFIG" = x; then + PKG_CONFIG="" + else + case $cross_compiling:$ac_tool_warned in +yes:) +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} +ac_tool_warned=yes ;; +esac + PKG_CONFIG=$ac_pt_PKG_CONFIG + fi +else + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" +fi + +fi +if test -n "$PKG_CONFIG"; then + _pkg_min_version=0.9.0 + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + PKG_CONFIG="" + fi +fi + case `pwd` in *\ * | *\ *) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 @@ -11491,7 +11620,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11494 "configure" +#line 11623 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11597,7 +11726,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11600 "configure" +#line 11729 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -15586,126 +15715,6 @@ fi if test "x$enable_jansson" != "xno"; then - - - - - - - -if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. -set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -PKG_CONFIG=$ac_cv_path_PKG_CONFIG -if test -n "$PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 -$as_echo "$PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_path_PKG_CONFIG"; then - ac_pt_PKG_CONFIG=$PKG_CONFIG - # Extract the first word of "pkg-config", so it can be a program name with args. -set dummy pkg-config; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $ac_pt_PKG_CONFIG in - [\\/]* | ?:[\\/]*) - ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG -if test -n "$ac_pt_PKG_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 -$as_echo "$ac_pt_PKG_CONFIG" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_pt_PKG_CONFIG" = x; then - PKG_CONFIG="" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - PKG_CONFIG=$ac_pt_PKG_CONFIG - fi -else - PKG_CONFIG="$ac_cv_path_PKG_CONFIG" -fi - -fi -if test -n "$PKG_CONFIG"; then - _pkg_min_version=0.9.0 - { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 -$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } - if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - PKG_CONFIG="" - fi -fi if test -n "$PKG_CONFIG"; then : @@ -17052,8 +17061,8 @@ $as_echo "#define HAVE_DECL_GETOPT 1" >>confdefs.h fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -17072,6 +17081,126 @@ fi + +# Check whether --with-zstd was given. +if test "${with_zstd+set}" = set; then : + withval=$with_zstd; +else + with_zstd=auto +fi + + +if test "$with_zstd" != no; then + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 +$as_echo_n "checking for libzstd... " >&6; } + +if test -n "$ZSTD_CFLAGS"; then + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$ZSTD_LIBS"; then + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + +if test $pkg_failed = no; then + pkg_save_LDFLAGS="$LDFLAGS" + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +else + pkg_failed=yes +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + LDFLAGS=$pkg_save_LDFLAGS +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` + else + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$ZSTD_PKG_ERRORS" >&5 + + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + if test "$with_zstd" = yes; then + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 + fi + +else + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h + + +fi +fi + + # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string # concatenation, or ostring.sed otherwise. This is to support the diff --git a/ld/configure.ac b/ld/configure.ac index eb55904c090..a491f9f1116 100644 --- a/ld/configure.ac +++ b/ld/configure.ac @@ -34,6 +34,7 @@ AC_PROG_GREP AC_GNU_SOURCE AC_USE_SYSTEM_EXTENSIONS AC_PROG_INSTALL +PKG_PROG_PKG_CONFIG LT_INIT ACX_LARGEFILE @@ -297,7 +298,6 @@ AC_ARG_ENABLE([jansson], [enable_jansson="no"]) if test "x$enable_jansson" != "xno"; then - PKG_PROG_PKG_CONFIG AS_IF([test -n "$PKG_CONFIG"], [ PKG_CHECK_MODULES(JANSSON, [jansson], @@ -386,9 +386,10 @@ if test $ld_cv_decl_getopt_unistd_h = yes; then [Is the prototype for getopt in <unistd.h> in the expected format?]) fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. AM_ZLIB +AM_ZSTD # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string diff --git a/ld/emultempl/elf.em b/ld/emultempl/elf.em index c52484f35d3..acd66f907d1 100644 --- a/ld/emultempl/elf.em +++ b/ld/emultempl/elf.em @@ -668,6 +668,15 @@ gld${EMULATION_NAME}_handle_option (int optc) link_info.compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD + link_info.compress_debug = COMPRESS_DEBUG_ZSTD; +#else + einfo (_ ("%F%P: --compress-debug-sections=zstd: ld is not built " + "with zstd support\n")); +#endif + } else einfo (_("%F%P: invalid --compress-debug-sections option: \`%s'\n"), optarg); diff --git a/ld/ld.texi b/ld/ld.texi index eabbec8faa9..9daed2e7e9f 100644 --- a/ld/ld.texi +++ b/ld/ld.texi @@ -2863,10 +2863,12 @@ but for most Linux based systems it will be @code{both}. @kindex --compress-debug-sections=zlib @kindex --compress-debug-sections=zlib-gnu @kindex --compress-debug-sections=zlib-gabi +@kindex --compress-debug-sections=zstd @item --compress-debug-sections=none @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd On ELF platforms, these options control how DWARF debug sections are compressed using zlib. @@ -2880,6 +2882,9 @@ sets the SHF_COMPRESSED flag in the sections' headers. The @option{--compress-debug-sections=zlib} option is an alias for @option{--compress-debug-sections=zlib-gabi}. +@option{--compress-debug-sections=zstd} compresses DWARF debug sections using +zstd. + Note that this option overrides any compression in input debug sections, so if a binary is linked with @option{--compress-debug-sections=none} for example, then any compressed debug sections in input files will be diff --git a/ld/ldmain.c b/ld/ldmain.c index 1bbddaaad32..71af5935b74 100644 --- a/ld/ldmain.c +++ b/ld/ldmain.c @@ -506,8 +506,12 @@ main (int argc, char **argv) if ((link_info.compress_debug & COMPRESS_DEBUG)) { link_info.output_bfd->flags |= BFD_COMPRESS; - if (link_info.compress_debug == COMPRESS_DEBUG_GABI_ZLIB) - link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug != COMPRESS_DEBUG_GNU_ZLIB) + { + link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug == COMPRESS_DEBUG_ZSTD) + link_info.output_bfd->flags |= BFD_COMPRESS_ZSTD; + } } ldwrite (); diff --git a/ld/lexsup.c b/ld/lexsup.c index 9225f71b3ce..299371fb775 100644 --- a/ld/lexsup.c +++ b/ld/lexsup.c @@ -2146,8 +2146,8 @@ elf_static_list_options (FILE *file) fprintf (file, _("\ --package-metadata[=JSON] Generate package metadata note\n")); fprintf (file, _("\ - --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ - Compress DWARF debug sections using zlib\n")); + --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi|zstd]\n\ + Compress DWARF debug sections\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (file, _("\ Default: zlib-gabi\n")); diff --git a/ld/testsuite/ld-elf/compress.exp b/ld/testsuite/ld-elf/compress.exp index cfdd767b8ab..7b8a31cc41d 100644 --- a/ld/testsuite/ld-elf/compress.exp +++ b/ld/testsuite/ld-elf/compress.exp @@ -254,3 +254,20 @@ if { [regexp_diff tmpdir/$test.out $srcdir/$subdir/$test.rt] } then { } else { pass "$test_name" } + +if { ![ld_assemble $as "--compress-debug-sections=zstd $srcdir/$subdir/empty.s" tmpdir/emptyzstd.o ] } { + return +} +set build_tests { + {"Build libzstdfoo.so with zstd compressed debug sections" + "-shared" "-fPIC -g -Wa,--compress-debug-sections=zstd -Wl,--compress-debug-sections=zstd" + {foo.c} {} "libzstdfoo.so"} +} +set run_tests { + {"Run zstdnormal with libzstdfoo.so with zstd compressed debug sections" + "tmpdir/begin.o tmpdir/libzstdfoo.so tmpdir/end.o -Wl,--compress-debug-sections=zstd" "" + {main.c} "zstdnormal" "normal.out" "-Wa,--compress-debug-sections=zstd"} +} + +run_cc_link_tests $build_tests +run_ld_link_exec_tests $run_tests -- 2.37.3.998.g577e59143f-goog ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 18:20 ` Fangrui Song @ 2022-09-23 18:57 ` Simon Marchi 2022-09-23 20:34 ` Fangrui Song 0 siblings, 1 reply; 25+ messages in thread From: Simon Marchi @ 2022-09-23 18:57 UTC (permalink / raw) To: Fangrui Song, Nick Clifton; +Cc: Alan Modra, Jan Beulich, binutils, gdb-patches On 2022-09-23 14:20, Fangrui Song wrote: > On 2022-09-23, Simon Marchi wrote: >> >> >> On 2022-09-23 11:45, Nick Clifton wrote: >>> Hi Fangrui, >>> >>>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>>> compressed debug sections for most tools. > > Hi Simon and Nick, > > I have checked that `make all-gdb` works in a --enable-gdb --without-debuginfod build. > > I then inspected the 2020 commits adding AC_DEBUGINFOD to > gdb/configure.ac and binutils/configure.ac. They come with no > PKG_PROG_PKG_CONFIG and config/debuginfod.m4 does not have > PKG_PROG_PKG_CONFIG, but they still work. So I think AC_ZSTD doesn't > need PKG_PROG_PKG_CONFIG, either. Though I have no insight why it > behaves that way:) Ok, I am a bit confused. Based on my comprehension, since the first call to PKG_CHECK_MODULES may not happen do to a condition: if test "x$with_debuginfod" != xno; then PKG_CHECK_MODULES([DEBUGINFOD], [libdebuginfod >= 0.179], ... fi ... the probing for pkg-config should not happen if --without-debuginfod is used. But in gdb/configure, I see that the probing for pkg-config actually appears outside the condition. Anyway, good if it works. > +diff --git a/gdb/NEWS b/gdb/NEWS > +index 9619842bc03..1457c99ff04 100644 > +--- a/gdb/NEWS > ++++ b/gdb/NEWS > +@@ -57,6 +57,8 @@ > + + * The Windows native target now supports target async. > + ++* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. > ++ I'm not sure I understand the diff above, but I think it's missing an empty line between the previous bullet and the new bullet. Otherwise, that is OK for GDB. Simon ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 18:57 ` Simon Marchi @ 2022-09-23 20:34 ` Fangrui Song 2022-09-24 5:43 ` Eli Zaretskii 0 siblings, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-23 20:34 UTC (permalink / raw) To: Simon Marchi; +Cc: Nick Clifton, Alan Modra, Jan Beulich, binutils, gdb-patches On 2022-09-23, Simon Marchi wrote: > > >On 2022-09-23 14:20, Fangrui Song wrote: >> On 2022-09-23, Simon Marchi wrote: >>> >>> >>> On 2022-09-23 11:45, Nick Clifton wrote: >>>> Hi Fangrui, >>>> >>>>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>>>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>>>> compressed debug sections for most tools. >> >> Hi Simon and Nick, >> >> I have checked that `make all-gdb` works in a --enable-gdb --without-debuginfod build. >> >> I then inspected the 2020 commits adding AC_DEBUGINFOD to >> gdb/configure.ac and binutils/configure.ac. They come with no >> PKG_PROG_PKG_CONFIG and config/debuginfod.m4 does not have >> PKG_PROG_PKG_CONFIG, but they still work. So I think AC_ZSTD doesn't >> need PKG_PROG_PKG_CONFIG, either. Though I have no insight why it >> behaves that way:) > >Ok, I am a bit confused. Based on my comprehension, since the first >call to PKG_CHECK_MODULES may not happen do to a condition: > > if test "x$with_debuginfod" != xno; then > PKG_CHECK_MODULES([DEBUGINFOD], [libdebuginfod >= 0.179], > ... > fi > >... the probing for pkg-config should not happen if --without-debuginfod >is used. But in gdb/configure, I see that the probing for pkg-config >actually appears outside the condition. > >Anyway, good if it works. > >> +diff --git a/gdb/NEWS b/gdb/NEWS >> +index 9619842bc03..1457c99ff04 100644 >> +--- a/gdb/NEWS >> ++++ b/gdb/NEWS >> +@@ -57,6 +57,8 @@ >> + + * The Windows native target now supports target async. >> + ++* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. >> ++ > >I'm not sure I understand the diff above, but I think it's missing an >empty line between the previous bullet and the new bullet. Otherwise, >that is OK for GDB. > >Simon Thanks! The gdb news entry looks like (immediately before "* New commands") --- a/gdb/NEWS +++ b/gdb/NEWS @@ -57,6 +57,8 @@ * The Windows native target now supports target async. +* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. + * New commands maintenance set ignore-prologue-end-flag on|off ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 20:34 ` Fangrui Song @ 2022-09-24 5:43 ` Eli Zaretskii 0 siblings, 0 replies; 25+ messages in thread From: Eli Zaretskii @ 2022-09-24 5:43 UTC (permalink / raw) To: Fangrui Song; +Cc: simon.marchi, gdb-patches, binutils, nickc, jbeulich, amodra > Date: Fri, 23 Sep 2022 13:34:17 -0700 > From: Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> > Cc: gdb-patches@sourceware.org, binutils@sourceware.org, > Nick Clifton <nickc@redhat.com>, Jan Beulich <jbeulich@suse.com>, > Alan Modra <amodra@gmail.com> > > Thanks! The gdb news entry looks like (immediately before "* New commands") > > --- a/gdb/NEWS > +++ b/gdb/NEWS > @@ -57,6 +57,8 @@ > > * The Windows native target now supports target async. > > +* gdb now supports zstd compressed debug sections (ELFCOMPRESS_ZSTD) for ELF. > + > * New commands > > maintenance set ignore-prologue-end-flag on|off This is OK, thanks. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song 2022-09-23 14:32 ` Simon Marchi 2022-09-23 15:45 ` Nick Clifton @ 2022-09-24 6:53 ` Enze Li 2022-09-24 7:13 ` Fangrui Song 2022-09-27 18:08 ` Tom Tromey 2022-09-29 11:43 ` [PATCH v3] binutils, gdb: support zstd compressed debug sections Martin Liška 4 siblings, 1 reply; 25+ messages in thread From: Enze Li @ 2022-09-24 6:53 UTC (permalink / raw) To: Fangrui Song via Gdb-patches Cc: Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi, Fangrui Song, binutils Hi Fangrui, One nit, maybe not an important one. A proper committed message should include a link to the relevant bug on Bugzilla. Like this, Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29563 For more detail, see here[1]. [1] https://sourceware.org/gdb/wiki/ContributionChecklist#Properly_formatted_commit_messages Thanks, Enze On Thu, Sep 22 2022 at 09:08:37 PM -0700, Fangrui Song via Gdb-patches wrote: > PR29397 PR29563: The new configure option --with-zstd defaults to auto. > If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd > compressed debug sections for most tools. > > * bfd: for addr2line, objdump --dwarf, gdb, etc > * gas: support --compress-debug-sections=zstd > * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd > * objcopy: support ELFCOMPRESS_ZSTD input for > --decompress-debug-sections and --compress-debug-sections=zstd > * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd > symbols, so gdb has to link against -lzstd in this patch. > > If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We > can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this > is too heavyweight, so don't do it for now. > > ``` > % ld/ld-new a.o > ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support > ... > > % ld/ld-new a.o --compress-debug-sections=zstd > ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support > > % binutils/objcopy --compress-debug-sections=zstd a.o b.o > binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support > > % binutils/objcopy b.o --decompress-debug-sections > binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support > ... > ``` > --- > Changes from v1: > * use PKG_CHECK_MODULES to check libzstd.pc > > Changes from v2: > * Improve PKG_CHECK_MODULES, autoreconf -vf > * objcopy: check --compress-debug-sections=zstd in a !HAVE_ZSTD build > --- > bfd/Makefile.am | 4 +- > bfd/Makefile.in | 13 +- > bfd/aclocal.m4 | 2 + > bfd/bfd-in.h | 3 +- > bfd/bfd-in2.h | 11 +- > bfd/bfd.c | 26 +- > bfd/compress.c | 72 +++- > bfd/config.in | 3 + > bfd/configure | 268 ++++++++++++- > bfd/configure.ac | 3 +- > bfd/elf.c | 12 + > bfd/elfxx-target.h | 6 +- > bfd/section.c | 3 +- > binutils/Makefile.in | 5 +- > binutils/aclocal.m4 | 1 + > binutils/config.in | 3 + > binutils/configure | 135 ++++++- > binutils/configure.ac | 3 +- > binutils/doc/binutils.texi | 16 +- > binutils/objcopy.c | 19 +- > binutils/testsuite/binutils-all/compress.exp | 44 +++ > config/zstd.m4 | 24 ++ > configure | 10 + > configure.ac | 3 + > gas/Makefile.am | 4 +- > gas/Makefile.in | 13 +- > gas/aclocal.m4 | 2 + > gas/as.c | 13 +- > gas/compress-debug.c | 60 ++- > gas/compress-debug.h | 10 +- > gas/config.in | 3 + > gas/configure | 268 ++++++++++++- > gas/configure.ac | 3 +- > gas/doc/as.texi | 11 +- > gas/write.c | 36 +- > gdb/Makefile.in | 8 +- > gdb/acinclude.m4 | 3 +- > gdb/config.in | 3 + > gdb/configure | 136 ++++++- > gdb/configure.ac | 4 +- > ld/Makefile.am | 5 +- > ld/Makefile.in | 11 +- > ld/aclocal.m4 | 1 + > ld/config.in | 3 + > ld/configure | 389 ++++++++++++------- > ld/configure.ac | 7 +- > ld/emultempl/elf.em | 9 + > ld/ld.texi | 5 + > ld/ldmain.c | 8 +- > ld/lexsup.c | 4 +- > ld/testsuite/ld-elf/compress.exp | 17 + > 51 files changed, 1470 insertions(+), 255 deletions(-) > create mode 100644 config/zstd.m4 > > diff --git a/bfd/Makefile.am b/bfd/Makefile.am > index c23dff6cac3..cfe903c63f0 100644 > --- a/bfd/Makefile.am > +++ b/bfd/Makefile.am > @@ -57,7 +57,7 @@ ZLIBINC = @zlibinc@ > > WARN_CFLAGS = @WARN_CFLAGS@ > NO_WERROR = @NO_WERROR@ > -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) > +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) > AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' @LARGEFILE_CPPFLAGS@ > if PLUGINS > bfdinclude_HEADERS += $(INCDIR)/plugin-api.h > @@ -776,7 +776,7 @@ ofiles: stamp-ofiles ; @true > libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) > EXTRA_libbfd_la_SOURCES = $(CFILES) > libbfd_la_DEPENDENCIES = $(OFILES) ofiles > -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) > +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) > libbfd_la_LDFLAGS += -release `cat libtool-soversion` @SHARED_LDFLAGS@ > > # libtool will build .libs/libbfd.a. We create libbfd.a in the build > diff --git a/bfd/Makefile.in b/bfd/Makefile.in > index 82843d2d61d..6be41a72a59 100644 > --- a/bfd/Makefile.in > +++ b/bfd/Makefile.in > @@ -122,10 +122,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../config/acx.m4 \ > $(top_srcdir)/../config/lead-dot.m4 \ > $(top_srcdir)/../config/nls.m4 \ > $(top_srcdir)/../config/override.m4 \ > + $(top_srcdir)/../config/pkg.m4 \ > $(top_srcdir)/../config/plugins.m4 \ > $(top_srcdir)/../config/po.m4 \ > $(top_srcdir)/../config/progtest.m4 \ > - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ > + $(top_srcdir)/../config/zlib.m4 \ > + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ > $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ > $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ > $(top_srcdir)/bfd.m4 $(top_srcdir)/warning.m4 \ > @@ -399,6 +401,9 @@ PACKAGE_URL = @PACKAGE_URL@ > PACKAGE_VERSION = @PACKAGE_VERSION@ > PATH_SEPARATOR = @PATH_SEPARATOR@ > PKGVERSION = @PKGVERSION@ > +PKG_CONFIG = @PKG_CONFIG@ > +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ > +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ > POSUB = @POSUB@ > RANLIB = @RANLIB@ > REPORT_BUGS_TEXI = @REPORT_BUGS_TEXI@ > @@ -416,6 +421,8 @@ WARN_CFLAGS = @WARN_CFLAGS@ > WARN_CFLAGS_FOR_BUILD = @WARN_CFLAGS_FOR_BUILD@ > WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ > XGETTEXT = @XGETTEXT@ > +ZSTD_CFLAGS = @ZSTD_CFLAGS@ > +ZSTD_LIBS = @ZSTD_LIBS@ > abs_builddir = @abs_builddir@ > abs_srcdir = @abs_srcdir@ > abs_top_builddir = @abs_top_builddir@ > @@ -520,7 +527,7 @@ libbfd_la_LDFLAGS = $(am__append_1) -release `cat libtool-soversion` \ > # case both are empty. > ZLIB = @zlibdir@ -lz > ZLIBINC = @zlibinc@ > -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) > +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) > AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' \ > @LARGEFILE_CPPFLAGS@ @HDEFINES@ @COREFLAG@ @TDEFINES@ \ > $(CSEARCH) $(CSWITCHES) $(HAVEVECS) @INCINTL@ > @@ -1199,7 +1206,7 @@ OFILES = $(BFD_BACKENDS) $(BFD_MACHINES) @COREFILE@ @bfd64_libs@ > libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) > EXTRA_libbfd_la_SOURCES = $(CFILES) > libbfd_la_DEPENDENCIES = $(OFILES) ofiles > -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) > +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD_LIBS) > > # libtool will build .libs/libbfd.a. We create libbfd.a in the build > # directory so that we don't have to convert all the programs that use > diff --git a/bfd/aclocal.m4 b/bfd/aclocal.m4 > index 0f8aa1d4518..09b849dfc87 100644 > --- a/bfd/aclocal.m4 > +++ b/bfd/aclocal.m4 > @@ -1176,10 +1176,12 @@ m4_include([../config/largefile.m4]) > m4_include([../config/lead-dot.m4]) > m4_include([../config/nls.m4]) > m4_include([../config/override.m4]) > +m4_include([../config/pkg.m4]) > m4_include([../config/plugins.m4]) > m4_include([../config/po.m4]) > m4_include([../config/progtest.m4]) > m4_include([../config/zlib.m4]) > +m4_include([../config/zstd.m4]) > m4_include([../libtool.m4]) > m4_include([../ltoptions.m4]) > m4_include([../ltsugar.m4]) > diff --git a/bfd/bfd-in.h b/bfd/bfd-in.h > index 8605056aefe..4765ea80536 100644 > --- a/bfd/bfd-in.h > +++ b/bfd/bfd-in.h > @@ -342,7 +342,8 @@ enum compressed_debug_section_type > COMPRESS_DEBUG_NONE = 0, > COMPRESS_DEBUG = 1 << 0, > COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, > - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 > + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, > + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 > }; > > /* This structure is used to keep track of stabs in sections > diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h > index 79fcc4eb912..5c80956c79c 100644 > --- a/bfd/bfd-in2.h > +++ b/bfd/bfd-in2.h > @@ -349,7 +349,8 @@ enum compressed_debug_section_type > COMPRESS_DEBUG_NONE = 0, > COMPRESS_DEBUG = 1 << 0, > COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, > - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 > + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, > + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 > }; > > /* This structure is used to keep track of stabs in sections > @@ -962,7 +963,8 @@ typedef struct bfd_section > unsigned int compress_status : 2; > #define COMPRESS_SECTION_NONE 0 > #define COMPRESS_SECTION_DONE 1 > -#define DECOMPRESS_SECTION_SIZED 2 > +#define DECOMPRESS_SECTION_ZLIB 2 > +#define DECOMPRESS_SECTION_ZSTD 3 > > /* The following flags are used by the ELF linker. */ > > @@ -6637,12 +6639,14 @@ struct bfd > #define BFD_ARCHIVE_FULL_PATH 0x100000 > > #define BFD_CLOSED_BY_CACHE 0x200000 > + /* Compress sections in this BFD with SHF_COMPRESSED zstd. */ > +#define BFD_COMPRESS_ZSTD 0x400000 > > /* Flags bits to be saved in bfd_preserve_save. */ > #define BFD_FLAGS_SAVED \ > (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ > | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ > - | BFD_USE_ELF_STT_COMMON) > + | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) > > /* Flags bits which are for BFD use only. */ > #define BFD_FLAGS_FOR_BFD_USE_MASK \ > @@ -7271,6 +7275,7 @@ void bfd_update_compression_header > > bool bfd_check_compression_header > (bfd *abfd, bfd_byte *contents, asection *sec, > + unsigned int *ch_type, > bfd_size_type *uncompressed_size, > unsigned int *uncompressed_alignment_power); > > diff --git a/bfd/bfd.c b/bfd/bfd.c > index 0a21db11fd6..5f2033bee7a 100644 > --- a/bfd/bfd.c > +++ b/bfd/bfd.c > @@ -177,12 +177,15 @@ CODE_FRAGMENT > .#define BFD_ARCHIVE_FULL_PATH 0x100000 > . > .#define BFD_CLOSED_BY_CACHE 0x200000 > + > +. {* Compress sections in this BFD with SHF_COMPRESSED zstd. *} > +.#define BFD_COMPRESS_ZSTD 0x400000 > . > . {* Flags bits to be saved in bfd_preserve_save. *} > .#define BFD_FLAGS_SAVED \ > . (BFD_IN_MEMORY | BFD_COMPRESS | BFD_DECOMPRESS | BFD_LINKER_CREATED \ > . | BFD_PLUGIN | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON \ > -. | BFD_USE_ELF_STT_COMMON) > +. | BFD_USE_ELF_STT_COMMON | BFD_COMPRESS_ZSTD) > . > . {* Flags bits which are for BFD use only. *} > .#define BFD_FLAGS_FOR_BFD_USE_MASK \ > @@ -2500,6 +2503,9 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, > { > const struct elf_backend_data *bed = get_elf_backend_data (abfd); > struct bfd_elf_section_data * esd = elf_section_data (sec); > + const unsigned int ch_type = abfd->flags & BFD_COMPRESS_ZSTD > + ? ELFCOMPRESS_ZSTD > + : ELFCOMPRESS_ZLIB; > > /* Set the SHF_COMPRESSED bit. */ > elf_section_flags (sec) |= SHF_COMPRESSED; > @@ -2507,7 +2513,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, > if (bed->s->elfclass == ELFCLASS32) > { > Elf32_External_Chdr *echdr = (Elf32_External_Chdr *) contents; > - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); > + bfd_put_32 (abfd, ch_type, &echdr->ch_type); > bfd_put_32 (abfd, sec->size, &echdr->ch_size); > bfd_put_32 (abfd, 1u << sec->alignment_power, > &echdr->ch_addralign); > @@ -2518,7 +2524,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, > else > { > Elf64_External_Chdr *echdr = (Elf64_External_Chdr *) contents; > - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); > + bfd_put_32 (abfd, ch_type, &echdr->ch_type); > bfd_put_32 (abfd, 0, &echdr->ch_reserved); > bfd_put_64 (abfd, sec->size, &echdr->ch_size); > bfd_put_64 (abfd, UINT64_C (1) << sec->alignment_power, > @@ -2553,14 +2559,15 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, > SYNOPSIS > bool bfd_check_compression_header > (bfd *abfd, bfd_byte *contents, asection *sec, > + unsigned int *ch_type, > bfd_size_type *uncompressed_size, > unsigned int *uncompressed_alignment_power); > > DESCRIPTION > - Check the compression header at CONTENTS of SEC in ABFD and > - store the uncompressed size in UNCOMPRESSED_SIZE and the > - uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER > - if the compression header is valid. > + Check the compression header at CONTENTS of SEC in ABFD and store the > + ch_type in CH_TYPE, uncompressed size in UNCOMPRESSED_SIZE, and the > + uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER if the > + compression header is valid. > > RETURNS > Return TRUE if the compression header is valid. > @@ -2569,6 +2576,7 @@ RETURNS > bool > bfd_check_compression_header (bfd *abfd, bfd_byte *contents, > asection *sec, > + unsigned int *ch_type, > bfd_size_type *uncompressed_size, > unsigned int *uncompressed_alignment_power) > { > @@ -2591,7 +2599,9 @@ bfd_check_compression_header (bfd *abfd, bfd_byte *contents, > chdr.ch_size = bfd_get_64 (abfd, &echdr->ch_size); > chdr.ch_addralign = bfd_get_64 (abfd, &echdr->ch_addralign); > } > - if (chdr.ch_type == ELFCOMPRESS_ZLIB > + *ch_type = chdr.ch_type; > + if ((chdr.ch_type == ELFCOMPRESS_ZLIB > + || chdr.ch_type == ELFCOMPRESS_ZSTD) > && chdr.ch_addralign == (chdr.ch_addralign & -chdr.ch_addralign)) > { > *uncompressed_size = chdr.ch_size; > diff --git a/bfd/compress.c b/bfd/compress.c > index b2e39826e38..95c1f4c42d1 100644 > --- a/bfd/compress.c > +++ b/bfd/compress.c > @@ -20,18 +20,31 @@ > > #include "sysdep.h" > #include <zlib.h> > +#ifdef HAVE_ZSTD > +#include <zstd.h> > +#endif > #include "bfd.h" > +#include "elf-bfd.h" > #include "libbfd.h" > #include "safe-ctype.h" > > #define MAX_COMPRESSION_HEADER_SIZE 24 > > static bool > -decompress_contents (bfd_byte *compressed_buffer, > +decompress_contents (bool is_zstd, bfd_byte *compressed_buffer, > bfd_size_type compressed_size, > bfd_byte *uncompressed_buffer, > bfd_size_type uncompressed_size) > { > + if (is_zstd) > + { > +#ifdef HAVE_ZSTD > + size_t ret = ZSTD_decompress (uncompressed_buffer, uncompressed_size, > + compressed_buffer, compressed_size); > + return !ZSTD_isError (ret); > +#endif > + } > + > z_stream strm; > int rc; > > @@ -69,7 +82,7 @@ decompress_contents (bfd_byte *compressed_buffer, > } > > /* Compress data of the size specified in @var{uncompressed_size} > - and pointed to by @var{uncompressed_buffer} using zlib and store > + and pointed to by @var{uncompressed_buffer} using zlib/zstd and store > as the contents field. This function assumes the contents > field was allocated using bfd_malloc() or equivalent. > > @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, > sec->size = orig_uncompressed_size; > if (decompress) > { > - if (!decompress_contents (uncompressed_buffer > - + orig_compression_header_size, > - zlib_size, buffer, buffer_size)) > + if (!decompress_contents ( > + sec->compress_status == DECOMPRESS_SECTION_ZSTD, > + uncompressed_buffer + orig_compression_header_size, > + zlib_size, buffer, buffer_size)) > { > bfd_set_error (bfd_error_bad_value); > bfd_release (abfd, buffer); > @@ -175,10 +189,23 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, > } > else > { > - if (compress ((Bytef*) buffer + header_size, > - &compressed_size, > - (const Bytef*) uncompressed_buffer, > - uncompressed_size) != Z_OK) > + if (abfd->flags & BFD_COMPRESS_ZSTD) > + { > +#if HAVE_ZSTD > + compressed_size = ZSTD_compress ( > + buffer + header_size, compressed_size, uncompressed_buffer, > + uncompressed_size, ZSTD_CLEVEL_DEFAULT); > + if (ZSTD_isError (compressed_size)) > + { > + bfd_release (abfd, buffer); > + bfd_set_error (bfd_error_bad_value); > + return 0; > + } > +#endif > + } > + else if (compress ((Bytef *)buffer + header_size, &compressed_size, > + (const Bytef *)uncompressed_buffer, uncompressed_size) > + != Z_OK) > { > bfd_release (abfd, buffer); > bfd_set_error (bfd_error_bad_value); > @@ -237,6 +264,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) > bfd_size_type save_rawsize; > bfd_byte *compressed_buffer; > unsigned int compression_header_size; > + const unsigned int compress_status = sec->compress_status; > > if (abfd->direction != write_direction && sec->rawsize != 0) > sz = sec->rawsize; > @@ -248,7 +276,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) > return true; > } > > - switch (sec->compress_status) > + switch (compress_status) > { > case COMPRESS_SECTION_NONE: > if (p == NULL) > @@ -298,7 +326,8 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) > *ptr = p; > return true; > > - case DECOMPRESS_SECTION_SIZED: > + case DECOMPRESS_SECTION_ZLIB: > + case DECOMPRESS_SECTION_ZSTD: > /* Read in the full compressed section contents. */ > compressed_buffer = (bfd_byte *) bfd_malloc (sec->compressed_size); > if (compressed_buffer == NULL) > @@ -316,7 +345,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) > /* Restore rawsize and size. */ > sec->rawsize = save_rawsize; > sec->size = save_size; > - sec->compress_status = DECOMPRESS_SECTION_SIZED; > + sec->compress_status = compress_status; > if (!ret) > goto fail_compressed; > > @@ -330,8 +359,10 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) > /* Set header size to the zlib header size if it is a > SHF_COMPRESSED section. */ > compression_header_size = 12; > - if (!decompress_contents (compressed_buffer + compression_header_size, > - sec->compressed_size - compression_header_size, p, sz)) > + if (!decompress_contents (compress_status == DECOMPRESS_SECTION_ZSTD, > + compressed_buffer + compression_header_size, > + sec->compressed_size - compression_header_size, > + p, sz)) > { > bfd_set_error (bfd_error_bad_value); > if (p != *ptr) > @@ -381,7 +412,8 @@ DESCRIPTION > void > bfd_cache_section_contents (asection *sec, void *contents) > { > - if (sec->compress_status == DECOMPRESS_SECTION_SIZED) > + if (sec->compress_status == DECOMPRESS_SECTION_ZLIB > + || sec->compress_status == DECOMPRESS_SECTION_ZSTD) > sec->compress_status = COMPRESS_SECTION_DONE; > sec->contents = contents; > sec->flags |= SEC_IN_MEMORY; > @@ -418,6 +450,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, > int compression_header_size; > int header_size; > unsigned int saved = sec->compress_status; > + unsigned int ch_type; > bool compressed; > > *uncompressed_align_pow_p = 0; > @@ -448,7 +481,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, > { > if (compression_header_size != 0) > { > - if (!bfd_check_compression_header (abfd, header, sec, > + if (!bfd_check_compression_header (abfd, header, sec, &ch_type, > uncompressed_size_p, > uncompressed_align_pow_p)) > compression_header_size = -1; > @@ -507,7 +540,7 @@ SYNOPSIS > DESCRIPTION > Record compressed section size, update section size with > decompressed size and set compress_status to > - DECOMPRESS_SECTION_SIZED. > + DECOMPRESS_SECTION_{ZLIB,ZSTD}. > > Return @code{FALSE} if the section is not a valid compressed > section. Otherwise, return @code{TRUE}. > @@ -521,6 +554,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) > int header_size; > bfd_size_type uncompressed_size; > unsigned int uncompressed_alignment_power = 0; > + unsigned int ch_type; > z_stream strm; > > compression_header_size = bfd_get_compression_header_size (abfd, sec); > @@ -550,6 +584,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) > uncompressed_size = bfd_getb64 (header + 4); > } > else if (!bfd_check_compression_header (abfd, header, sec, > + &ch_type, > &uncompressed_size, > &uncompressed_alignment_power)) > { > @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) > sec->compressed_size = sec->size; > sec->size = uncompressed_size; > bfd_set_section_alignment (sec, uncompressed_alignment_power); > - sec->compress_status = DECOMPRESS_SECTION_SIZED; > + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD > + : DECOMPRESS_SECTION_ZLIB; > > return true; > } > diff --git a/bfd/config.in b/bfd/config.in > index f54a3cacbea..a59304e0a66 100644 > --- a/bfd/config.in > +++ b/bfd/config.in > @@ -232,6 +232,9 @@ > /* Define to 1 if you have the <windows.h> header file. */ > #undef HAVE_WINDOWS_H > > +/* Define to 1 if zstd is enabled. */ > +#undef HAVE_ZSTD > + > /* Define to the sub-directory in which libtool stores uninstalled libraries. > */ > #undef LT_OBJDIR > diff --git a/bfd/configure b/bfd/configure > index 075d2ee0a1b..d905a7bf7a8 100755 > --- a/bfd/configure > +++ b/bfd/configure > @@ -652,6 +652,11 @@ TDEFINES > SHARED_LIBADD > SHARED_LDFLAGS > LIBM > +ZSTD_LIBS > +ZSTD_CFLAGS > +PKG_CONFIG_LIBDIR > +PKG_CONFIG_PATH > +PKG_CONFIG > zlibinc > zlibdir > EXEEXT_FOR_BUILD > @@ -839,6 +844,7 @@ enable_maintainer_mode > enable_install_libbfd > enable_nls > with_system_zlib > +with_zstd > ' > ac_precious_vars='build_alias > host_alias > @@ -848,7 +854,12 @@ CFLAGS > LDFLAGS > LIBS > CPPFLAGS > -CPP' > +CPP > +PKG_CONFIG > +PKG_CONFIG_PATH > +PKG_CONFIG_LIBDIR > +ZSTD_CFLAGS > +ZSTD_LIBS' > > > # Initialize some variables set by options. > @@ -1511,6 +1522,8 @@ Optional Packages: > Binutils" > --with-bugurl=URL Direct users to URL to report a bug > --with-system-zlib use installed libz > + --with-zstd support zstd compressed debug sections > + (default=auto) > > Some influential environment variables: > CC C compiler command > @@ -1521,6 +1534,13 @@ Some influential environment variables: > CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if > you have headers in a nonstandard directory <include dir> > CPP C preprocessor > + PKG_CONFIG path to pkg-config utility > + PKG_CONFIG_PATH > + directories to add to pkg-config's search path > + PKG_CONFIG_LIBDIR > + path overriding pkg-config's built-in search path > + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config > + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config > > Use these variables to override the choices made by `configure' or to help > it to find libraries and programs with nonstandard names/locations. > @@ -11086,7 +11106,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 11089 "configure" > +#line 11109 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -11192,7 +11212,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 11195 "configure" > +#line 11215 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -12995,7 +13015,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h > ;; > esac > > -# Link in zlib if we can. This allows us to read compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. > # This is used only by compress.c. > > # Use the system's zlib library. > @@ -13015,6 +13035,246 @@ fi > > > > + > + > + > + > + > + > +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then > + if test -n "$ac_tool_prefix"; then > + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. > +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +PKG_CONFIG=$ac_cv_path_PKG_CONFIG > +if test -n "$PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 > +$as_echo "$PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + > +fi > +if test -z "$ac_cv_path_PKG_CONFIG"; then > + ac_pt_PKG_CONFIG=$PKG_CONFIG > + # Extract the first word of "pkg-config", so it can be a program name with args. > +set dummy pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $ac_pt_PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG > +if test -n "$ac_pt_PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 > +$as_echo "$ac_pt_PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + if test "x$ac_pt_PKG_CONFIG" = x; then > + PKG_CONFIG="" > + else > + case $cross_compiling:$ac_tool_warned in > +yes:) > +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 > +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} > +ac_tool_warned=yes ;; > +esac > + PKG_CONFIG=$ac_pt_PKG_CONFIG > + fi > +else > + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" > +fi > + > +fi > +if test -n "$PKG_CONFIG"; then > + _pkg_min_version=0.9.0 > + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 > +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } > + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + PKG_CONFIG="" > + fi > +fi > + > + > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +else > + with_zstd=auto > +fi > + > + > +if test "$with_zstd" != no; then > + > +pkg_failed=no > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 > +$as_echo_n "checking for libzstd... " >&6; } > + > +if test -n "$ZSTD_CFLAGS"; then > + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > +if test -n "$ZSTD_LIBS"; then > + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > + > +if test $pkg_failed = no; then > + pkg_save_LDFLAGS="$LDFLAGS" > + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" > + cat confdefs.h - <<_ACEOF >conftest.$ac_ext > +/* end confdefs.h. */ > + > +int > +main () > +{ > + > + ; > + return 0; > +} > +_ACEOF > +if ac_fn_c_try_link "$LINENO"; then : > + > +else > + pkg_failed=yes > +fi > +rm -f core conftest.err conftest.$ac_objext \ > + conftest$ac_exeext conftest.$ac_ext > + LDFLAGS=$pkg_save_LDFLAGS > +fi > + > + > + > +if test $pkg_failed = yes; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then > + _pkg_short_errors_supported=yes > +else > + _pkg_short_errors_supported=no > +fi > + if test $_pkg_short_errors_supported = yes; then > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` > + else > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` > + fi > + # Put the nasty error message in config.log where it belongs > + echo "$ZSTD_PKG_ERRORS" >&5 > + > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +elif test $pkg_failed = untried; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +else > + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS > + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + > + > +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h > + > + > +fi > +fi > + > + > save_CFLAGS="$CFLAGS" > CFLAGS="$CFLAGS -Werror" > { $as_echo "$as_me:${as_lineno-$LINENO}: checking compiler support for hidden visibility" >&5 > diff --git a/bfd/configure.ac b/bfd/configure.ac > index 28f3d1afce6..d1b2f76b31a 100644 > --- a/bfd/configure.ac > +++ b/bfd/configure.ac > @@ -230,9 +230,10 @@ AC_CHECK_DECLS([basename, ffs, stpcpy, asprintf, vasprintf, strnlen]) > > BFD_BINARY_FOPEN > > -# Link in zlib if we can. This allows us to read compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. > # This is used only by compress.c. > AM_ZLIB > +AM_ZSTD > > save_CFLAGS="$CFLAGS" > CFLAGS="$CFLAGS -Werror" > diff --git a/bfd/elf.c b/bfd/elf.c > index 16cea4f8aeb..e52fa458a5a 100644 > --- a/bfd/elf.c > +++ b/bfd/elf.c > @@ -1260,6 +1260,18 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, > abfd, name); > return false; > } > +#ifndef HAVE_ZSTD > + if (newsect->compress_status == DECOMPRESS_SECTION_ZSTD) > + { > + _bfd_error_handler > + /* xgettext:c-format */ > + (_ ("%pB: section %s is compressed with zstd, but BFD " > + "is not built with zstd support"), > + abfd, name); > + newsect->compress_status = COMPRESS_SECTION_NONE; > + return false; > + } > +#endif > } > > if (abfd->is_linker_input) > diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h > index feaba84bf2e..ca600bb5ddf 100644 > --- a/bfd/elfxx-target.h > +++ b/bfd/elfxx-target.h > @@ -989,7 +989,8 @@ const bfd_target TARGET_BIG_SYM = > /* object_flags: mask of all file flags */ > (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS > | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS > - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), > + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON > + | BFD_USE_ELF_STT_COMMON), > > /* section_flags: mask of all section flags */ > (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY > @@ -1093,7 +1094,8 @@ const bfd_target TARGET_LITTLE_SYM = > /* object_flags: mask of all file flags */ > (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS > | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS > - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), > + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON > + | BFD_USE_ELF_STT_COMMON), > > /* section_flags: mask of all section flags */ > (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY > diff --git a/bfd/section.c b/bfd/section.c > index c7a02d729f2..614570e976e 100644 > --- a/bfd/section.c > +++ b/bfd/section.c > @@ -392,7 +392,8 @@ CODE_FRAGMENT > . unsigned int compress_status : 2; > .#define COMPRESS_SECTION_NONE 0 > .#define COMPRESS_SECTION_DONE 1 > -.#define DECOMPRESS_SECTION_SIZED 2 > +.#define DECOMPRESS_SECTION_ZLIB 2 > +.#define DECOMPRESS_SECTION_ZSTD 3 > . > . {* The following flags are used by the ELF linker. *} > . > diff --git a/binutils/Makefile.in b/binutils/Makefile.in > index 78d32b350e3..6de4e239408 100644 > --- a/binutils/Makefile.in > +++ b/binutils/Makefile.in > @@ -156,7 +156,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ > $(top_srcdir)/../config/plugins.m4 \ > $(top_srcdir)/../config/po.m4 \ > $(top_srcdir)/../config/progtest.m4 \ > - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ > + $(top_srcdir)/../config/zlib.m4 \ > + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ > $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ > $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ > $(top_srcdir)/../bfd/version.m4 \ > @@ -575,6 +576,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ > XGETTEXT = @XGETTEXT@ > YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` > YFLAGS = -d > +ZSTD_CFLAGS = @ZSTD_CFLAGS@ > +ZSTD_LIBS = @ZSTD_LIBS@ > abs_builddir = @abs_builddir@ > abs_srcdir = @abs_srcdir@ > abs_top_builddir = @abs_top_builddir@ > diff --git a/binutils/aclocal.m4 b/binutils/aclocal.m4 > index a877fa7f873..28271f56279 100644 > --- a/binutils/aclocal.m4 > +++ b/binutils/aclocal.m4 > @@ -1205,6 +1205,7 @@ m4_include([../config/plugins.m4]) > m4_include([../config/po.m4]) > m4_include([../config/progtest.m4]) > m4_include([../config/zlib.m4]) > +m4_include([../config/zstd.m4]) > m4_include([../libtool.m4]) > m4_include([../ltoptions.m4]) > m4_include([../ltsugar.m4]) > diff --git a/binutils/config.in b/binutils/config.in > index c5fb919aa95..bee8c07e2f7 100644 > --- a/binutils/config.in > +++ b/binutils/config.in > @@ -157,6 +157,9 @@ > /* Define to 1 if you have the <windows.h> header file. */ > #undef HAVE_WINDOWS_H > > +/* Define to 1 if zstd is enabled. */ > +#undef HAVE_ZSTD > + > /* Define as const if the declaration of iconv() needs const. */ > #undef ICONV_CONST > > diff --git a/binutils/configure b/binutils/configure > index 1c518227f57..54d0f184da1 100755 > --- a/binutils/configure > +++ b/binutils/configure > @@ -650,6 +650,8 @@ LTLIBICONV > LIBICONV > MSGPACK_LIBS > MSGPACK_CFLAGS > +ZSTD_LIBS > +ZSTD_CFLAGS > zlibinc > zlibdir > DEMANGLER_NAME > @@ -832,6 +834,7 @@ enable_build_warnings > enable_nls > enable_maintainer_mode > with_system_zlib > +with_zstd > with_msgpack > enable_rpath > with_libiconv_prefix > @@ -853,6 +856,8 @@ DEBUGINFOD_CFLAGS > DEBUGINFOD_LIBS > YACC > YFLAGS > +ZSTD_CFLAGS > +ZSTD_LIBS > MSGPACK_CFLAGS > MSGPACK_LIBS' > > @@ -1517,6 +1522,8 @@ Optional Packages: > --with-debuginfod Enable debuginfo lookups with debuginfod > (auto/yes/no) > --with-system-zlib use installed libz > + --with-zstd support zstd compressed debug sections > + (default=auto) > --with-msgpack Enable msgpack support (auto/yes/no) > --with-gnu-ld assume the C compiler uses GNU ld default=no > --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib > @@ -1547,6 +1554,8 @@ Some influential environment variables: > YFLAGS The list of arguments that will be passed by default to $YACC. > This script will default YFLAGS to the empty string to avoid a > default value of `-d' given by some make applications. > + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config > + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config > MSGPACK_CFLAGS > C compiler flags for MSGPACK, overriding pkg-config > MSGPACK_LIBS > @@ -10804,7 +10813,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 10807 "configure" > +#line 10816 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -10910,7 +10919,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 10913 "configure" > +#line 10922 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -13468,7 +13477,7 @@ cat >>confdefs.h <<_ACEOF > _ACEOF > > > -# Link in zlib if we can. This allows us to read compressed debug > +# Link in zlib/zstd if we can. This allows us to read compressed debug > # sections. This is used only by readelf.c (objdump uses bfd for > # reading compressed sections). > > @@ -13490,6 +13499,126 @@ fi > > > > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +else > + with_zstd=auto > +fi > + > + > +if test "$with_zstd" != no; then > + > +pkg_failed=no > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 > +$as_echo_n "checking for libzstd... " >&6; } > + > +if test -n "$ZSTD_CFLAGS"; then > + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > +if test -n "$ZSTD_LIBS"; then > + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > + > +if test $pkg_failed = no; then > + pkg_save_LDFLAGS="$LDFLAGS" > + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" > + cat confdefs.h - <<_ACEOF >conftest.$ac_ext > +/* end confdefs.h. */ > + > +int > +main () > +{ > + > + ; > + return 0; > +} > +_ACEOF > +if ac_fn_c_try_link "$LINENO"; then : > + > +else > + pkg_failed=yes > +fi > +rm -f core conftest.err conftest.$ac_objext \ > + conftest$ac_exeext conftest.$ac_ext > + LDFLAGS=$pkg_save_LDFLAGS > +fi > + > + > + > +if test $pkg_failed = yes; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then > + _pkg_short_errors_supported=yes > +else > + _pkg_short_errors_supported=no > +fi > + if test $_pkg_short_errors_supported = yes; then > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` > + else > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` > + fi > + # Put the nasty error message in config.log where it belongs > + echo "$ZSTD_PKG_ERRORS" >&5 > + > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +elif test $pkg_failed = untried; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +else > + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS > + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + > + > +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h > + > + > +fi > +fi > + > + > + > case "${host}" in > *-*-msdos* | *-*-go32* | *-*-mingw32* | *-*-cygwin* | *-*-windows*) > > diff --git a/binutils/configure.ac b/binutils/configure.ac > index ec002d3f88f..c152b7f6733 100644 > --- a/binutils/configure.ac > +++ b/binutils/configure.ac > @@ -265,10 +265,11 @@ fi > > AC_CHECK_DECLS([asprintf, environ, getc_unlocked, stpcpy, strnlen]) > > -# Link in zlib if we can. This allows us to read compressed debug > +# Link in zlib/zstd if we can. This allows us to read compressed debug > # sections. This is used only by readelf.c (objdump uses bfd for > # reading compressed sections). > AM_ZLIB > +AM_ZSTD > > BFD_BINARY_FOPEN > > diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi > index 1499db5728c..34a4164eb94 100644 > --- a/binutils/doc/binutils.texi > +++ b/binutils/doc/binutils.texi > @@ -2159,21 +2159,23 @@ ELF ABI. Note - if compression would actually make a section > @itemx --compress-debug-sections=zlib > @itemx --compress-debug-sections=zlib-gnu > @itemx --compress-debug-sections=zlib-gabi > +@itemx --compress-debug-sections=zstd > For ELF files, these options control how DWARF debug sections are > compressed. @option{--compress-debug-sections=none} is equivalent > to @option{--decompress-debug-sections}. > @option{--compress-debug-sections=zlib} and > @option{--compress-debug-sections=zlib-gabi} are equivalent to > @option{--compress-debug-sections}. > -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug > -sections using zlib. The debug sections are renamed to begin with > -@samp{.zdebug} instead of @samp{.debug}. Note - if compression would > -actually make a section @emph{larger}, then it is not compressed nor > -renamed. > +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections > +using the obsoleted zlib-gnu format. The debug sections are renamed to begin > +with @samp{.zdebug}. > +@option{--compress-debug-sections=zstd} compresses DWARF debug > +sections using zstd. Note - if compression would actually make a section > +@emph{larger}, then it is not compressed nor renamed. > > @item --decompress-debug-sections > -Decompress DWARF debug sections using zlib. The original section > -names of the compressed sections are restored. > +Decompress DWARF debug sections. For a @samp{.zdebug} section, the original > +name is restored. > > @item --elf-stt-common=yes > @itemx --elf-stt-common=no > diff --git a/binutils/objcopy.c b/binutils/objcopy.c > index 43261756a42..fc668f00bbc 100644 > --- a/binutils/objcopy.c > +++ b/binutils/objcopy.c > @@ -232,7 +232,8 @@ static enum > compress_zlib = compress | 1 << 1, > compress_gnu_zlib = compress | 1 << 2, > compress_gabi_zlib = compress | 1 << 3, > - decompress = 1 << 4 > + compress_zstd = compress | 1 << 4, > + decompress = 1 << 5 > } do_debug_sections = nothing; > > /* Whether to generate ELF common symbols with the STT_COMMON type. */ > @@ -678,8 +679,8 @@ copy_usage (FILE *stream, int exit_status) > <commit>\n\ > --subsystem <name>[:<version>]\n\ > Set PE subsystem to <name> [& <version>]\n\ > - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ > - Compress DWARF debug sections using zlib\n\ > + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ > + Compress DWARF debug sections\n\ > --decompress-debug-sections Decompress DWARF debug sections using zlib\n\ > --elf-stt-common=[yes|no] Generate ELF common symbols with STT_COMMON\n\ > type\n\ > @@ -2659,7 +2660,8 @@ copy_object (bfd *ibfd, bfd *obfd, const bfd_arch_info_type *input_arch) > if ((do_debug_sections & compress) != 0 > && do_debug_sections != compress) > { > - non_fatal (_("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi] is unsupported on `%s'"), > + non_fatal (_ ("--compress-debug-sections=[zlib|zlib-gnu|zlib-gabi|" > + "zstd] is unsupported on `%s'"), > bfd_get_archive_filename (ibfd)); > return false; > } > @@ -3807,6 +3809,13 @@ copy_file (const char *input_filename, const char *output_filename, int ofd, > if (do_debug_sections != compress_gnu_zlib) > ibfd->flags |= BFD_COMPRESS_GABI; > break; > + case compress_zstd: > + ibfd->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; > +#ifndef HAVE_ZSTD > + fatal (_ ("--compress-debug-sections=zstd: binutils is not built with " > + "zstd support")); > +#endif > + break; > case decompress: > ibfd->flags |= BFD_DECOMPRESS; > break; > @@ -5469,6 +5478,8 @@ copy_main (int argc, char *argv[]) > do_debug_sections = compress_gnu_zlib; > else if (strcasecmp (optarg, "zlib-gabi") == 0) > do_debug_sections = compress_gabi_zlib; > + else if (strcasecmp (optarg, "zstd") == 0) > + do_debug_sections = compress_zstd; > else > fatal (_("unrecognized --compress-debug-sections type `%s'"), > optarg); > diff --git a/binutils/testsuite/binutils-all/compress.exp b/binutils/testsuite/binutils-all/compress.exp > index c88da74a987..4e74c824664 100644 > --- a/binutils/testsuite/binutils-all/compress.exp > +++ b/binutils/testsuite/binutils-all/compress.exp > @@ -576,6 +576,50 @@ if { [regexp_diff objdump.out $srcdir/$subdir/dw2-3gabi.W] } then { > pass "$testname" > } > > +if { [binutils_assemble_flags $srcdir/$subdir/dw2-1.S ${compressedfile}zstd.o --compress-debug-sections=zstd] } then { > + set testname "objcopy compress debug sections with zstd" > + set got [binutils_run $OBJCOPY "--compress-debug-sections=zstd ${testfile}.o ${copyfile}zstd.o"] > + if ![string match "" $got] then { > + fail "objcopy ($testname)" > + return > + } > + send_log "cmp ${compressedfile}zstd.o ${copyfile}zstd.o\n" > + verbose "cmp ${compressedfile}zstd.o ${copyfile}zstd.o" > + set src1 ${compressedfile}zstd.o > + set src2 ${copyfile}zstd.o > + set status [remote_exec build cmp "${src1} ${src2}"] > + set exec_output [lindex $status 1] > + set exec_output [prune_warnings $exec_output] > + if ![string match "" $exec_output] then { > + send_log "$exec_output\n" > + verbose "$exec_output" 1 > + fail "objcopy ($testname)" > + } else { > + pass "objcopy ($testname)" > + } > + > + set testname "objcopy decompress compressed debug sections with zstd" > + set got [binutils_run $OBJCOPY "--decompress-debug-sections ${compressedfile}zstd.o ${copyfile}zstd.o"] > + if ![string match "" $got] then { > + fail "objcopy ($testname)" > + return > + } > + send_log "cmp ${testfile}.o ${copyfile}zstd.o\n" > + verbose "cmp ${testfile}.o ${copyfile}zstd.o" > + set src1 ${testfile}.o > + set src2 ${copyfile}zstd.o > + set status [remote_exec build cmp "${src1} ${src2}"] > + set exec_output [lindex $status 1] > + set exec_output [prune_warnings $exec_output] > + if ![string match "" $exec_output] then { > + send_log "$exec_output\n" > + verbose "$exec_output" 1 > + fail "objcopy ($testname)" > + } else { > + pass "objcopy ($testname)" > + } > +} > + > proc convert_test { testname as_flags objcop_flags } { > global srcdir > global subdir > diff --git a/config/zstd.m4 b/config/zstd.m4 > new file mode 100644 > index 00000000000..93ae17b89d3 > --- /dev/null > +++ b/config/zstd.m4 > @@ -0,0 +1,24 @@ > +dnl Copyright (C) 2022 Free Software Foundation, Inc. > +dnl This file is free software, distributed under the terms of the GNU > +dnl General Public License. As a special exception to the GNU General > +dnl Public License, this file may be distributed as part of a program > +dnl that contains a configuration script generated by Autoconf, under > +dnl the same distribution terms as the rest of that program. > + > +dnl Enable features using the zstd library. > +AC_DEFUN([AM_ZSTD], > +[ > +AC_ARG_WITH(zstd, > + [AS_HELP_STRING([--with-zstd], [support zstd compressed debug sections (default=auto)])], > + [], [with_zstd=auto]) > + > +if test "$with_zstd" != no; then > + PKG_CHECK_MODULES(ZSTD, [libzstd], [ > + AC_DEFINE(HAVE_ZSTD, 1, [Define to 1 if zstd is enabled.]) > + ], [ > + if test "$with_zstd" = yes; then > + AC_MSG_ERROR([--with-zstd was given, but pkgconfig/libzstd.pc is not found]) > + fi > + ]) > +fi > +]) > diff --git a/configure b/configure > index d75f47a1e95..f14e0efd675 100755 > --- a/configure > +++ b/configure > @@ -785,6 +785,7 @@ ac_user_opts=' > enable_option_checking > with_build_libsubdir > with_system_zlib > +with_zstd > enable_as_accelerator_for > enable_offload_targets > enable_gold > @@ -1567,6 +1568,8 @@ Optional Packages: > --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) > --with-build-libsubdir=DIR Directory where to find libraries for build system > --with-system-zlib use installed libz > + --with-zstd Support zstd compressed debug sections > + (default=auto) > --with-mpc=PATH specify prefix directory for installed MPC package. > Equivalent to --with-mpc-include=PATH/include plus > --with-mpc-lib=PATH/lib > @@ -2925,6 +2928,13 @@ if test x$with_system_zlib = xyes ; then > noconfigdirs="$noconfigdirs zlib" > fi > > + > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +fi > + > + > # Don't compile the bundled readline/libreadline.a if --with-system-readline > # is provided. > if test x$with_system_readline = xyes ; then > diff --git a/configure.ac b/configure.ac > index ae18d436aca..0152c69292e 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -246,6 +246,9 @@ if test x$with_system_zlib = xyes ; then > noconfigdirs="$noconfigdirs zlib" > fi > > +AC_ARG_WITH(zstd, > +[AS_HELP_STRING([--with-zstd], [Support zstd compressed debug sections (default=auto)])]) > + > # Don't compile the bundled readline/libreadline.a if --with-system-readline > # is provided. > if test x$with_system_readline = xyes ; then > diff --git a/gas/Makefile.am b/gas/Makefile.am > index bd597398671..5f0f24abf8d 100644 > --- a/gas/Makefile.am > +++ b/gas/Makefile.am > @@ -42,7 +42,7 @@ am__skipyacc = > > WARN_CFLAGS = @WARN_CFLAGS@ @WARN_WRITE_STRINGS@ > NO_WERROR = @NO_WERROR@ > -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) > +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) > > TARG_CPU = @target_cpu_type@ > TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c > @@ -407,7 +407,7 @@ STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) > > as_new_SOURCES = $(GAS_CFILES) > as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ > - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) > + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) > as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ > $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) > EXTRA_as_new_SOURCES = $(CFILES) $(HFILES) $(TARGET_CPU_CFILES) \ > diff --git a/gas/Makefile.in b/gas/Makefile.in > index c57d78f82c4..5a4dd702252 100644 > --- a/gas/Makefile.in > +++ b/gas/Makefile.in > @@ -140,10 +140,12 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ > $(top_srcdir)/../config/lead-dot.m4 \ > $(top_srcdir)/../config/nls.m4 \ > $(top_srcdir)/../config/override.m4 \ > + $(top_srcdir)/../config/pkg.m4 \ > $(top_srcdir)/../config/plugins.m4 \ > $(top_srcdir)/../config/po.m4 \ > $(top_srcdir)/../config/progtest.m4 \ > - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ > + $(top_srcdir)/../config/zlib.m4 \ > + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ > $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ > $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ > $(top_srcdir)/acinclude.m4 $(top_srcdir)/../bfd/version.m4 \ > @@ -429,6 +431,9 @@ PACKAGE_TARNAME = @PACKAGE_TARNAME@ > PACKAGE_URL = @PACKAGE_URL@ > PACKAGE_VERSION = @PACKAGE_VERSION@ > PATH_SEPARATOR = @PATH_SEPARATOR@ > +PKG_CONFIG = @PKG_CONFIG@ > +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ > +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ > POSUB = @POSUB@ > RANLIB = @RANLIB@ > SED = @SED@ > @@ -443,6 +448,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ > XGETTEXT = @XGETTEXT@ > YACC = `if [ -f ../bison/bison ] ; then echo ../bison/bison -y -L../bison/bison ; else echo @YACC@ ; fi` > YFLAGS = @YFLAGS@ > +ZSTD_CFLAGS = @ZSTD_CFLAGS@ > +ZSTD_LIBS = @ZSTD_LIBS@ > abs_builddir = @abs_builddir@ > abs_srcdir = @abs_srcdir@ > abs_top_builddir = @abs_top_builddir@ > @@ -524,7 +531,7 @@ ZLIBINC = @zlibinc@ > # maintainer mode is disabled. Avoid this. > am__skiplex = > am__skipyacc = > -AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) > +AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) $(ZSTD_CFLAGS) > TARG_CPU = @target_cpu_type@ > TARG_CPU_C = $(srcdir)/config/tc-@target_cpu_type@.c > TARG_CPU_O = config/tc-@target_cpu_type@.@OBJEXT@ > @@ -874,7 +881,7 @@ GASLIBS = @OPCODES_LIB@ ../bfd/libbfd.la ../libiberty/libiberty.a > STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) > as_new_SOURCES = $(GAS_CFILES) > as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ > - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) > + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD_LIBS) > > as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ > $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) > diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 > index 70183124da7..722030c776f 100644 > --- a/gas/aclocal.m4 > +++ b/gas/aclocal.m4 > @@ -1196,10 +1196,12 @@ m4_include([../config/lcmessage.m4]) > m4_include([../config/lead-dot.m4]) > m4_include([../config/nls.m4]) > m4_include([../config/override.m4]) > +m4_include([../config/pkg.m4]) > m4_include([../config/plugins.m4]) > m4_include([../config/po.m4]) > m4_include([../config/progtest.m4]) > m4_include([../config/zlib.m4]) > +m4_include([../config/zstd.m4]) > m4_include([../libtool.m4]) > m4_include([../ltoptions.m4]) > m4_include([../ltsugar.m4]) > diff --git a/gas/as.c b/gas/as.c > index 6268779cf90..35ad6b3ab3b 100644 > --- a/gas/as.c > +++ b/gas/as.c > @@ -252,14 +252,14 @@ Options:\n\ > --alternate initially turn on alternate macro syntax\n")); > #ifdef DEFAULT_FLAG_COMPRESS_DEBUG > fprintf (stream, _("\ > - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ > + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ > compress DWARF debug sections using zlib [default]\n")); > fprintf (stream, _("\ > --nocompress-debug-sections\n\ > don't compress DWARF debug sections\n")); > #else > fprintf (stream, _("\ > - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ > + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ > compress DWARF debug sections using zlib\n")); > fprintf (stream, _("\ > --nocompress-debug-sections\n\ > @@ -736,6 +736,15 @@ This program has absolutely no warranty.\n")); > flag_compress_debug = COMPRESS_DEBUG_GNU_ZLIB; > else if (strcasecmp (optarg, "zlib-gabi") == 0) > flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB; > + else if (strcasecmp (optarg, "zstd") == 0) > + { > +#ifdef HAVE_ZSTD > + flag_compress_debug = COMPRESS_DEBUG_ZSTD; > +#else > + as_fatal (_ ("--compress-debug-sections=zstd: gas is not " > + "built with zstd support")); > +#endif > + } > else > as_fatal (_("Invalid --compress-debug-sections option: `%s'"), > optarg); > diff --git a/gas/compress-debug.c b/gas/compress-debug.c > index c80dbeec4e7..3cd175f6e57 100644 > --- a/gas/compress-debug.c > +++ b/gas/compress-debug.c > @@ -21,14 +21,23 @@ > #include "config.h" > #include <stdio.h> > #include <zlib.h> > +#if HAVE_ZSTD > +#include <zstd.h> > +#endif > #include "ansidecl.h" > #include "compress-debug.h" > > /* Initialize the compression engine. */ > > -struct z_stream_s * > -compress_init (void) > +void * > +compress_init (bool use_zstd) > { > + if (use_zstd) { > +#if HAVE_ZSTD > + return ZSTD_createCCtx (); > +#endif > + } > + > static struct z_stream_s strm; > > strm.zalloc = NULL; > @@ -42,22 +51,37 @@ compress_init (void) > from the engine goes into the current frag on the obstack. */ > > int > -compress_data (struct z_stream_s *strm, const char **next_in, > - int *avail_in, char **next_out, int *avail_out) > +compress_data (bool use_zstd, void *ctx, const char **next_in, int *avail_in, > + char **next_out, int *avail_out) > { > - int out_size = 0; > - int x; > + if (use_zstd) > + { > +#if HAVE_ZSTD > + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; > + ZSTD_inBuffer ib = { *next_in, *avail_in, 0 }; > + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_continue); > + *next_in += ib.pos; > + *avail_in -= ib.pos; > + *next_out += ob.pos; > + *avail_out -= ob.pos; > + if (ZSTD_isError (ret)) > + return -1; > + return (int)ob.pos; > +#endif > + } > + > + struct z_stream_s *strm = ctx; > > strm->next_in = (Bytef *) (*next_in); > strm->avail_in = *avail_in; > strm->next_out = (Bytef *) (*next_out); > strm->avail_out = *avail_out; > > - x = deflate (strm, Z_NO_FLUSH); > + int x = deflate (strm, Z_NO_FLUSH); > if (x != Z_OK) > return -1; > > - out_size = *avail_out - strm->avail_out; > + int out_size = *avail_out - strm->avail_out; > *next_in = (char *) (strm->next_in); > *avail_in = strm->avail_in; > *next_out = (char *) (strm->next_out); > @@ -71,10 +95,28 @@ compress_data (struct z_stream_s *strm, const char **next_in, > needed. */ > > int > -compress_finish (struct z_stream_s *strm, char **next_out, > +compress_finish (bool use_zstd, void *ctx, char **next_out, > int *avail_out, int *out_size) > { > + if (use_zstd) > + { > +#if HAVE_ZSTD > + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; > + ZSTD_inBuffer ib = { 0 }; > + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_end); > + *out_size = ob.pos; > + *next_out += ob.pos; > + *avail_out -= ob.pos; > + if (ZSTD_isError (ret)) > + return -1; > + if (ret == 0) > + ZSTD_freeCCtx (ctx); > + return ret ? 1 : 0; > +#endif > + } > + > int x; > + struct z_stream_s *strm = ctx; > > strm->avail_in = 0; > strm->next_out = (Bytef *) (*next_out); > diff --git a/gas/compress-debug.h b/gas/compress-debug.h > index 0183e269fef..87de0f8444e 100644 > --- a/gas/compress-debug.h > +++ b/gas/compress-debug.h > @@ -21,19 +21,19 @@ > #ifndef COMPRESS_DEBUG_H > #define COMPRESS_DEBUG_H > > +#include <stdbool.h> > + > struct z_stream_s; > > /* Initialize the compression engine. */ > -extern struct z_stream_s * > -compress_init (void); > +extern void *compress_init (bool); > > /* Stream the contents of a frag to the compression engine. Output > from the engine goes into the current frag on the obstack. */ > -extern int > -compress_data (struct z_stream_s *, const char **, int *, char **, int *); > +extern int compress_data (bool, void *, const char **, int *, char **, int *); > > /* Finish the compression and consume the remaining compressed output. */ > extern int > -compress_finish (struct z_stream_s *, char **, int *, int *); > +compress_finish (bool, void *, char **, int *, int *); > > #endif /* COMPRESS_DEBUG_H */ > diff --git a/gas/config.in b/gas/config.in > index e243fd277ee..0d1668a3eac 100644 > --- a/gas/config.in > +++ b/gas/config.in > @@ -134,6 +134,9 @@ > /* Define to 1 if you have the <windows.h> header file. */ > #undef HAVE_WINDOWS_H > > +/* Define to 1 if zstd is enabled. */ > +#undef HAVE_ZSTD > + > /* Using i386 COFF? */ > #undef I386COFF > > diff --git a/gas/configure b/gas/configure > index d0449a1d7ab..1833adffaa8 100755 > --- a/gas/configure > +++ b/gas/configure > @@ -633,6 +633,11 @@ ac_subst_vars='am__EXEEXT_FALSE > am__EXEEXT_TRUE > LTLIBOBJS > LIBOBJS > +ZSTD_LIBS > +ZSTD_CFLAGS > +PKG_CONFIG_LIBDIR > +PKG_CONFIG_PATH > +PKG_CONFIG > zlibinc > zlibdir > LIBM > @@ -817,6 +822,7 @@ with_cpu > enable_nls > enable_maintainer_mode > with_system_zlib > +with_zstd > ' > ac_precious_vars='build_alias > host_alias > @@ -828,7 +834,12 @@ LIBS > CPPFLAGS > CPP > YACC > -YFLAGS' > +YFLAGS > +PKG_CONFIG > +PKG_CONFIG_PATH > +PKG_CONFIG_LIBDIR > +ZSTD_CFLAGS > +ZSTD_LIBS' > > > # Initialize some variables set by options. > @@ -1493,6 +1504,8 @@ Optional Packages: > --with-cpu=CPU default cpu variant is CPU (currently only supported > on ARC) > --with-system-zlib use installed libz > + --with-zstd support zstd compressed debug sections > + (default=auto) > > Some influential environment variables: > CC C compiler command > @@ -1509,6 +1522,13 @@ Some influential environment variables: > YFLAGS The list of arguments that will be passed by default to $YACC. > This script will default YFLAGS to the empty string to avoid a > default value of `-d' given by some make applications. > + PKG_CONFIG path to pkg-config utility > + PKG_CONFIG_PATH > + directories to add to pkg-config's search path > + PKG_CONFIG_LIBDIR > + path overriding pkg-config's built-in search path > + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config > + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config > > Use these variables to override the choices made by `configure' or to help > it to find libraries and programs with nonstandard names/locations. > @@ -10702,7 +10722,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 10705 "configure" > +#line 10725 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -10808,7 +10828,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 10811 "configure" > +#line 10831 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -13945,7 +13965,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h > ;; > esac > > -# Link in zlib if we can. This allows us to write compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. > > # Use the system's zlib library. > zlibdir="-L\$(top_builddir)/../zlib" > @@ -13964,6 +13984,246 @@ fi > > > > + > + > + > + > + > + > +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then > + if test -n "$ac_tool_prefix"; then > + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. > +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +PKG_CONFIG=$ac_cv_path_PKG_CONFIG > +if test -n "$PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 > +$as_echo "$PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + > +fi > +if test -z "$ac_cv_path_PKG_CONFIG"; then > + ac_pt_PKG_CONFIG=$PKG_CONFIG > + # Extract the first word of "pkg-config", so it can be a program name with args. > +set dummy pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $ac_pt_PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG > +if test -n "$ac_pt_PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 > +$as_echo "$ac_pt_PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + if test "x$ac_pt_PKG_CONFIG" = x; then > + PKG_CONFIG="" > + else > + case $cross_compiling:$ac_tool_warned in > +yes:) > +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 > +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} > +ac_tool_warned=yes ;; > +esac > + PKG_CONFIG=$ac_pt_PKG_CONFIG > + fi > +else > + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" > +fi > + > +fi > +if test -n "$PKG_CONFIG"; then > + _pkg_min_version=0.9.0 > + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 > +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } > + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + PKG_CONFIG="" > + fi > +fi > + > + > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +else > + with_zstd=auto > +fi > + > + > +if test "$with_zstd" != no; then > + > +pkg_failed=no > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 > +$as_echo_n "checking for libzstd... " >&6; } > + > +if test -n "$ZSTD_CFLAGS"; then > + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > +if test -n "$ZSTD_LIBS"; then > + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > + > +if test $pkg_failed = no; then > + pkg_save_LDFLAGS="$LDFLAGS" > + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" > + cat confdefs.h - <<_ACEOF >conftest.$ac_ext > +/* end confdefs.h. */ > + > +int > +main () > +{ > + > + ; > + return 0; > +} > +_ACEOF > +if ac_fn_c_try_link "$LINENO"; then : > + > +else > + pkg_failed=yes > +fi > +rm -f core conftest.err conftest.$ac_objext \ > + conftest$ac_exeext conftest.$ac_ext > + LDFLAGS=$pkg_save_LDFLAGS > +fi > + > + > + > +if test $pkg_failed = yes; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then > + _pkg_short_errors_supported=yes > +else > + _pkg_short_errors_supported=no > +fi > + if test $_pkg_short_errors_supported = yes; then > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` > + else > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` > + fi > + # Put the nasty error message in config.log where it belongs > + echo "$ZSTD_PKG_ERRORS" >&5 > + > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +elif test $pkg_failed = untried; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +else > + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS > + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + > + > +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h > + > + > +fi > +fi > + > + > # Support for VMS timestamps via cross compile > > if test "$ac_cv_header_time_h" = yes; then > diff --git a/gas/configure.ac b/gas/configure.ac > index 21795a82bdd..8750ea21381 100644 > --- a/gas/configure.ac > +++ b/gas/configure.ac > @@ -1004,8 +1004,9 @@ AC_CHECK_DECLS([asprintf, mempcpy, stpcpy]) > > BFD_BINARY_FOPEN > > -# Link in zlib if we can. This allows us to write compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. > AM_ZLIB > +AM_ZSTD > > # Support for VMS timestamps via cross compile > > diff --git a/gas/doc/as.texi b/gas/doc/as.texi > index e915893da86..01f49434021 100644 > --- a/gas/doc/as.texi > +++ b/gas/doc/as.texi > @@ -711,16 +711,19 @@ given section @emph{larger} then it is not compressed. > @itemx --compress-debug-sections=zlib > @itemx --compress-debug-sections=zlib-gnu > @itemx --compress-debug-sections=zlib-gabi > +@itemx --compress-debug-sections=zstd > These options control how DWARF debug sections are compressed. > @option{--compress-debug-sections=none} is equivalent to > @option{--nocompress-debug-sections}. > @option{--compress-debug-sections=zlib} and > @option{--compress-debug-sections=zlib-gabi} are equivalent to > @option{--compress-debug-sections}. > -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug > -sections using zlib. The debug sections are renamed to begin with > -@samp{.zdebug}. Note if compression would make a given section > -@emph{larger} then it is not compressed nor renamed. > +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections > +using the obsoleted zlib-gnu format. The debug sections are renamed to begin > +with @samp{.zdebug}. > +@option{--compress-debug-sections=zstd} compresses DWARF debug > +sections using zstd. Note - if compression would actually make a section > +@emph{larger}, then it is not compressed nor renamed. > > @end ifset > > diff --git a/gas/write.c b/gas/write.c > index f76bbdb706e..0e49df7c03f 100644 > --- a/gas/write.c > +++ b/gas/write.c > @@ -1413,7 +1413,7 @@ write_relocs (bfd *abfd ATTRIBUTE_UNUSED, asection *sec, > } > > static int > -compress_frag (struct z_stream_s *strm, const char *contents, int in_size, > +compress_frag (bool use_zstd, void *ctx, const char *contents, int in_size, > fragS **last_newf, struct obstack *ob) > { > int out_size; > @@ -1442,10 +1442,10 @@ compress_frag (struct z_stream_s *strm, const char *contents, int in_size, > as_fatal (_("can't extend frag")); > next_out = obstack_next_free (ob); > obstack_blank_fast (ob, avail_out); > - out_size = compress_data (strm, &contents, &in_size, > - &next_out, &avail_out); > + out_size = compress_data (use_zstd, ctx, &contents, &in_size, &next_out, > + &avail_out); > if (out_size < 0) > - return -1; > + return -1; > > f->fr_fix += out_size; > total_out_size += out_size; > @@ -1471,7 +1471,6 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) > const char *section_name; > char *compressed_name; > char *header; > - struct z_stream_s *strm; > int x; > flagword flags = bfd_section_flags (sec); > unsigned int header_size, compression_header_size; > @@ -1485,21 +1484,22 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) > if (!startswith (section_name, ".debug_")) > return; > > - strm = compress_init (); > - if (strm == NULL) > + bool use_zstd = abfd->flags & BFD_COMPRESS_ZSTD; > + void *ctx = compress_init (use_zstd); > + if (ctx == NULL) > return; > > - if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) > + if (flag_compress_debug == COMPRESS_DEBUG_GNU_ZLIB) > + { > + compression_header_size = 0; > + header_size = 12; > + } > + else > { > compression_header_size > = bfd_get_compression_header_size (stdoutput, NULL); > header_size = compression_header_size; > } > - else > - { > - compression_header_size = 0; > - header_size = 12; > - } > > /* Create a new frag to contain the compression header. */ > first_newf = frag_alloc (ob); > @@ -1531,7 +1531,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) > gas_assert (f->fr_type == rs_fill); > if (f->fr_fix) > { > - out_size = compress_frag (strm, f->fr_literal, f->fr_fix, > + out_size = compress_frag (use_zstd, ctx, f->fr_literal, f->fr_fix, > &last_newf, ob); > if (out_size < 0) > return; > @@ -1545,8 +1545,8 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) > { > while (count--) > { > - out_size = compress_frag (strm, fill_literal, (int) fill_size, > - &last_newf, ob); > + out_size = compress_frag (use_zstd, ctx, fill_literal, > + (int)fill_size, &last_newf, ob); > if (out_size < 0) > return; > compressed_size += out_size; > @@ -1579,7 +1579,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) > as_fatal (_("can't extend frag")); > next_out = obstack_next_free (ob); > obstack_blank_fast (ob, avail_out); > - x = compress_finish (strm, &next_out, &avail_out, &out_size); > + x = compress_finish (use_zstd, ctx, &next_out, &avail_out, &out_size); > if (x < 0) > return; > > @@ -2540,6 +2540,8 @@ write_object_file (void) > { > if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) > stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI; > + else if (flag_compress_debug == COMPRESS_DEBUG_ZSTD) > + stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; > else > stdoutput->flags |= BFD_COMPRESS; > bfd_map_over_sections (stdoutput, compress_debug, (char *) 0); > diff --git a/gdb/Makefile.in b/gdb/Makefile.in > index 2598b81d205..c528ee5aa80 100644 > --- a/gdb/Makefile.in > +++ b/gdb/Makefile.in > @@ -173,6 +173,9 @@ BFD_CFLAGS = -I$(BFD_DIR) -I$(BFD_SRC) > ZLIB = @zlibdir@ -lz > ZLIBINC = @zlibinc@ > > +ZSTD_CFLAGS = @ZSTD_CFLAGS@ > +ZSTD_LIBS = @ZSTD_LIBS@ > + > # Where is the decnumber library? Typically in ../libdecnumber. > LIBDECNUMBER_DIR = ../libdecnumber > LIBDECNUMBER = $(LIBDECNUMBER_DIR)/libdecnumber.a > @@ -625,7 +628,7 @@ INTERNAL_CPPFLAGS = $(CPPFLAGS) @GUILE_CPPFLAGS@ @PYTHON_CPPFLAGS@ \ > INTERNAL_CFLAGS_BASE = \ > $(GLOBAL_CFLAGS) $(PROFILE_CFLAGS) \ > $(GDB_CFLAGS) $(OPCODES_CFLAGS) $(READLINE_CFLAGS) $(ZLIBINC) \ > - $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ > + $(ZSTD_CFLAGS) $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \ > $(INTL_CFLAGS) $(INCGNU) $(INCSUPPORT) $(LIBBACKTRACE_INC) \ > $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS) $(SRCHIGH_CFLAGS) \ > $(TOP_CFLAGS) $(PTHREAD_CFLAGS) $(DEBUGINFOD_CFLAGS) > @@ -647,7 +650,7 @@ INTERNAL_LDFLAGS = \ > # Libraries and corresponding dependencies for compiling gdb. > # XM_CLIBS, defined in *config files, have host-dependent libs. > # LIBIBERTY appears twice on purpose. > -CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \ > +CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) $(ZSTD_LIBS) \ > $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \ > $(XM_CLIBS) $(GDBTKLIBS) $(LIBBACKTRACE_LIB) \ > @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \ > @@ -2298,6 +2301,7 @@ aclocal_m4_deps = \ > ../config/lcmessage.m4 \ > ../config/codeset.m4 \ > ../config/zlib.m4 \ > + ../config/zstd.m4 \ > ../config/ax_pthread.m4 > > $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) > diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 > index 95ff2b6f35e..28846119dcb 100644 > --- a/gdb/acinclude.m4 > +++ b/gdb/acinclude.m4 > @@ -43,6 +43,7 @@ m4_include([../config/lib-link.m4]) > m4_include([../config/iconv.m4]) > > m4_include([../config/zlib.m4]) > +m4_include([../config/zstd.m4]) > > m4_include([../gdbsupport/common.m4]) > > @@ -233,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ > # always want our bfd. > CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" > ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` > - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" > + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" > intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` > LIBS="-lbfd -liberty -lz $intl $LIBS" > AC_CACHE_CHECK( > diff --git a/gdb/config.in b/gdb/config.in > index a4975c68aad..e13a409ec2d 100644 > --- a/gdb/config.in > +++ b/gdb/config.in > @@ -589,6 +589,9 @@ > /* Define to 1 if you have the `XML_StopParser' function. */ > #undef HAVE_XML_STOPPARSER > > +/* Define to 1 if zstd is enabled. */ > +#undef HAVE_ZSTD > + > /* Define to 1 if your system has the _etext variable. */ > #undef HAVE__ETEXT > > diff --git a/gdb/configure b/gdb/configure > index 4dbd0c3b13c..57cd165088a 100755 > --- a/gdb/configure > +++ b/gdb/configure > @@ -747,6 +747,8 @@ READLINE_DEPS > READLINE > LTLIBICONV > LIBICONV > +ZSTD_LIBS > +ZSTD_CFLAGS > zlibinc > zlibdir > MIG > @@ -893,6 +895,7 @@ enable_codesign > with_pkgversion > with_bugurl > with_system_zlib > +with_zstd > with_gnu_ld > enable_rpath > with_libiconv_prefix > @@ -961,6 +964,8 @@ DEBUGINFOD_CFLAGS > DEBUGINFOD_LIBS > YACC > YFLAGS > +ZSTD_CFLAGS > +ZSTD_LIBS > XMKMF' > ac_subdirs_all='testsuite > gdbtk' > @@ -1641,6 +1646,8 @@ Optional Packages: > --with-pkgversion=PKG Use PKG in the version string in place of "GDB" > --with-bugurl=URL Direct users to URL to report a bug > --with-system-zlib use installed libz > + --with-zstd support zstd compressed debug sections > + (default=auto) > --with-gnu-ld assume the C compiler uses GNU ld default=no > --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib > --without-libiconv-prefix don't search for libiconv in includedir and libdir > @@ -1721,6 +1728,8 @@ Some influential environment variables: > YFLAGS The list of arguments that will be passed by default to $YACC. > This script will default YFLAGS to the empty string to avoid a > default value of `-d' given by some make applications. > + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config > + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config > XMKMF Path to xmkmf, Makefile generator for X Window System > > Use these variables to override the choices made by `configure' or to help > @@ -8242,7 +8251,8 @@ if test "$ac_res" != no; then : > fi > > > -# Link in zlib if we can. This allows us to read compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to read compressed debug > +# sections. > > # Use the system's zlib library. > zlibdir="-L\$(top_builddir)/../zlib" > @@ -8262,6 +8272,126 @@ fi > > > > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +else > + with_zstd=auto > +fi > + > + > +if test "$with_zstd" != no; then > + > +pkg_failed=no > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 > +$as_echo_n "checking for libzstd... " >&6; } > + > +if test -n "$ZSTD_CFLAGS"; then > + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > +if test -n "$ZSTD_LIBS"; then > + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > + > +if test $pkg_failed = no; then > + pkg_save_LDFLAGS="$LDFLAGS" > + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" > + cat confdefs.h - <<_ACEOF >conftest.$ac_ext > +/* end confdefs.h. */ > + > +int > +main () > +{ > + > + ; > + return 0; > +} > +_ACEOF > +if ac_fn_c_try_link "$LINENO"; then : > + > +else > + pkg_failed=yes > +fi > +rm -f core conftest.err conftest.$ac_objext \ > + conftest$ac_exeext conftest.$ac_ext > + LDFLAGS=$pkg_save_LDFLAGS > +fi > + > + > + > +if test $pkg_failed = yes; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then > + _pkg_short_errors_supported=yes > +else > + _pkg_short_errors_supported=no > +fi > + if test $_pkg_short_errors_supported = yes; then > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` > + else > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` > + fi > + # Put the nasty error message in config.log where it belongs > + echo "$ZSTD_PKG_ERRORS" >&5 > + > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +elif test $pkg_failed = untried; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +else > + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS > + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + > + > +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h > + > + > +fi > +fi > + > + > + > if test "X$prefix" = "XNONE"; then > acl_final_prefix="$ac_default_prefix" > else > @@ -17281,7 +17411,7 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" > # always want our bfd. > CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" > ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` > - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" > + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" > intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` > LIBS="-lbfd -liberty -lz $intl $LIBS" > { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 > @@ -17396,7 +17526,7 @@ fi > # always want our bfd. > CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" > ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` > - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" > + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" > intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` > LIBS="-lbfd -liberty -lz $intl $LIBS" > { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 > diff --git a/gdb/configure.ac b/gdb/configure.ac > index 215a91c5b27..e8cf808d280 100644 > --- a/gdb/configure.ac > +++ b/gdb/configure.ac > @@ -460,8 +460,10 @@ AC_SEARCH_LIBS(gethostbyname, nsl) > # Some systems (e.g. Solaris) have `socketpair' in libsocket. > AC_SEARCH_LIBS(socketpair, socket) > > -# Link in zlib if we can. This allows us to read compressed debug sections. > +# Link in zlib/zstd if we can. This allows us to read compressed debug > +# sections. > AM_ZLIB > +AM_ZSTD > > AM_ICONV > > diff --git a/ld/Makefile.am b/ld/Makefile.am > index d31021c13e2..bbe25f3aca2 100644 > --- a/ld/Makefile.am > +++ b/ld/Makefile.am > @@ -45,7 +45,7 @@ ELF_CLFAGS=-DELF_LIST_OPTIONS=@elf_list_options@ \ > -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ > WARN_CFLAGS = @WARN_CFLAGS@ > NO_WERROR = @NO_WERROR@ > -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) > +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) > > # We put the scripts in the directory $(scriptdir)/ldscripts. > # We can't put the scripts in $(datadir) because the SEARCH_DIR > @@ -959,7 +959,8 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai > ldbuildid.c > ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ > $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) > -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) > +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ > + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) > > # Dependency tracking for the generated emulation files. > EXTRA_ld_new_SOURCES += $(ALL_EMULATION_SOURCES) $(ALL_64_EMULATION_SOURCES) > diff --git a/ld/Makefile.in b/ld/Makefile.in > index ee0c98f65b0..400eed2717e 100644 > --- a/ld/Makefile.in > +++ b/ld/Makefile.in > @@ -126,7 +126,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ > $(top_srcdir)/../config/plugins.m4 \ > $(top_srcdir)/../config/po.m4 \ > $(top_srcdir)/../config/progtest.m4 \ > - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ > + $(top_srcdir)/../config/zlib.m4 \ > + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ > $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ > $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ > $(top_srcdir)/../bfd/version.m4 $(top_srcdir)/configure.ac > @@ -477,6 +478,8 @@ WARN_WRITE_STRINGS = @WARN_WRITE_STRINGS@ > XGETTEXT = @XGETTEXT@ > YACC = `if [ -f ../bison/bison ]; then echo ../bison/bison -y -L$(srcdir)/../bison/; else echo @YACC@; fi` > YFLAGS = -d > +ZSTD_CFLAGS = @ZSTD_CFLAGS@ > +ZSTD_LIBS = @ZSTD_LIBS@ > abs_builddir = @abs_builddir@ > abs_srcdir = @abs_srcdir@ > abs_top_builddir = @abs_top_builddir@ > @@ -564,7 +567,7 @@ ELF_CLFAGS = -DELF_LIST_OPTIONS=@elf_list_options@ \ > -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ > -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ > > -AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) > +AM_CFLAGS = $(WARN_CFLAGS) $(ELF_CLFAGS) $(JANSSON_CFLAGS) $(ZSTD_CFLAGS) > > # We put the scripts in the directory $(scriptdir)/ldscripts. > # We can't put the scripts in $(datadir) because the SEARCH_DIR > @@ -1011,7 +1014,9 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai > ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ > $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) > > -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) > +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) \ > + $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD_LIBS) $(JANSSON_LIBS) > + > # > # > # Build a dummy plugin using libtool. > diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 > index d20c542eed5..893e973e4f2 100644 > --- a/ld/aclocal.m4 > +++ b/ld/aclocal.m4 > @@ -1203,6 +1203,7 @@ m4_include([../config/plugins.m4]) > m4_include([../config/po.m4]) > m4_include([../config/progtest.m4]) > m4_include([../config/zlib.m4]) > +m4_include([../config/zstd.m4]) > m4_include([../libtool.m4]) > m4_include([../ltoptions.m4]) > m4_include([../ltsugar.m4]) > diff --git a/ld/config.in b/ld/config.in > index 0ccd79d59cd..3916740eee4 100644 > --- a/ld/config.in > +++ b/ld/config.in > @@ -162,6 +162,9 @@ > /* Define to 1 if you have the <windows.h> header file. */ > #undef HAVE_WINDOWS_H > > +/* Define to 1 if zstd is enabled. */ > +#undef HAVE_ZSTD > + > /* Define to the sub-directory in which libtool stores uninstalled libraries. > */ > #undef LT_OBJDIR > diff --git a/ld/configure b/ld/configure > index a1a07005400..175dd995b2b 100755 > --- a/ld/configure > +++ b/ld/configure > @@ -646,6 +646,8 @@ elf_plt_unwind_list_options > elf_shlib_list_options > elf_list_options > STRINGIFY > +ZSTD_LIBS > +ZSTD_CFLAGS > zlibinc > zlibdir > NATIVE_LIB_DIRS > @@ -679,9 +681,6 @@ WARN_CFLAGS_FOR_BUILD > WARN_CFLAGS > JANSSON_LIBS > JANSSON_CFLAGS > -PKG_CONFIG_LIBDIR > -PKG_CONFIG_PATH > -PKG_CONFIG > enable_libctf > ENABLE_LIBCTF_FALSE > ENABLE_LIBCTF_TRUE > @@ -711,6 +710,9 @@ LD > FGREP > SED > LIBTOOL > +PKG_CONFIG_LIBDIR > +PKG_CONFIG_PATH > +PKG_CONFIG > EGREP > CPP > GREP > @@ -855,6 +857,7 @@ enable_werror > enable_build_warnings > enable_nls > with_system_zlib > +with_zstd > ' > ac_precious_vars='build_alias > host_alias > @@ -868,14 +871,16 @@ CXX > CXXFLAGS > CCC > CPP > -CXXCPP > PKG_CONFIG > PKG_CONFIG_PATH > PKG_CONFIG_LIBDIR > +CXXCPP > JANSSON_CFLAGS > JANSSON_LIBS > YACC > -YFLAGS' > +YFLAGS > +ZSTD_CFLAGS > +ZSTD_LIBS' > > > # Initialize some variables set by options. > @@ -1552,6 +1557,8 @@ Optional Packages: > --with-lib-path=dir1:dir2... set default LIB_PATH > --with-sysroot=DIR Search for usr/lib et al within DIR. > --with-system-zlib use installed libz > + --with-zstd support zstd compressed debug sections > + (default=auto) > > Some influential environment variables: > CC C compiler command > @@ -1564,12 +1571,12 @@ Some influential environment variables: > CXX C++ compiler command > CXXFLAGS C++ compiler flags > CPP C preprocessor > - CXXCPP C++ preprocessor > PKG_CONFIG path to pkg-config utility > PKG_CONFIG_PATH > directories to add to pkg-config's search path > PKG_CONFIG_LIBDIR > path overriding pkg-config's built-in search path > + CXXCPP C++ preprocessor > JANSSON_CFLAGS > C compiler flags for JANSSON, overriding pkg-config > JANSSON_LIBS > @@ -1580,6 +1587,8 @@ Some influential environment variables: > YFLAGS The list of arguments that will be passed by default to $YACC. > This script will default YFLAGS to the empty string to avoid a > default value of `-d' given by some make applications. > + ZSTD_CFLAGS C compiler flags for ZSTD, overriding pkg-config > + ZSTD_LIBS linker flags for ZSTD, overriding pkg-config > > Use these variables to override the choices made by `configure' or to help > it to find libraries and programs with nonstandard names/locations. > @@ -5388,6 +5397,126 @@ $as_echo "$ac_cv_safe_to_define___extensions__" >&6; } > > > > + > + > + > + > + > + > +if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then > + if test -n "$ac_tool_prefix"; then > + # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. > +set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +PKG_CONFIG=$ac_cv_path_PKG_CONFIG > +if test -n "$PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 > +$as_echo "$PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + > +fi > +if test -z "$ac_cv_path_PKG_CONFIG"; then > + ac_pt_PKG_CONFIG=$PKG_CONFIG > + # Extract the first word of "pkg-config", so it can be a program name with args. > +set dummy pkg-config; ac_word=$2 > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > +$as_echo_n "checking for $ac_word... " >&6; } > +if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : > + $as_echo_n "(cached) " >&6 > +else > + case $ac_pt_PKG_CONFIG in > + [\\/]* | ?:[\\/]*) > + ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. > + ;; > + *) > + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > +for as_dir in $PATH > +do > + IFS=$as_save_IFS > + test -z "$as_dir" && as_dir=. > + for ac_exec_ext in '' $ac_executable_extensions; do > + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > + ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > + break 2 > + fi > +done > + done > +IFS=$as_save_IFS > + > + ;; > +esac > +fi > +ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG > +if test -n "$ac_pt_PKG_CONFIG"; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 > +$as_echo "$ac_pt_PKG_CONFIG" >&6; } > +else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > +fi > + > + if test "x$ac_pt_PKG_CONFIG" = x; then > + PKG_CONFIG="" > + else > + case $cross_compiling:$ac_tool_warned in > +yes:) > +{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 > +$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} > +ac_tool_warned=yes ;; > +esac > + PKG_CONFIG=$ac_pt_PKG_CONFIG > + fi > +else > + PKG_CONFIG="$ac_cv_path_PKG_CONFIG" > +fi > + > +fi > +if test -n "$PKG_CONFIG"; then > + _pkg_min_version=0.9.0 > + { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 > +$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } > + if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + else > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + PKG_CONFIG="" > + fi > +fi > + > case `pwd` in > *\ * | *\ *) > { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 > @@ -11491,7 +11620,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 11494 "configure" > +#line 11623 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -11597,7 +11726,7 @@ else > lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 > lt_status=$lt_dlunknown > cat > conftest.$ac_ext <<_LT_EOF > -#line 11600 "configure" > +#line 11729 "configure" > #include "confdefs.h" > > #if HAVE_DLFCN_H > @@ -15586,126 +15715,6 @@ fi > > > if test "x$enable_jansson" != "xno"; then > - > - > - > - > - > - > - > -if test "x$ac_cv_env_PKG_CONFIG_set" != "xset"; then > - if test -n "$ac_tool_prefix"; then > - # Extract the first word of "${ac_tool_prefix}pkg-config", so it can be a program name with args. > -set dummy ${ac_tool_prefix}pkg-config; ac_word=$2 > -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > -$as_echo_n "checking for $ac_word... " >&6; } > -if ${ac_cv_path_PKG_CONFIG+:} false; then : > - $as_echo_n "(cached) " >&6 > -else > - case $PKG_CONFIG in > - [\\/]* | ?:[\\/]*) > - ac_cv_path_PKG_CONFIG="$PKG_CONFIG" # Let the user override the test with a path. > - ;; > - *) > - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > -for as_dir in $PATH > -do > - IFS=$as_save_IFS > - test -z "$as_dir" && as_dir=. > - for ac_exec_ext in '' $ac_executable_extensions; do > - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > - ac_cv_path_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > - break 2 > - fi > -done > - done > -IFS=$as_save_IFS > - > - ;; > -esac > -fi > -PKG_CONFIG=$ac_cv_path_PKG_CONFIG > -if test -n "$PKG_CONFIG"; then > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $PKG_CONFIG" >&5 > -$as_echo "$PKG_CONFIG" >&6; } > -else > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > -$as_echo "no" >&6; } > -fi > - > - > -fi > -if test -z "$ac_cv_path_PKG_CONFIG"; then > - ac_pt_PKG_CONFIG=$PKG_CONFIG > - # Extract the first word of "pkg-config", so it can be a program name with args. > -set dummy pkg-config; ac_word=$2 > -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 > -$as_echo_n "checking for $ac_word... " >&6; } > -if ${ac_cv_path_ac_pt_PKG_CONFIG+:} false; then : > - $as_echo_n "(cached) " >&6 > -else > - case $ac_pt_PKG_CONFIG in > - [\\/]* | ?:[\\/]*) > - ac_cv_path_ac_pt_PKG_CONFIG="$ac_pt_PKG_CONFIG" # Let the user override the test with a path. > - ;; > - *) > - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR > -for as_dir in $PATH > -do > - IFS=$as_save_IFS > - test -z "$as_dir" && as_dir=. > - for ac_exec_ext in '' $ac_executable_extensions; do > - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then > - ac_cv_path_ac_pt_PKG_CONFIG="$as_dir/$ac_word$ac_exec_ext" > - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 > - break 2 > - fi > -done > - done > -IFS=$as_save_IFS > - > - ;; > -esac > -fi > -ac_pt_PKG_CONFIG=$ac_cv_path_ac_pt_PKG_CONFIG > -if test -n "$ac_pt_PKG_CONFIG"; then > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_pt_PKG_CONFIG" >&5 > -$as_echo "$ac_pt_PKG_CONFIG" >&6; } > -else > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > -$as_echo "no" >&6; } > -fi > - > - if test "x$ac_pt_PKG_CONFIG" = x; then > - PKG_CONFIG="" > - else > - case $cross_compiling:$ac_tool_warned in > -yes:) > -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 > -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} > -ac_tool_warned=yes ;; > -esac > - PKG_CONFIG=$ac_pt_PKG_CONFIG > - fi > -else > - PKG_CONFIG="$ac_cv_path_PKG_CONFIG" > -fi > - > -fi > -if test -n "$PKG_CONFIG"; then > - _pkg_min_version=0.9.0 > - { $as_echo "$as_me:${as_lineno-$LINENO}: checking pkg-config is at least version $_pkg_min_version" >&5 > -$as_echo_n "checking pkg-config is at least version $_pkg_min_version... " >&6; } > - if $PKG_CONFIG --atleast-pkgconfig-version $_pkg_min_version; then > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > -$as_echo "yes" >&6; } > - else > - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > -$as_echo "no" >&6; } > - PKG_CONFIG="" > - fi > -fi > if test -n "$PKG_CONFIG"; then : > > > @@ -17052,8 +17061,8 @@ $as_echo "#define HAVE_DECL_GETOPT 1" >>confdefs.h > > fi > > -# Link in zlib if we can. This allows us to read and write > -# compressed CTF sections. > +# Link in zlib/zstd if we can. This allows us to read and write > +# compressed debug sections. > > # Use the system's zlib library. > zlibdir="-L\$(top_builddir)/../zlib" > @@ -17072,6 +17081,126 @@ fi > > > > + > +# Check whether --with-zstd was given. > +if test "${with_zstd+set}" = set; then : > + withval=$with_zstd; > +else > + with_zstd=auto > +fi > + > + > +if test "$with_zstd" != no; then > + > +pkg_failed=no > +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd" >&5 > +$as_echo_n "checking for libzstd... " >&6; } > + > +if test -n "$ZSTD_CFLAGS"; then > + pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > +if test -n "$ZSTD_LIBS"; then > + pkg_cv_ZSTD_LIBS="$ZSTD_LIBS" > + elif test -n "$PKG_CONFIG"; then > + if test -n "$PKG_CONFIG" && \ > + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd\""; } >&5 > + ($PKG_CONFIG --exists --print-errors "libzstd") 2>&5 > + ac_status=$? > + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 > + test $ac_status = 0; }; then > + pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd" 2>/dev/null` > + test "x$?" != "x0" && pkg_failed=yes > +else > + pkg_failed=yes > +fi > + else > + pkg_failed=untried > +fi > + > +if test $pkg_failed = no; then > + pkg_save_LDFLAGS="$LDFLAGS" > + LDFLAGS="$LDFLAGS $pkg_cv_ZSTD_LIBS" > + cat confdefs.h - <<_ACEOF >conftest.$ac_ext > +/* end confdefs.h. */ > + > +int > +main () > +{ > + > + ; > + return 0; > +} > +_ACEOF > +if ac_fn_c_try_link "$LINENO"; then : > + > +else > + pkg_failed=yes > +fi > +rm -f core conftest.err conftest.$ac_objext \ > + conftest$ac_exeext conftest.$ac_ext > + LDFLAGS=$pkg_save_LDFLAGS > +fi > + > + > + > +if test $pkg_failed = yes; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then > + _pkg_short_errors_supported=yes > +else > + _pkg_short_errors_supported=no > +fi > + if test $_pkg_short_errors_supported = yes; then > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd" 2>&1` > + else > + ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd" 2>&1` > + fi > + # Put the nasty error message in config.log where it belongs > + echo "$ZSTD_PKG_ERRORS" >&5 > + > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +elif test $pkg_failed = untried; then > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 > +$as_echo "no" >&6; } > + > + if test "$with_zstd" = yes; then > + as_fn_error $? "--with-zstd was given, but pkgconfig/libzstd.pc is not found" "$LINENO" 5 > + fi > + > +else > + ZSTD_CFLAGS=$pkg_cv_ZSTD_CFLAGS > + ZSTD_LIBS=$pkg_cv_ZSTD_LIBS > + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 > +$as_echo "yes" >&6; } > + > + > +$as_echo "#define HAVE_ZSTD 1" >>confdefs.h > + > + > +fi > +fi > + > + > # When converting linker scripts into strings for use in emulation > # files, use astring.sed if the compiler supports ANSI string > # concatenation, or ostring.sed otherwise. This is to support the > diff --git a/ld/configure.ac b/ld/configure.ac > index eb55904c090..a491f9f1116 100644 > --- a/ld/configure.ac > +++ b/ld/configure.ac > @@ -34,6 +34,7 @@ AC_PROG_GREP > AC_GNU_SOURCE > AC_USE_SYSTEM_EXTENSIONS > AC_PROG_INSTALL > +PKG_PROG_PKG_CONFIG > > LT_INIT > ACX_LARGEFILE > @@ -297,7 +298,6 @@ AC_ARG_ENABLE([jansson], > [enable_jansson="no"]) > > if test "x$enable_jansson" != "xno"; then > - PKG_PROG_PKG_CONFIG > AS_IF([test -n "$PKG_CONFIG"], > [ > PKG_CHECK_MODULES(JANSSON, [jansson], > @@ -386,9 +386,10 @@ if test $ld_cv_decl_getopt_unistd_h = yes; then > [Is the prototype for getopt in <unistd.h> in the expected format?]) > fi > > -# Link in zlib if we can. This allows us to read and write > -# compressed CTF sections. > +# Link in zlib/zstd if we can. This allows us to read and write > +# compressed debug sections. > AM_ZLIB > +AM_ZSTD > > # When converting linker scripts into strings for use in emulation > # files, use astring.sed if the compiler supports ANSI string > diff --git a/ld/emultempl/elf.em b/ld/emultempl/elf.em > index c52484f35d3..acd66f907d1 100644 > --- a/ld/emultempl/elf.em > +++ b/ld/emultempl/elf.em > @@ -668,6 +668,15 @@ gld${EMULATION_NAME}_handle_option (int optc) > link_info.compress_debug = COMPRESS_DEBUG_GNU_ZLIB; > else if (strcasecmp (optarg, "zlib-gabi") == 0) > link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB; > + else if (strcasecmp (optarg, "zstd") == 0) > + { > +#ifdef HAVE_ZSTD > + link_info.compress_debug = COMPRESS_DEBUG_ZSTD; > +#else > + einfo (_ ("%F%P: --compress-debug-sections=zstd: ld is not built " > + "with zstd support\n")); > +#endif > + } > else > einfo (_("%F%P: invalid --compress-debug-sections option: \`%s'\n"), > optarg); > diff --git a/ld/ld.texi b/ld/ld.texi > index eabbec8faa9..9daed2e7e9f 100644 > --- a/ld/ld.texi > +++ b/ld/ld.texi > @@ -2863,10 +2863,12 @@ but for most Linux based systems it will be @code{both}. > @kindex --compress-debug-sections=zlib > @kindex --compress-debug-sections=zlib-gnu > @kindex --compress-debug-sections=zlib-gabi > +@kindex --compress-debug-sections=zstd > @item --compress-debug-sections=none > @itemx --compress-debug-sections=zlib > @itemx --compress-debug-sections=zlib-gnu > @itemx --compress-debug-sections=zlib-gabi > +@itemx --compress-debug-sections=zstd > On ELF platforms, these options control how DWARF debug sections are > compressed using zlib. > > @@ -2880,6 +2882,9 @@ sets the SHF_COMPRESSED flag in the sections' headers. > The @option{--compress-debug-sections=zlib} option is an alias for > @option{--compress-debug-sections=zlib-gabi}. > > +@option{--compress-debug-sections=zstd} compresses DWARF debug sections using > +zstd. > + > Note that this option overrides any compression in input debug > sections, so if a binary is linked with @option{--compress-debug-sections=none} > for example, then any compressed debug sections in input files will be > diff --git a/ld/ldmain.c b/ld/ldmain.c > index 1bbddaaad32..71af5935b74 100644 > --- a/ld/ldmain.c > +++ b/ld/ldmain.c > @@ -506,8 +506,12 @@ main (int argc, char **argv) > if ((link_info.compress_debug & COMPRESS_DEBUG)) > { > link_info.output_bfd->flags |= BFD_COMPRESS; > - if (link_info.compress_debug == COMPRESS_DEBUG_GABI_ZLIB) > - link_info.output_bfd->flags |= BFD_COMPRESS_GABI; > + if (link_info.compress_debug != COMPRESS_DEBUG_GNU_ZLIB) > + { > + link_info.output_bfd->flags |= BFD_COMPRESS_GABI; > + if (link_info.compress_debug == COMPRESS_DEBUG_ZSTD) > + link_info.output_bfd->flags |= BFD_COMPRESS_ZSTD; > + } > } > > ldwrite (); > diff --git a/ld/lexsup.c b/ld/lexsup.c > index 9225f71b3ce..299371fb775 100644 > --- a/ld/lexsup.c > +++ b/ld/lexsup.c > @@ -2146,8 +2146,8 @@ elf_static_list_options (FILE *file) > fprintf (file, _("\ > --package-metadata[=JSON] Generate package metadata note\n")); > fprintf (file, _("\ > - --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ > - Compress DWARF debug sections using zlib\n")); > + --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi|zstd]\n\ > + Compress DWARF debug sections\n")); > #ifdef DEFAULT_FLAG_COMPRESS_DEBUG > fprintf (file, _("\ > Default: zlib-gabi\n")); > diff --git a/ld/testsuite/ld-elf/compress.exp b/ld/testsuite/ld-elf/compress.exp > index cfdd767b8ab..7b8a31cc41d 100644 > --- a/ld/testsuite/ld-elf/compress.exp > +++ b/ld/testsuite/ld-elf/compress.exp > @@ -254,3 +254,20 @@ if { [regexp_diff tmpdir/$test.out $srcdir/$subdir/$test.rt] } then { > } else { > pass "$test_name" > } > + > +if { ![ld_assemble $as "--compress-debug-sections=zstd $srcdir/$subdir/empty.s" tmpdir/emptyzstd.o ] } { > + return > +} > +set build_tests { > + {"Build libzstdfoo.so with zstd compressed debug sections" > + "-shared" "-fPIC -g -Wa,--compress-debug-sections=zstd -Wl,--compress-debug-sections=zstd" > + {foo.c} {} "libzstdfoo.so"} > +} > +set run_tests { > + {"Run zstdnormal with libzstdfoo.so with zstd compressed debug sections" > + "tmpdir/begin.o tmpdir/libzstdfoo.so tmpdir/end.o -Wl,--compress-debug-sections=zstd" "" > + {main.c} "zstdnormal" "normal.out" "-Wa,--compress-debug-sections=zstd"} > +} > + > +run_cc_link_tests $build_tests > +run_ld_link_exec_tests $run_tests ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-24 6:53 ` Enze Li @ 2022-09-24 7:13 ` Fangrui Song 2022-09-27 18:06 ` Tom Tromey 0 siblings, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-24 7:13 UTC (permalink / raw) To: Enze Li Cc: Fangrui Song via Gdb-patches, Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi, binutils On 2022-09-24, Enze Li wrote: >Hi Fangrui, > >One nit, maybe not an important one. > >A proper committed message should include a link to the relevant bug on >Bugzilla. Like this, > > Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=29563 > >For more detail, see here[1]. > >[1] https://sourceware.org/gdb/wiki/ContributionChecklist#Properly_formatted_commit_messages > >Thanks, >Enze Hi Enze, Thanks for the notice. I see that gdb started to use "Bug:" since 2021-07. In binutils, there isn't such a convention and there is no commit using "Bug:". So I will keep using the umbrella link PR29397. I am on the fence whether the gdb one needs to use "Bug:" but I'll probably not use it this time. Ideally the gdb patch should be separate, but since the bfd's zstd dependency needs gdb build system's cooperation it is placed in the same patch:) I am still waiting for an approval from binutils side. Relatedly, I believe ChangeLog is no longer strictly enforced. There are many recent commits don't using ChangeLog. I intentionally skip ChangeLog for this patch since I believe it would provide little value. (I see "ANNOUNCEMENT: ChangeLog policy change after GDB 11".) ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-24 7:13 ` Fangrui Song @ 2022-09-27 18:06 ` Tom Tromey 0 siblings, 0 replies; 25+ messages in thread From: Tom Tromey @ 2022-09-27 18:06 UTC (permalink / raw) To: Fangrui Song via Gdb-patches Cc: Enze Li, Fangrui Song, Nick Clifton, Alan Modra, binutils, Jan Beulich > In binutils, there isn't such a convention and there is no commit using > "Bug:". So I will keep using the umbrella link PR29397. I think the commit script only looks for strings matching "PR name/NUM", like "PR binutils/29397" in this case. Just "PR29397" may not end up putting the commit info into the bug. Tom ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song ` (2 preceding siblings ...) 2022-09-24 6:53 ` Enze Li @ 2022-09-27 18:08 ` Tom Tromey 2022-09-27 18:53 ` Fangrui Song 2022-09-29 11:43 ` [PATCH v3] binutils, gdb: support zstd compressed debug sections Martin Liška 4 siblings, 1 reply; 25+ messages in thread From: Tom Tromey @ 2022-09-27 18:08 UTC (permalink / raw) To: Fangrui Song via Gdb-patches Cc: Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi, Fangrui Song, binutils >>>>> Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> writes: > PR29397 PR29563: The new configure option --with-zstd defaults to auto. > If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd > compressed debug sections for most tools. > * bfd: for addr2line, objdump --dwarf, gdb, etc > * gas: support --compress-debug-sections=zstd > * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd > * objcopy: support ELFCOMPRESS_ZSTD input for > --decompress-debug-sections and --compress-debug-sections=zstd > * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd > symbols, so gdb has to link against -lzstd in this patch. This also needs some change to the sims. I see failures like this with git master: make[4]: Entering directory '/home/tromey/gdb/build/sim/bpf' CCLD run ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_decompress' ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_isError' (unknown):176: error: undefined reference to 'ZSTD_compress' (unknown):179: error: undefined reference to 'ZSTD_isError' collect2: error: ld returned 1 exit status You'll see them all if you do a build with all sims enabled. thanks, Tom ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-27 18:08 ` Tom Tromey @ 2022-09-27 18:53 ` Fangrui Song 2022-09-28 11:52 ` [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) Pedro Alves 0 siblings, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-27 18:53 UTC (permalink / raw) To: Tom Tromey Cc: Fangrui Song via Gdb-patches, Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi, binutils On Tue, Sep 27, 2022 at 11:08 AM Tom Tromey <tom@tromey.com> wrote: > > >>>>> Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> writes: > > > PR29397 PR29563: The new configure option --with-zstd defaults to auto. > > If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd > > compressed debug sections for most tools. > > > * bfd: for addr2line, objdump --dwarf, gdb, etc > > * gas: support --compress-debug-sections=zstd > > * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd > > * objcopy: support ELFCOMPRESS_ZSTD input for > > --decompress-debug-sections and --compress-debug-sections=zstd > > * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd > > symbols, so gdb has to link against -lzstd in this patch. > > This also needs some change to the sims. > I see failures like this with git master: > > make[4]: Entering directory '/home/tromey/gdb/build/sim/bpf' > CCLD run > ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_decompress' > ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_isError' > (unknown):176: error: undefined reference to 'ZSTD_compress' > (unknown):179: error: undefined reference to 'ZSTD_isError' > collect2: error: ld returned 1 exit status > > You'll see them all if you do a build with all sims enabled. > > thanks, > Tom Thanks for the report and Tom's IRC chat with me about the `../../configure --enable-targets --enable-sim; make all-gdb` build. Tom confirmed that the latest commit `sim: Link ZSTD_LIBS` has fixed the problem. ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) 2022-09-27 18:53 ` Fangrui Song @ 2022-09-28 11:52 ` Pedro Alves 2022-09-28 19:00 ` Simon Marchi 0 siblings, 1 reply; 25+ messages in thread From: Pedro Alves @ 2022-09-28 11:52 UTC (permalink / raw) To: Fangrui Song, Tom Tromey; +Cc: Simon Marchi, Fangrui Song via Gdb-patches [-binutils] On 2022-09-27 7:53 p.m., Fangrui Song via Binutils wrote: > On Tue, Sep 27, 2022 at 11:08 AM Tom Tromey <tom@tromey.com> wrote: >> >>>>>>> Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> writes: >> >>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>> compressed debug sections for most tools. >> >>> * bfd: for addr2line, objdump --dwarf, gdb, etc >>> * gas: support --compress-debug-sections=zstd >>> * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd >>> * objcopy: support ELFCOMPRESS_ZSTD input for >>> --decompress-debug-sections and --compress-debug-sections=zstd >>> * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd >>> symbols, so gdb has to link against -lzstd in this patch. >> >> This also needs some change to the sims. >> I see failures like this with git master: >> >> make[4]: Entering directory '/home/tromey/gdb/build/sim/bpf' >> CCLD run >> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_decompress' >> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_isError' >> (unknown):176: error: undefined reference to 'ZSTD_compress' >> (unknown):179: error: undefined reference to 'ZSTD_isError' >> collect2: error: ld returned 1 exit status >> >> You'll see them all if you do a build with all sims enabled. >> >> thanks, >> Tom > > Thanks for the report and Tom's IRC chat with me about the > `../../configure --enable-targets --enable-sim; make all-gdb` build. > Tom confirmed that the latest commit `sim: Link ZSTD_LIBS` has fixed > the problem. > Hi! I'm running into a similar gdb build/link failure with current master. See patch below. -- >8 -- From a461b259690b62906978450533688b71ec939310 Mon Sep 17 00:00:00 2001 From: Pedro Alves <pedro@palves.net> Date: Wed, 28 Sep 2022 11:33:30 +0100 Subject: [PATCH] Fix GDB build: ELF support check & -lzstd GDB fails to build for me, on Ubuntu 20.04. I get: ... CXXLD gdb /usr/bin/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)': /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:1831: undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' /usr/bin/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)': /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:2117: undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' collect2: error: ld returned 1 exit status make[2]: *** [Makefile:2149: gdb] Error 1 make[2]: Leaving directory '/home/pedro/gdb/binutils-gdb/build/gdb' make[1]: *** [Makefile:11847: all-gdb] Error 2 make[1]: Leaving directory '/home/pedro/gdb/binutils-gdb/build' make: *** [Makefile:1004: all] Error 2 Those undefined functions exist in gdb/gcore-elf.c, which is only included in the build if GDB's configure thinks that the target you're configuring for is an ELF target. GDB's configure thinks my system isn't ELF, which is incorrect. For the ELF support check, gdb/config.log shows: configure:17387: checking for ELF support in BFD configure:17407: gcc -o conftest -I/home/pedro/gdb/binutils-gdb/src/gdb/../include -I../bfd -I/home/pedro/gdb/binutils-gdb/src/gdb/../bfd -g3 -O0 -L../bfd -L../libiberty -lzstd conftest.c -lbfd -liberty -lz -lncursesw -lm -ldl >&5 /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `decompress_contents': /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:42: undefined reference to `ZSTD_decompress' /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:44: undefined reference to `ZSTD_isError' /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `bfd_compress_section_contents': /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:195: undefined reference to `ZSTD_compress' /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:198: undefined reference to `ZSTD_isError' collect2: error: ld returned 1 exit status configure:17407: $? = 1 ... configure:17417: result: no Note how above, in the gcc command line, "-lzstd" appears before "-lbfd". That explain the link failure. It should appear after, like -lz does. This commit fixes it, by moving ZSTD_LIBS from LDFLAGS to LIBS, next to -lz, in GDB_AC_CHECK_BFD, and regenerating gdb/configure. Change-Id: I1f4128dde634e8ea04c9002904f1005a8b3a6863 --- gdb/acinclude.m4 | 4 ++-- gdb/configure | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 index 28846119dcb..62fa66c7af3 100644 --- a/gdb/acinclude.m4 +++ b/gdb/acinclude.m4 @@ -234,9 +234,9 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` - LIBS="-lbfd -liberty -lz $intl $LIBS" + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" AC_CACHE_CHECK( [$1], [$2], diff --git a/gdb/configure b/gdb/configure index 238b66f3c60..33677262783 100755 --- a/gdb/configure +++ b/gdb/configure @@ -17412,9 +17412,9 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` - LIBS="-lbfd -liberty -lz $intl $LIBS" + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 $as_echo_n "checking for ELF support in BFD... " >&6; } if ${gdb_cv_var_elf+:} false; then : @@ -17527,9 +17527,9 @@ fi # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` - LIBS="-lbfd -liberty -lz $intl $LIBS" + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 $as_echo_n "checking for Mach-O support in BFD... " >&6; } if ${gdb_cv_var_macho+:} false; then : base-commit: 67d1991b785bdfef1d70cddfa0202b99b43ccce9 -- 2.36.0 ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) 2022-09-28 11:52 ` [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) Pedro Alves @ 2022-09-28 19:00 ` Simon Marchi 2022-09-28 19:27 ` Pedro Alves 2022-09-28 19:30 ` Fangrui Song 0 siblings, 2 replies; 25+ messages in thread From: Simon Marchi @ 2022-09-28 19:00 UTC (permalink / raw) To: Pedro Alves, Fangrui Song, Tom Tromey; +Cc: Fangrui Song via Gdb-patches On 9/28/22 07:52, Pedro Alves wrote: > [-binutils] > > On 2022-09-27 7:53 p.m., Fangrui Song via Binutils wrote: >> On Tue, Sep 27, 2022 at 11:08 AM Tom Tromey <tom@tromey.com> wrote: >>> >>>>>>>> Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> writes: >>> >>>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>>> compressed debug sections for most tools. >>> >>>> * bfd: for addr2line, objdump --dwarf, gdb, etc >>>> * gas: support --compress-debug-sections=zstd >>>> * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd >>>> * objcopy: support ELFCOMPRESS_ZSTD input for >>>> --decompress-debug-sections and --compress-debug-sections=zstd >>>> * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd >>>> symbols, so gdb has to link against -lzstd in this patch. >>> >>> This also needs some change to the sims. >>> I see failures like this with git master: >>> >>> make[4]: Entering directory '/home/tromey/gdb/build/sim/bpf' >>> CCLD run >>> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_decompress' >>> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_isError' >>> (unknown):176: error: undefined reference to 'ZSTD_compress' >>> (unknown):179: error: undefined reference to 'ZSTD_isError' >>> collect2: error: ld returned 1 exit status >>> >>> You'll see them all if you do a build with all sims enabled. >>> >>> thanks, >>> Tom >> >> Thanks for the report and Tom's IRC chat with me about the >> `../../configure --enable-targets --enable-sim; make all-gdb` build. >> Tom confirmed that the latest commit `sim: Link ZSTD_LIBS` has fixed >> the problem. >> > > Hi! > > I'm running into a similar gdb build/link failure with current master. > > See patch below. > > -- >8 -- > From a461b259690b62906978450533688b71ec939310 Mon Sep 17 00:00:00 2001 > From: Pedro Alves <pedro@palves.net> > Date: Wed, 28 Sep 2022 11:33:30 +0100 > Subject: [PATCH] Fix GDB build: ELF support check & -lzstd > > GDB fails to build for me, on Ubuntu 20.04. I get: > > ... > CXXLD gdb > /usr/bin/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)': > /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:1831: undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' > /usr/bin/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)': > /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:2117: undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' > collect2: error: ld returned 1 exit status > make[2]: *** [Makefile:2149: gdb] Error 1 > make[2]: Leaving directory '/home/pedro/gdb/binutils-gdb/build/gdb' > make[1]: *** [Makefile:11847: all-gdb] Error 2 > make[1]: Leaving directory '/home/pedro/gdb/binutils-gdb/build' > make: *** [Makefile:1004: all] Error 2 > > Those undefined functions exist in gdb/gcore-elf.c, which is only > included in the build if GDB's configure thinks that the target you're > configuring for is an ELF target. GDB's configure thinks my system > isn't ELF, which is incorrect. > > For the ELF support check, gdb/config.log shows: > > configure:17387: checking for ELF support in BFD > configure:17407: gcc -o conftest -I/home/pedro/gdb/binutils-gdb/src/gdb/../include -I../bfd -I/home/pedro/gdb/binutils-gdb/src/gdb/../bfd -g3 -O0 -L../bfd -L../libiberty -lzstd conftest.c -lbfd -liberty -lz -lncursesw -lm -ldl >&5 > /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `decompress_contents': > /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:42: undefined reference to `ZSTD_decompress' > /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:44: undefined reference to `ZSTD_isError' > /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `bfd_compress_section_contents': > /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:195: undefined reference to `ZSTD_compress' > /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:198: undefined reference to `ZSTD_isError' > collect2: error: ld returned 1 exit status > configure:17407: $? = 1 > ... > configure:17417: result: no > > Note how above, in the gcc command line, "-lzstd" appears before > "-lbfd". That explain the link failure. It should appear after, like > -lz does. > > This commit fixes it, by moving ZSTD_LIBS from LDFLAGS to LIBS, next > to -lz, in GDB_AC_CHECK_BFD, and regenerating gdb/configure. > > Change-Id: I1f4128dde634e8ea04c9002904f1005a8b3a6863 > --- > gdb/acinclude.m4 | 4 ++-- > gdb/configure | 8 ++++---- > 2 files changed, 6 insertions(+), 6 deletions(-) > > diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 > index 28846119dcb..62fa66c7af3 100644 > --- a/gdb/acinclude.m4 > +++ b/gdb/acinclude.m4 > @@ -234,9 +234,9 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ > # always want our bfd. > CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" > ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` > - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" > + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" > intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` > - LIBS="-lbfd -liberty -lz $intl $LIBS" > + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" > AC_CACHE_CHECK( > [$1], > [$2], LGTM. Someone opened a bug about this: https://sourceware.org/bugzilla/show_bug.cgi?id=29630 Simon ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) 2022-09-28 19:00 ` Simon Marchi @ 2022-09-28 19:27 ` Pedro Alves 2022-09-28 19:30 ` Fangrui Song 1 sibling, 0 replies; 25+ messages in thread From: Pedro Alves @ 2022-09-28 19:27 UTC (permalink / raw) To: Simon Marchi, Fangrui Song, Tom Tromey; +Cc: Fangrui Song via Gdb-patches On 2022-09-28 8:00 p.m., Simon Marchi wrote: > On 9/28/22 07:52, Pedro Alves wrote: >> Note how above, in the gcc command line, "-lzstd" appears before >> "-lbfd". That explain the link failure. It should appear after, like >> -lz does. >> >> This commit fixes it, by moving ZSTD_LIBS from LDFLAGS to LIBS, next >> to -lz, in GDB_AC_CHECK_BFD, and regenerating gdb/configure. >> >> Change-Id: I1f4128dde634e8ea04c9002904f1005a8b3a6863 >> --- >> gdb/acinclude.m4 | 4 ++-- >> gdb/configure | 8 ++++---- >> 2 files changed, 6 insertions(+), 6 deletions(-) >> >> diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 >> index 28846119dcb..62fa66c7af3 100644 >> --- a/gdb/acinclude.m4 >> +++ b/gdb/acinclude.m4 >> @@ -234,9 +234,9 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ >> # always want our bfd. >> CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" >> ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` >> - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" >> + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" >> intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` >> - LIBS="-lbfd -liberty -lz $intl $LIBS" >> + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" >> AC_CACHE_CHECK( >> [$1], >> [$2], > > LGTM. Someone opened a bug about this: > > https://sourceware.org/bugzilla/show_bug.cgi?id=29630 Thanks. I added a Bug: tag and pushed it. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) 2022-09-28 19:00 ` Simon Marchi 2022-09-28 19:27 ` Pedro Alves @ 2022-09-28 19:30 ` Fangrui Song 2022-09-29 18:30 ` Pedro Alves 1 sibling, 1 reply; 25+ messages in thread From: Fangrui Song @ 2022-09-28 19:30 UTC (permalink / raw) To: Pedro Alves; +Cc: Simon Marchi, Tom Tromey, Fangrui Song via Gdb-patches On 2022-09-28, Simon Marchi wrote: >On 9/28/22 07:52, Pedro Alves wrote: >> [-binutils] >> >> On 2022-09-27 7:53 p.m., Fangrui Song via Binutils wrote: >>> On Tue, Sep 27, 2022 at 11:08 AM Tom Tromey <tom@tromey.com> wrote: >>>> >>>>>>>>> Fangrui Song via Gdb-patches <gdb-patches@sourceware.org> writes: >>>> >>>>> PR29397 PR29563: The new configure option --with-zstd defaults to auto. >>>>> If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support zstd >>>>> compressed debug sections for most tools. >>>> >>>>> * bfd: for addr2line, objdump --dwarf, gdb, etc >>>>> * gas: support --compress-debug-sections=zstd >>>>> * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd >>>>> * objcopy: support ELFCOMPRESS_ZSTD input for >>>>> --decompress-debug-sections and --compress-debug-sections=zstd >>>>> * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd >>>>> symbols, so gdb has to link against -lzstd in this patch. >>>> >>>> This also needs some change to the sims. >>>> I see failures like this with git master: >>>> >>>> make[4]: Entering directory '/home/tromey/gdb/build/sim/bpf' >>>> CCLD run >>>> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_decompress' >>>> ../../bfd/libbfd.a(compress.o):compress.c:function decompress_contents: error: undefined reference to 'ZSTD_isError' >>>> (unknown):176: error: undefined reference to 'ZSTD_compress' >>>> (unknown):179: error: undefined reference to 'ZSTD_isError' >>>> collect2: error: ld returned 1 exit status >>>> >>>> You'll see them all if you do a build with all sims enabled. >>>> >>>> thanks, >>>> Tom >>> >>> Thanks for the report and Tom's IRC chat with me about the >>> `../../configure --enable-targets --enable-sim; make all-gdb` build. >>> Tom confirmed that the latest commit `sim: Link ZSTD_LIBS` has fixed >>> the problem. >>> >> >> Hi! >> >> I'm running into a similar gdb build/link failure with current master. >> >> See patch below. >> >> -- >8 -- >> From a461b259690b62906978450533688b71ec939310 Mon Sep 17 00:00:00 2001 >> From: Pedro Alves <pedro@palves.net> >> Date: Wed, 28 Sep 2022 11:33:30 +0100 >> Subject: [PATCH] Fix GDB build: ELF support check & -lzstd >> >> GDB fails to build for me, on Ubuntu 20.04. I get: >> >> ... >> CXXLD gdb >> /usr/bin/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)': >> /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:1831: undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' >> /usr/bin/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)': >> /home/pedro/gdb/binutils-gdb/src/gdb/linux-tdep.c:2117: undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)' >> collect2: error: ld returned 1 exit status >> make[2]: *** [Makefile:2149: gdb] Error 1 >> make[2]: Leaving directory '/home/pedro/gdb/binutils-gdb/build/gdb' >> make[1]: *** [Makefile:11847: all-gdb] Error 2 >> make[1]: Leaving directory '/home/pedro/gdb/binutils-gdb/build' >> make: *** [Makefile:1004: all] Error 2 >> >> Those undefined functions exist in gdb/gcore-elf.c, which is only >> included in the build if GDB's configure thinks that the target you're >> configuring for is an ELF target. GDB's configure thinks my system >> isn't ELF, which is incorrect. >> >> For the ELF support check, gdb/config.log shows: >> >> configure:17387: checking for ELF support in BFD >> configure:17407: gcc -o conftest -I/home/pedro/gdb/binutils-gdb/src/gdb/../include -I../bfd -I/home/pedro/gdb/binutils-gdb/src/gdb/../bfd -g3 -O0 -L../bfd -L../libiberty -lzstd conftest.c -lbfd -liberty -lz -lncursesw -lm -ldl >&5 >> /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `decompress_contents': >> /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:42: undefined reference to `ZSTD_decompress' >> /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:44: undefined reference to `ZSTD_isError' >> /usr/bin/ld: ../bfd/libbfd.a(compress.o): in function `bfd_compress_section_contents': >> /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:195: undefined reference to `ZSTD_compress' >> /usr/bin/ld: /home/pedro/gdb/binutils-gdb/src/bfd/compress.c:198: undefined reference to `ZSTD_isError' >> collect2: error: ld returned 1 exit status >> configure:17407: $? = 1 >> ... >> configure:17417: result: no >> >> Note how above, in the gcc command line, "-lzstd" appears before >> "-lbfd". That explain the link failure. It should appear after, like >> -lz does. >> >> This commit fixes it, by moving ZSTD_LIBS from LDFLAGS to LIBS, next >> to -lz, in GDB_AC_CHECK_BFD, and regenerating gdb/configure. >> >> Change-Id: I1f4128dde634e8ea04c9002904f1005a8b3a6863 >> --- >> gdb/acinclude.m4 | 4 ++-- >> gdb/configure | 8 ++++---- >> 2 files changed, 6 insertions(+), 6 deletions(-) >> >> diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 >> index 28846119dcb..62fa66c7af3 100644 >> --- a/gdb/acinclude.m4 >> +++ b/gdb/acinclude.m4 >> @@ -234,9 +234,9 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ >> # always want our bfd. >> CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" >> ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` >> - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $ZSTD_LIBS $LDFLAGS" >> + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" >> intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` >> - LIBS="-lbfd -liberty -lz $intl $LIBS" >> + LIBS="-lbfd -liberty -lz $ZSTD_LIBS $intl $LIBS" >> AC_CACHE_CHECK( >> [$1], >> [$2], > >LGTM. Someone opened a bug about this: > >https://sourceware.org/bugzilla/show_bug.cgi?id=29630 > >Simon Thanks for the fix! I probably missed this because I use lld as the system linker and its archive semantic is more tolerant without --warn-backrefs... ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) 2022-09-28 19:30 ` Fangrui Song @ 2022-09-29 18:30 ` Pedro Alves 0 siblings, 0 replies; 25+ messages in thread From: Pedro Alves @ 2022-09-29 18:30 UTC (permalink / raw) To: Fangrui Song; +Cc: Simon Marchi, Tom Tromey, Fangrui Song via Gdb-patches On 2022-09-28 8:30 p.m., Fangrui Song wrote: > Thanks for the fix! I probably missed this because I use lld as the > system linker and its archive semantic is more tolerant without --warn-backrefs... Aaaaah, cool. I was wondering why more people weren't seeing this! I stumbled on this with GNU ld. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song ` (3 preceding siblings ...) 2022-09-27 18:08 ` Tom Tromey @ 2022-09-29 11:43 ` Martin Liška 2022-09-29 20:17 ` Fangrui Song 4 siblings, 1 reply; 25+ messages in thread From: Martin Liška @ 2022-09-29 11:43 UTC (permalink / raw) To: Fangrui Song, Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi Cc: binutils, gdb-patches, Richard Biener, Michael Matz Hello. Thanks for the patch set. Right now, we configure our binutils package with: --enable-compressed-debug-sections=gas and we would appreciate a new configure option that will selected default compression algorithm. Having that, we will proudly switch to zstd. Can you please prepare such patch? Thanks, Martin ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections 2022-09-29 11:43 ` [PATCH v3] binutils, gdb: support zstd compressed debug sections Martin Liška @ 2022-09-29 20:17 ` Fangrui Song 0 siblings, 0 replies; 25+ messages in thread From: Fangrui Song @ 2022-09-29 20:17 UTC (permalink / raw) To: Martin Liška Cc: Alan Modra, Jan Beulich, Nick Clifton, Simon Marchi, binutils, gdb-patches, Richard Biener, Michael Matz On 2022-09-29, Martin Liška wrote: >Hello. > >Thanks for the patch set. Right now, we configure our binutils package with: >--enable-compressed-debug-sections=gas and we would appreciate a new configure >option that will selected default compression algorithm. Having that, we will >proudly switch to zstd. > >Can you please prepare such patch? > >Thanks, >Martin > I left a comment on https://sourceware.org/bugzilla/show_bug.cgi?id=19109 that the --enable-compressed-debug-sections=gas syntax needs extension to support zstd. Perhaps the easiest solution (and circumvents the syntax problem) is to add a new option: --enable-zstd-compressed-debug-sections, but then there is some duplication... ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2022-09-29 20:17 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-09-23 4:08 [PATCH v3] binutils, gdb: support zstd compressed debug sections Fangrui Song 2022-09-23 14:32 ` Simon Marchi 2022-09-26 5:12 ` Alan Modra 2022-09-26 7:20 ` Fangrui Song 2022-09-26 13:30 ` Alan Modra 2022-09-26 14:08 ` Simon Marchi 2022-09-27 0:33 ` Alan Modra 2022-09-23 15:45 ` Nick Clifton 2022-09-23 15:58 ` Simon Marchi 2022-09-23 18:20 ` Fangrui Song 2022-09-23 18:57 ` Simon Marchi 2022-09-23 20:34 ` Fangrui Song 2022-09-24 5:43 ` Eli Zaretskii 2022-09-24 6:53 ` Enze Li 2022-09-24 7:13 ` Fangrui Song 2022-09-27 18:06 ` Tom Tromey 2022-09-27 18:08 ` Tom Tromey 2022-09-27 18:53 ` Fangrui Song 2022-09-28 11:52 ` [PATCH] Fix GDB build: ELF support check & -lzstd (was: Re: [PATCH v3] binutils, gdb: support zstd compressed debug sections) Pedro Alves 2022-09-28 19:00 ` Simon Marchi 2022-09-28 19:27 ` Pedro Alves 2022-09-28 19:30 ` Fangrui Song 2022-09-29 18:30 ` Pedro Alves 2022-09-29 11:43 ` [PATCH v3] binutils, gdb: support zstd compressed debug sections Martin Liška 2022-09-29 20:17 ` Fangrui Song
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).