From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <3juknYwcKCCEJ7PHO7VDLLDIB.9LJ8FKRQFIPPLRO9BT7OB.LOD@flex--maskray.bounces.google.com> Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by sourceware.org (Postfix) with ESMTPS id 7627F3858D3C for ; Mon, 19 Sep 2022 04:01:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7627F3858D3C Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=flex--maskray.bounces.google.com Received: by mail-yb1-xb4a.google.com with SMTP id n85-20020a254058000000b006b0148d96f7so12156727yba.2 for ; Sun, 18 Sep 2022 21:01:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date; bh=0Zc+s+y++eipo4V6/LKRCI66UMpf3KZw88VrVSz4EFI=; b=NqNSOSTNajIE4EW5iiMS5ay8CCBwNoYxFbLcvAHGkhq0HO4a4zx0hQuCnNxlKOQNMx Sl1WxkMHXxn60yy9Y9E8hhbSaVDpXv0a0ztHEIicbQCLPZUrZB7JNsHfPnR7Hi1QXa3D kmyYEZn0Mo6smu0rIPnrJpxzyWirq8vpsxCTSbsw7r+Py8saFd+UF2LOpFxxDb2yaR6F 4hyXIca+iuUDHhv3V9n3AGywjT2mw2TXiKt3aXouD0SvovPt+MIHsFAnC6lNdNM2+LI6 mLNPwDx3IDYRzCYjpGtu4Vh4waMSvw2A8Z57w6QnaQuDUyhYchY9h/tqqQpCSoeZ4c6+ NqIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date; bh=0Zc+s+y++eipo4V6/LKRCI66UMpf3KZw88VrVSz4EFI=; b=rEUWsFJe7HsSMTVuGsqBbr+4QFQp0cggbZlqhpBDAZo+UvvG/H/tv5prBxEn6Av4tI KRoRO6YVSaVO1e9mhswpdHoz8wj/K6teX8YaqYv8E1v5nqoqT7VCUKCpFPiTSXDqU1Bs 9Ck4i0wHguCwS0KeEZtFcQKLHvWq1lYY2XiX7MKDvLRdHFOWS+Bfl1us3sW8Cvix4NXU 56IDeCpb2Ml70AYACW4d/Z3quqZzrWaMeW6a5fy5et7fTDPRX36+xoURYVp3xmhnbCps gBIsY8/gH7s27sI1v9VfUYr0RwNzGRTGkNc1GtN72ArBxgtIvAG2o3fKMACKym5jufbR bihA== X-Gm-Message-State: ACrzQf2Gqw7dDR0zexRQuhD+1L8KeYXK/eKiTZ4S6xo4SfTFn7Lpcsyr vuOxiHY7GgnEf62gwwGksDiU86hGIAQxJf7azTsnX+WiXZQD99miFFayN4aj8zeb4Rowyvm13ur cmjsU5b0/pPMDr1bcJ3BnELIJf1Es6iW6SpPEh4Ltm2B9snumxl7d/NLESsa/vuaL3g== X-Google-Smtp-Source: AMsMyM6sZbMHwsv7MH/duCr5xT6ycE/MKtAWID2vUa6Az1uIIYnptuwdXky/L65q2OMKobh1XRlWJFTQtzju X-Received: from maskray1.svl.corp.google.com ([2620:15c:2ce:200:6d89:a024:3449:73c8]) (user=maskray job=sendgmr) by 2002:a25:af13:0:b0:6ae:3166:1aee with SMTP id a19-20020a25af13000000b006ae31661aeemr12762496ybh.288.1663560078813; Sun, 18 Sep 2022 21:01:18 -0700 (PDT) Date: Sun, 18 Sep 2022 21:01:15 -0700 Mime-Version: 1.0 Message-ID: <20220919040115.31326-1-maskray@google.com> Subject: [PATCH] binutils, gdb: support zstd compressed debug sections From: Fangrui Song To: binutils@sourceware.org, gdb-patches@sourceware.org Cc: Fangrui Song Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-21.1 required=5.0 tests=BAYES_00,DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: PR29397 PR29563: If --with-system-zstd is specified, define HAVE_ZSTD_H and support zstd compressed debug sections for most tools. * bfd: for addr2line, objdump --dwarf, gdb, etc * gas: support --compress-debug-sections=zstd * ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd * objcopy: support ELFCOMPRESS_ZSTD input for --decompress-debug-sections and --compress-debug-sections=zstd * objdump: --dwarf supports ELFCOMPRESS_ZSTD input * gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd symbols, so gdb has to link against -lzstd in this patch. If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We can avoid HAVE_ZSTD_H if binutils-gdb has a zstd/ subproject like zlib/, but this is too heavyweight, so don't do it for now. ``` % ld/ld-new a.o ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... % ld/ld-new a.o --compress-debug-sections=zstd ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support % binutils/objcopy a.o --compress-debug-sections=zstd binutils/objcopy: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support ... ``` --- bfd/Makefile.am | 4 +- bfd/Makefile.in | 7 +- bfd/aclocal.m4 | 1 + bfd/bfd-in.h | 3 +- bfd/bfd-in2.h | 9 ++- bfd/bfd.c | 24 +++++-- bfd/compress.c | 72 +++++++++++++++----- bfd/config.in | 3 + bfd/configure | 24 ++++++- bfd/configure.ac | 3 +- bfd/elf.c | 11 +++ bfd/elfxx-target.h | 6 +- bfd/section.c | 3 +- binutils/doc/binutils.texi | 16 +++-- binutils/objcopy.c | 12 +++- binutils/testsuite/binutils-all/compress.exp | 44 ++++++++++++ config/zstd.m4 | 12 ++++ configure | 9 +++ configure.ac | 3 + gas/Makefile.am | 4 +- gas/Makefile.in | 7 +- gas/aclocal.m4 | 1 + gas/as.c | 13 +++- gas/compress-debug.c | 60 +++++++++++++--- gas/compress-debug.h | 10 +-- gas/config.in | 3 + gas/configure | 24 ++++++- gas/configure.ac | 3 +- gas/doc/as.texi | 11 +-- gas/write.c | 36 +++++----- gdb/Makefile.in | 5 +- gdb/acinclude.m4 | 3 +- gdb/configure | 25 ++++++- gdb/configure.ac | 4 +- ld/Makefile.am | 6 +- ld/Makefile.in | 9 ++- ld/aclocal.m4 | 1 + ld/configure | 26 +++++-- ld/configure.ac | 5 +- ld/emultempl/elf.em | 9 +++ ld/ld.texi | 5 ++ ld/ldmain.c | 8 ++- ld/lexsup.c | 4 +- 43 files changed, 435 insertions(+), 113 deletions(-) create mode 100644 config/zstd.m4 diff --git a/bfd/Makefile.am b/bfd/Makefile.am index 04fb335d47d..c350c60e0fd 100644 --- a/bfd/Makefile.am +++ b/bfd/Makefile.am @@ -55,6 +55,8 @@ endif ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ + WARN_CFLAGS = @WARN_CFLAGS@ NO_WERROR = @NO_WERROR@ AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) @@ -776,7 +778,7 @@ ofiles: stamp-ofiles ; @true libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD) libbfd_la_LDFLAGS += -release `cat libtool-soversion` @SHARED_LDFLAGS@ # libtool will build .libs/libbfd.a. We create libbfd.a in the build diff --git a/bfd/Makefile.in b/bfd/Makefile.in index e05b0c9d156..5ecd997d3b7 100644 --- a/bfd/Makefile.in +++ b/bfd/Makefile.in @@ -125,7 +125,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../config/acx.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/bfd.m4 $(top_srcdir)/warning.m4 \ @@ -492,6 +493,7 @@ top_srcdir = @top_srcdir@ wordsize = @wordsize@ zlibdir = @zlibdir@ zlibinc = @zlibinc@ +zstdlib = @zstdlib@ AUTOMAKE_OPTIONS = no-dist foreign info-in-builddir no-texinfo.tex ACLOCAL_AMFLAGS = -I . -I .. -I ../config MOSTLYCLEANFILES = ofiles stamp-ofiles $(MKDOC) doc/*.o doc/*.stamp @@ -520,6 +522,7 @@ libbfd_la_LDFLAGS = $(am__append_1) -release `cat libtool-soversion` \ # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ AM_CFLAGS = $(WARN_CFLAGS) $(ZLIBINC) AM_CPPFLAGS = -DBINDIR='"$(bindir)"' -DLIBDIR='"$(libdir)"' \ @LARGEFILE_CPPFLAGS@ @HDEFINES@ @COREFLAG@ @TDEFINES@ \ @@ -1199,7 +1202,7 @@ OFILES = $(BFD_BACKENDS) $(BFD_MACHINES) @COREFILE@ @bfd64_libs@ libbfd_la_SOURCES = $(BFD32_LIBS_CFILES) EXTRA_libbfd_la_SOURCES = $(CFILES) libbfd_la_DEPENDENCIES = $(OFILES) ofiles -libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) +libbfd_la_LIBADD = `cat ofiles` @SHARED_LIBADD@ $(LIBDL) $(ZLIB) $(ZSTD) # libtool will build .libs/libbfd.a. We create libbfd.a in the build # directory so that we don't have to convert all the programs that use diff --git a/bfd/aclocal.m4 b/bfd/aclocal.m4 index 0f8aa1d4518..6e52bc63311 100644 --- a/bfd/aclocal.m4 +++ b/bfd/aclocal.m4 @@ -1180,6 +1180,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/bfd/bfd-in.h b/bfd/bfd-in.h index 8605056aefe..4765ea80536 100644 --- a/bfd/bfd-in.h +++ b/bfd/bfd-in.h @@ -342,7 +342,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections diff --git a/bfd/bfd-in2.h b/bfd/bfd-in2.h index 79fcc4eb912..f5cc4db4333 100644 --- a/bfd/bfd-in2.h +++ b/bfd/bfd-in2.h @@ -349,7 +349,8 @@ enum compressed_debug_section_type COMPRESS_DEBUG_NONE = 0, COMPRESS_DEBUG = 1 << 0, COMPRESS_DEBUG_GNU_ZLIB = COMPRESS_DEBUG | 1 << 1, - COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2 + COMPRESS_DEBUG_GABI_ZLIB = COMPRESS_DEBUG | 1 << 2, + COMPRESS_DEBUG_ZSTD = COMPRESS_DEBUG | 1 << 3 }; /* This structure is used to keep track of stabs in sections @@ -962,7 +963,8 @@ typedef struct bfd_section unsigned int compress_status : 2; #define COMPRESS_SECTION_NONE 0 #define COMPRESS_SECTION_DONE 1 -#define DECOMPRESS_SECTION_SIZED 2 +#define DECOMPRESS_SECTION_ZLIB 2 +#define DECOMPRESS_SECTION_ZSTD 3 /* The following flags are used by the ELF linker. */ @@ -6637,6 +6639,8 @@ struct bfd #define BFD_ARCHIVE_FULL_PATH 0x100000 #define BFD_CLOSED_BY_CACHE 0x200000 + /* Compress sections in this BFD with SHF_COMPRESSED zstd. */ +#define BFD_COMPRESS_ZSTD 0x400000 /* Flags bits to be saved in bfd_preserve_save. */ #define BFD_FLAGS_SAVED \ @@ -7271,6 +7275,7 @@ void bfd_update_compression_header bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); diff --git a/bfd/bfd.c b/bfd/bfd.c index 0a21db11fd6..db5aafc783c 100644 --- a/bfd/bfd.c +++ b/bfd/bfd.c @@ -177,6 +177,9 @@ CODE_FRAGMENT .#define BFD_ARCHIVE_FULL_PATH 0x100000 . .#define BFD_CLOSED_BY_CACHE 0x200000 + +. {* Compress sections in this BFD with SHF_COMPRESSED zstd. *} +.#define BFD_COMPRESS_ZSTD 0x400000 . . {* Flags bits to be saved in bfd_preserve_save. *} .#define BFD_FLAGS_SAVED \ @@ -2500,6 +2503,9 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, { const struct elf_backend_data *bed = get_elf_backend_data (abfd); struct bfd_elf_section_data * esd = elf_section_data (sec); + const unsigned int ch_type = abfd->flags & BFD_COMPRESS_ZSTD + ? ELFCOMPRESS_ZSTD + : ELFCOMPRESS_ZLIB; /* Set the SHF_COMPRESSED bit. */ elf_section_flags (sec) |= SHF_COMPRESSED; @@ -2507,7 +2513,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, if (bed->s->elfclass == ELFCLASS32) { Elf32_External_Chdr *echdr = (Elf32_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, sec->size, &echdr->ch_size); bfd_put_32 (abfd, 1u << sec->alignment_power, &echdr->ch_addralign); @@ -2518,7 +2524,7 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, else { Elf64_External_Chdr *echdr = (Elf64_External_Chdr *) contents; - bfd_put_32 (abfd, ELFCOMPRESS_ZLIB, &echdr->ch_type); + bfd_put_32 (abfd, ch_type, &echdr->ch_type); bfd_put_32 (abfd, 0, &echdr->ch_reserved); bfd_put_64 (abfd, sec->size, &echdr->ch_size); bfd_put_64 (abfd, UINT64_C (1) << sec->alignment_power, @@ -2553,14 +2559,15 @@ bfd_update_compression_header (bfd *abfd, bfd_byte *contents, SYNOPSIS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power); DESCRIPTION - Check the compression header at CONTENTS of SEC in ABFD and - store the uncompressed size in UNCOMPRESSED_SIZE and the - uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER - if the compression header is valid. + Check the compression header at CONTENTS of SEC in ABFD and store the + ch_type in CH_TYPE, uncompressed size in UNCOMPRESSED_SIZE, and the + uncompressed data alignment in UNCOMPRESSED_ALIGNMENT_POWER if the + compression header is valid. RETURNS Return TRUE if the compression header is valid. @@ -2569,6 +2576,7 @@ RETURNS bool bfd_check_compression_header (bfd *abfd, bfd_byte *contents, asection *sec, + unsigned int *ch_type, bfd_size_type *uncompressed_size, unsigned int *uncompressed_alignment_power) { @@ -2591,7 +2599,9 @@ bfd_check_compression_header (bfd *abfd, bfd_byte *contents, chdr.ch_size = bfd_get_64 (abfd, &echdr->ch_size); chdr.ch_addralign = bfd_get_64 (abfd, &echdr->ch_addralign); } - if (chdr.ch_type == ELFCOMPRESS_ZLIB + *ch_type = chdr.ch_type; + if ((chdr.ch_type == ELFCOMPRESS_ZLIB + || chdr.ch_type == ELFCOMPRESS_ZSTD) && chdr.ch_addralign == (chdr.ch_addralign & -chdr.ch_addralign)) { *uncompressed_size = chdr.ch_size; diff --git a/bfd/compress.c b/bfd/compress.c index b2e39826e38..441144a66e1 100644 --- a/bfd/compress.c +++ b/bfd/compress.c @@ -20,18 +20,31 @@ #include "sysdep.h" #include +#ifdef HAVE_ZSTD_H +#include +#endif #include "bfd.h" +#include "elf-bfd.h" #include "libbfd.h" #include "safe-ctype.h" #define MAX_COMPRESSION_HEADER_SIZE 24 static bool -decompress_contents (bfd_byte *compressed_buffer, +decompress_contents (bool is_zstd, bfd_byte *compressed_buffer, bfd_size_type compressed_size, bfd_byte *uncompressed_buffer, bfd_size_type uncompressed_size) { + if (is_zstd) + { +#ifdef HAVE_ZSTD_H + size_t ret = ZSTD_decompress (uncompressed_buffer, uncompressed_size, + compressed_buffer, compressed_size); + return !ZSTD_isError (ret); +#endif + } + z_stream strm; int rc; @@ -69,7 +82,7 @@ decompress_contents (bfd_byte *compressed_buffer, } /* Compress data of the size specified in @var{uncompressed_size} - and pointed to by @var{uncompressed_buffer} using zlib and store + and pointed to by @var{uncompressed_buffer} using zlib/zstd and store as the contents field. This function assumes the contents field was allocated using bfd_malloc() or equivalent. @@ -150,9 +163,10 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, sec->size = orig_uncompressed_size; if (decompress) { - if (!decompress_contents (uncompressed_buffer - + orig_compression_header_size, - zlib_size, buffer, buffer_size)) + if (!decompress_contents ( + sec->compress_status == DECOMPRESS_SECTION_ZSTD, + uncompressed_buffer + orig_compression_header_size, + zlib_size, buffer, buffer_size)) { bfd_set_error (bfd_error_bad_value); bfd_release (abfd, buffer); @@ -175,10 +189,23 @@ bfd_compress_section_contents (bfd *abfd, sec_ptr sec, } else { - if (compress ((Bytef*) buffer + header_size, - &compressed_size, - (const Bytef*) uncompressed_buffer, - uncompressed_size) != Z_OK) + if (abfd->flags & BFD_COMPRESS_ZSTD) + { +#if HAVE_ZSTD_H + compressed_size = ZSTD_compress ( + buffer + header_size, compressed_size, uncompressed_buffer, + uncompressed_size, ZSTD_CLEVEL_DEFAULT); + if (ZSTD_isError (compressed_size)) + { + bfd_release (abfd, buffer); + bfd_set_error (bfd_error_bad_value); + return 0; + } +#endif + } + else if (compress ((Bytef *)buffer + header_size, &compressed_size, + (const Bytef *)uncompressed_buffer, uncompressed_size) + != Z_OK) { bfd_release (abfd, buffer); bfd_set_error (bfd_error_bad_value); @@ -237,6 +264,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) bfd_size_type save_rawsize; bfd_byte *compressed_buffer; unsigned int compression_header_size; + unsigned int compress_status = sec->compress_status; if (abfd->direction != write_direction && sec->rawsize != 0) sz = sec->rawsize; @@ -248,7 +276,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) return true; } - switch (sec->compress_status) + switch (compress_status) { case COMPRESS_SECTION_NONE: if (p == NULL) @@ -298,7 +326,8 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) *ptr = p; return true; - case DECOMPRESS_SECTION_SIZED: + case DECOMPRESS_SECTION_ZLIB: + case DECOMPRESS_SECTION_ZSTD: /* Read in the full compressed section contents. */ compressed_buffer = (bfd_byte *) bfd_malloc (sec->compressed_size); if (compressed_buffer == NULL) @@ -316,7 +345,7 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Restore rawsize and size. */ sec->rawsize = save_rawsize; sec->size = save_size; - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = compress_status; if (!ret) goto fail_compressed; @@ -330,8 +359,10 @@ bfd_get_full_section_contents (bfd *abfd, sec_ptr sec, bfd_byte **ptr) /* Set header size to the zlib header size if it is a SHF_COMPRESSED section. */ compression_header_size = 12; - if (!decompress_contents (compressed_buffer + compression_header_size, - sec->compressed_size - compression_header_size, p, sz)) + if (!decompress_contents (compress_status == DECOMPRESS_SECTION_ZSTD, + compressed_buffer + compression_header_size, + sec->compressed_size - compression_header_size, + p, sz)) { bfd_set_error (bfd_error_bad_value); if (p != *ptr) @@ -381,7 +412,8 @@ DESCRIPTION void bfd_cache_section_contents (asection *sec, void *contents) { - if (sec->compress_status == DECOMPRESS_SECTION_SIZED) + if (sec->compress_status == DECOMPRESS_SECTION_ZLIB + || sec->compress_status == DECOMPRESS_SECTION_ZSTD) sec->compress_status = COMPRESS_SECTION_DONE; sec->contents = contents; sec->flags |= SEC_IN_MEMORY; @@ -418,6 +450,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, int compression_header_size; int header_size; unsigned int saved = sec->compress_status; + unsigned int ch_type; bool compressed; *uncompressed_align_pow_p = 0; @@ -448,7 +481,7 @@ bfd_is_section_compressed_with_header (bfd *abfd, sec_ptr sec, { if (compression_header_size != 0) { - if (!bfd_check_compression_header (abfd, header, sec, + if (!bfd_check_compression_header (abfd, header, sec, &ch_type, uncompressed_size_p, uncompressed_align_pow_p)) compression_header_size = -1; @@ -507,7 +540,7 @@ SYNOPSIS DESCRIPTION Record compressed section size, update section size with decompressed size and set compress_status to - DECOMPRESS_SECTION_SIZED. + DECOMPRESS_SECTION_{ZLIB,ZSTD}. Return @code{FALSE} if the section is not a valid compressed section. Otherwise, return @code{TRUE}. @@ -521,6 +554,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) int header_size; bfd_size_type uncompressed_size; unsigned int uncompressed_alignment_power = 0; + unsigned int ch_type; z_stream strm; compression_header_size = bfd_get_compression_header_size (abfd, sec); @@ -550,6 +584,7 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) uncompressed_size = bfd_getb64 (header + 4); } else if (!bfd_check_compression_header (abfd, header, sec, + &ch_type, &uncompressed_size, &uncompressed_alignment_power)) { @@ -569,7 +604,8 @@ bfd_init_section_decompress_status (bfd *abfd, sec_ptr sec) sec->compressed_size = sec->size; sec->size = uncompressed_size; bfd_set_section_alignment (sec, uncompressed_alignment_power); - sec->compress_status = DECOMPRESS_SECTION_SIZED; + sec->compress_status = ch_type == ELFCOMPRESS_ZSTD ? DECOMPRESS_SECTION_ZSTD + : DECOMPRESS_SECTION_ZLIB; return true; } diff --git a/bfd/config.in b/bfd/config.in index f54a3cacbea..c0e9ce9e30d 100644 --- a/bfd/config.in +++ b/bfd/config.in @@ -232,6 +232,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if configured with zstd support. */ +#undef HAVE_ZSTD_H + /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR diff --git a/bfd/configure b/bfd/configure index 075d2ee0a1b..eed526a1222 100755 --- a/bfd/configure +++ b/bfd/configure @@ -652,6 +652,7 @@ TDEFINES SHARED_LIBADD SHARED_LDFLAGS LIBM +zstdlib zlibinc zlibdir EXEEXT_FOR_BUILD @@ -839,6 +840,7 @@ enable_maintainer_mode enable_install_libbfd enable_nls with_system_zlib +with_system_zstd ' ac_precious_vars='build_alias host_alias @@ -1511,6 +1513,7 @@ Optional Packages: Binutils" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-system-zstd use installed zstd Some influential environment variables: CC C compiler command @@ -11086,7 +11089,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11089 "configure" +#line 11092 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11192,7 +11195,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11195 "configure" +#line 11198 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -12995,7 +12998,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. # Use the system's zlib library. @@ -13015,6 +13018,21 @@ fi + zstdlib= + +# Check whether --with-system-zstd was given. +if test "${with_system_zstd+set}" = set; then : + withval=$with_system_zstd; if test x$with_system_zstd = xyes ; then + zstdlib=-lzstd + $as_echo "#define HAVE_ZSTD_H 1" >>confdefs.h + + fi + +fi + + + + save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" { $as_echo "$as_me:${as_lineno-$LINENO}: checking compiler support for hidden visibility" >&5 diff --git a/bfd/configure.ac b/bfd/configure.ac index 28f3d1afce6..d1b2f76b31a 100644 --- a/bfd/configure.ac +++ b/bfd/configure.ac @@ -230,9 +230,10 @@ AC_CHECK_DECLS([basename, ffs, stpcpy, asprintf, vasprintf, strnlen]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug sections. # This is used only by compress.c. AM_ZLIB +AM_ZSTD save_CFLAGS="$CFLAGS" CFLAGS="$CFLAGS -Werror" diff --git a/bfd/elf.c b/bfd/elf.c index 2eb2a97370a..5aeec6a9587 100644 --- a/bfd/elf.c +++ b/bfd/elf.c @@ -1259,6 +1259,17 @@ _bfd_elf_make_section_from_shdr (bfd *abfd, abfd, name); return false; } +#ifndef HAVE_ZSTD_H + if (newsect->compress_status & DECOMPRESS_SECTION_ZSTD) + { + _bfd_error_handler + /* xgettext:c-format */ + (_ ("%pB: section %s is compressed with zstd, but BFD " + "is not built with zstd support"), + abfd, name); + return false; + } +#endif } if (abfd->is_linker_input) diff --git a/bfd/elfxx-target.h b/bfd/elfxx-target.h index feaba84bf2e..ca600bb5ddf 100644 --- a/bfd/elfxx-target.h +++ b/bfd/elfxx-target.h @@ -989,7 +989,8 @@ const bfd_target TARGET_BIG_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY @@ -1093,7 +1094,8 @@ const bfd_target TARGET_LITTLE_SYM = /* object_flags: mask of all file flags */ (HAS_RELOC | EXEC_P | HAS_LINENO | HAS_DEBUG | HAS_SYMS | HAS_LOCALS | DYNAMIC | WP_TEXT | D_PAGED | BFD_COMPRESS | BFD_DECOMPRESS - | BFD_COMPRESS_GABI | BFD_CONVERT_ELF_COMMON | BFD_USE_ELF_STT_COMMON), + | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD | BFD_CONVERT_ELF_COMMON + | BFD_USE_ELF_STT_COMMON), /* section_flags: mask of all section flags */ (SEC_HAS_CONTENTS | SEC_ALLOC | SEC_LOAD | SEC_RELOC | SEC_READONLY diff --git a/bfd/section.c b/bfd/section.c index c7a02d729f2..614570e976e 100644 --- a/bfd/section.c +++ b/bfd/section.c @@ -392,7 +392,8 @@ CODE_FRAGMENT . unsigned int compress_status : 2; .#define COMPRESS_SECTION_NONE 0 .#define COMPRESS_SECTION_DONE 1 -.#define DECOMPRESS_SECTION_SIZED 2 +.#define DECOMPRESS_SECTION_ZLIB 2 +.#define DECOMPRESS_SECTION_ZSTD 3 . . {* The following flags are used by the ELF linker. *} . diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi index 1499db5728c..34a4164eb94 100644 --- a/binutils/doc/binutils.texi +++ b/binutils/doc/binutils.texi @@ -2159,21 +2159,23 @@ ELF ABI. Note - if compression would actually make a section @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd For ELF files, these options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--decompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug} instead of @samp{.debug}. Note - if compression would -actually make a section @emph{larger}, then it is not compressed nor -renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @item --decompress-debug-sections -Decompress DWARF debug sections using zlib. The original section -names of the compressed sections are restored. +Decompress DWARF debug sections. For a @samp{.zdebug} section, the original +name is restored. @item --elf-stt-common=yes @itemx --elf-stt-common=no diff --git a/binutils/objcopy.c b/binutils/objcopy.c index 75992e8a3f7..fcad52b746f 100644 --- a/binutils/objcopy.c +++ b/binutils/objcopy.c @@ -232,7 +232,8 @@ static enum compress_zlib = compress | 1 << 1, compress_gnu_zlib = compress | 1 << 2, compress_gabi_zlib = compress | 1 << 3, - decompress = 1 << 4 + compress_zstd = compress | 1 << 4, + decompress = 1 << 5 } do_debug_sections = nothing; /* Whether to generate ELF common symbols with the STT_COMMON type. */ @@ -678,8 +679,8 @@ copy_usage (FILE *stream, int exit_status) \n\ --subsystem [:]\n\ Set PE subsystem to [& ]\n\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ - Compress DWARF debug sections using zlib\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ + Compress DWARF debug sections\n\ --decompress-debug-sections Decompress DWARF debug sections using zlib\n\ --elf-stt-common=[yes|no] Generate ELF common symbols with STT_COMMON\n\ type\n\ @@ -3809,6 +3810,9 @@ copy_file (const char *input_filename, const char *output_filename, int ofd, if (do_debug_sections != compress_gnu_zlib) ibfd->flags |= BFD_COMPRESS_GABI; break; + case compress_zstd: + ibfd->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; + break; case decompress: ibfd->flags |= BFD_DECOMPRESS; break; @@ -5471,6 +5475,8 @@ copy_main (int argc, char *argv[]) do_debug_sections = compress_gnu_zlib; else if (strcasecmp (optarg, "zlib-gabi") == 0) do_debug_sections = compress_gabi_zlib; + else if (strcasecmp (optarg, "zstd") == 0) + do_debug_sections = compress_zstd; else fatal (_("unrecognized --compress-debug-sections type `%s'"), optarg); diff --git a/binutils/testsuite/binutils-all/compress.exp b/binutils/testsuite/binutils-all/compress.exp index c88da74a987..4e74c824664 100644 --- a/binutils/testsuite/binutils-all/compress.exp +++ b/binutils/testsuite/binutils-all/compress.exp @@ -576,6 +576,50 @@ if { [regexp_diff objdump.out $srcdir/$subdir/dw2-3gabi.W] } then { pass "$testname" } +if { [binutils_assemble_flags $srcdir/$subdir/dw2-1.S ${compressedfile}zstd.o --compress-debug-sections=zstd] } then { + set testname "objcopy compress debug sections with zstd" + set got [binutils_run $OBJCOPY "--compress-debug-sections=zstd ${testfile}.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${compressedfile}zstd.o ${copyfile}zstd.o\n" + verbose "cmp ${compressedfile}zstd.o ${copyfile}zstd.o" + set src1 ${compressedfile}zstd.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } + + set testname "objcopy decompress compressed debug sections with zstd" + set got [binutils_run $OBJCOPY "--decompress-debug-sections ${compressedfile}zstd.o ${copyfile}zstd.o"] + if ![string match "" $got] then { + fail "objcopy ($testname)" + return + } + send_log "cmp ${testfile}.o ${copyfile}zstd.o\n" + verbose "cmp ${testfile}.o ${copyfile}zstd.o" + set src1 ${testfile}.o + set src2 ${copyfile}zstd.o + set status [remote_exec build cmp "${src1} ${src2}"] + set exec_output [lindex $status 1] + set exec_output [prune_warnings $exec_output] + if ![string match "" $exec_output] then { + send_log "$exec_output\n" + verbose "$exec_output" 1 + fail "objcopy ($testname)" + } else { + pass "objcopy ($testname)" + } +} + proc convert_test { testname as_flags objcop_flags } { global srcdir global subdir diff --git a/config/zstd.m4 b/config/zstd.m4 new file mode 100644 index 00000000000..b8b3a199730 --- /dev/null +++ b/config/zstd.m4 @@ -0,0 +1,12 @@ +AC_DEFUN([AM_ZSTD], +[ + zstdlib= + AC_ARG_WITH(system-zstd, + [AS_HELP_STRING([--with-system-zstd], [use installed zstd])], + if test x$with_system_zstd = xyes ; then + zstdlib=-lzstd + AC_DEFINE(HAVE_ZSTD_H) + fi + ) + AC_SUBST(zstdlib) +]) diff --git a/configure b/configure index 16139e3bfa3..6e1d18e2d25 100755 --- a/configure +++ b/configure @@ -784,6 +784,7 @@ ac_user_opts=' enable_option_checking with_build_libsubdir with_system_zlib +with_system_zstd enable_as_accelerator_for enable_offload_targets enable_gold @@ -1566,6 +1567,7 @@ Optional Packages: --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) --with-build-libsubdir=DIR Directory where to find libraries for build system --with-system-zlib use installed libz + --with-system-zstd use installed zstd --with-mpc=PATH specify prefix directory for installed MPC package. Equivalent to --with-mpc-include=PATH/include plus --with-mpc-lib=PATH/lib @@ -2924,6 +2926,13 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi + +# Check whether --with-system-zstd was given. +if test "${with_system_zstd+set}" = set; then : + withval=$with_system_zstd; +fi + + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/configure.ac b/configure.ac index a5555a9c91b..f7316b24ac6 100644 --- a/configure.ac +++ b/configure.ac @@ -246,6 +246,9 @@ if test x$with_system_zlib = xyes ; then noconfigdirs="$noconfigdirs zlib" fi +AC_ARG_WITH(system-zstd, +[AS_HELP_STRING([--with-system-zstd], [use installed zstd])]) + # Don't compile the bundled readline/libreadline.a if --with-system-readline # is provided. if test x$with_system_readline = xyes ; then diff --git a/gas/Makefile.am b/gas/Makefile.am index bd597398671..b6700b68a7f 100644 --- a/gas/Makefile.am +++ b/gas/Makefile.am @@ -32,6 +32,8 @@ tooldir = $(exec_prefix)/$(target_alias) ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ + YACC = `if [ -f ../bison/bison ] ; then echo ../bison/bison -y -L../bison/bison ; else echo @YACC@ ; fi` LEX = `if [ -f ../flex/flex ] ; then echo ../flex/flex ; else echo @LEX@ ; fi` @@ -407,7 +409,7 @@ STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) EXTRA_as_new_SOURCES = $(CFILES) $(HFILES) $(TARGET_CPU_CFILES) \ diff --git a/gas/Makefile.in b/gas/Makefile.in index c57d78f82c4..3a5359a50f1 100644 --- a/gas/Makefile.in +++ b/gas/Makefile.in @@ -143,7 +143,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/../bfd/version.m4 \ @@ -508,6 +509,7 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ zlibdir = @zlibdir@ zlibinc = @zlibinc@ +zstdlib = @zstdlib@ AUTOMAKE_OPTIONS = dejagnu foreign no-dist subdir-objects info-in-builddir no-texinfo.tex ACLOCAL_AMFLAGS = -I .. -I ../config -I ../bfd MAINTAINERCLEANFILES = doc/asconfig.texi doc/as.info @@ -519,6 +521,7 @@ tooldir = $(exec_prefix)/$(target_alias) # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ # Automake 1.10+ disables lex and yacc output file regeneration if # maintainer mode is disabled. Avoid this. @@ -874,7 +877,7 @@ GASLIBS = @OPCODES_LIB@ ../bfd/libbfd.la ../libiberty/libiberty.a STAGESTUFF = *.@OBJEXT@ $(noinst_PROGRAMS) as_new_SOURCES = $(GAS_CFILES) as_new_LDADD = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ - $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) + $(extra_objects) $(GASLIBS) $(LIBINTL) $(LIBM) $(ZLIB) $(ZSTD) as_new_DEPENDENCIES = $(TARG_CPU_O) $(OBJ_FORMAT_O) $(ATOF_TARG_O) \ $(extra_objects) $(GASLIBS) $(LIBINTL_DEP) diff --git a/gas/aclocal.m4 b/gas/aclocal.m4 index 70183124da7..d88bfb66998 100644 --- a/gas/aclocal.m4 +++ b/gas/aclocal.m4 @@ -1200,6 +1200,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/gas/as.c b/gas/as.c index 6268779cf90..5d71af3210e 100644 --- a/gas/as.c +++ b/gas/as.c @@ -252,14 +252,14 @@ Options:\n\ --alternate initially turn on alternate macro syntax\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib [default]\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ don't compress DWARF debug sections\n")); #else fprintf (stream, _("\ - --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\ + --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\ compress DWARF debug sections using zlib\n")); fprintf (stream, _("\ --nocompress-debug-sections\n\ @@ -736,6 +736,15 @@ This program has absolutely no warranty.\n")); flag_compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD_H + flag_compress_debug = COMPRESS_DEBUG_ZSTD; +#else + as_fatal (_ ("--compress-debug-sections=zstd: gas is not " + "built with zstd support")); +#endif + } else as_fatal (_("Invalid --compress-debug-sections option: `%s'"), optarg); diff --git a/gas/compress-debug.c b/gas/compress-debug.c index c80dbeec4e7..58199d520ac 100644 --- a/gas/compress-debug.c +++ b/gas/compress-debug.c @@ -21,14 +21,23 @@ #include "config.h" #include #include +#if HAVE_ZSTD_H +#include +#endif #include "ansidecl.h" #include "compress-debug.h" /* Initialize the compression engine. */ -struct z_stream_s * -compress_init (void) +void * +compress_init (bool use_zstd) { + if (use_zstd) { +#if HAVE_ZSTD_H + return ZSTD_createCCtx (); +#endif + } + static struct z_stream_s strm; strm.zalloc = NULL; @@ -42,22 +51,37 @@ compress_init (void) from the engine goes into the current frag on the obstack. */ int -compress_data (struct z_stream_s *strm, const char **next_in, - int *avail_in, char **next_out, int *avail_out) +compress_data (bool use_zstd, void *ctx, const char **next_in, int *avail_in, + char **next_out, int *avail_out) { - int out_size = 0; - int x; + if (use_zstd) + { +#if HAVE_ZSTD_H + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { *next_in, *avail_in, 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_continue); + *next_in += ib.pos; + *avail_in -= ib.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + return (int)ob.pos; +#endif + } + + struct z_stream_s *strm = ctx; strm->next_in = (Bytef *) (*next_in); strm->avail_in = *avail_in; strm->next_out = (Bytef *) (*next_out); strm->avail_out = *avail_out; - x = deflate (strm, Z_NO_FLUSH); + int x = deflate (strm, Z_NO_FLUSH); if (x != Z_OK) return -1; - out_size = *avail_out - strm->avail_out; + int out_size = *avail_out - strm->avail_out; *next_in = (char *) (strm->next_in); *avail_in = strm->avail_in; *next_out = (char *) (strm->next_out); @@ -71,10 +95,28 @@ compress_data (struct z_stream_s *strm, const char **next_in, needed. */ int -compress_finish (struct z_stream_s *strm, char **next_out, +compress_finish (bool use_zstd, void *ctx, char **next_out, int *avail_out, int *out_size) { + if (use_zstd) + { +#if HAVE_ZSTD_H + ZSTD_outBuffer ob = { *next_out, *avail_out, 0 }; + ZSTD_inBuffer ib = { 0 }; + size_t ret = ZSTD_compressStream2 (ctx, &ob, &ib, ZSTD_e_end); + *out_size = ob.pos; + *next_out += ob.pos; + *avail_out -= ob.pos; + if (ZSTD_isError (ret)) + return -1; + if (ret == 0) + ZSTD_freeCCtx (ctx); + return ret ? 1 : 0; +#endif + } + int x; + struct z_stream_s *strm = ctx; strm->avail_in = 0; strm->next_out = (Bytef *) (*next_out); diff --git a/gas/compress-debug.h b/gas/compress-debug.h index 0183e269fef..87de0f8444e 100644 --- a/gas/compress-debug.h +++ b/gas/compress-debug.h @@ -21,19 +21,19 @@ #ifndef COMPRESS_DEBUG_H #define COMPRESS_DEBUG_H +#include + struct z_stream_s; /* Initialize the compression engine. */ -extern struct z_stream_s * -compress_init (void); +extern void *compress_init (bool); /* Stream the contents of a frag to the compression engine. Output from the engine goes into the current frag on the obstack. */ -extern int -compress_data (struct z_stream_s *, const char **, int *, char **, int *); +extern int compress_data (bool, void *, const char **, int *, char **, int *); /* Finish the compression and consume the remaining compressed output. */ extern int -compress_finish (struct z_stream_s *, char **, int *, int *); +compress_finish (bool, void *, char **, int *, int *); #endif /* COMPRESS_DEBUG_H */ diff --git a/gas/config.in b/gas/config.in index e243fd277ee..9fd6231a125 100644 --- a/gas/config.in +++ b/gas/config.in @@ -134,6 +134,9 @@ /* Define to 1 if you have the header file. */ #undef HAVE_WINDOWS_H +/* Define to 1 if configured with zstd support. */ +#undef HAVE_ZSTD_H + /* Using i386 COFF? */ #undef I386COFF diff --git a/gas/configure b/gas/configure index d0449a1d7ab..2ebf83bcc1e 100755 --- a/gas/configure +++ b/gas/configure @@ -633,6 +633,7 @@ ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS LIBOBJS +zstdlib zlibinc zlibdir LIBM @@ -817,6 +818,7 @@ with_cpu enable_nls enable_maintainer_mode with_system_zlib +with_system_zstd ' ac_precious_vars='build_alias host_alias @@ -1493,6 +1495,7 @@ Optional Packages: --with-cpu=CPU default cpu variant is CPU (currently only supported on ARC) --with-system-zlib use installed libz + --with-system-zstd use installed zstd Some influential environment variables: CC C compiler command @@ -10702,7 +10705,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10705 "configure" +#line 10708 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -10808,7 +10811,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 10811 "configure" +#line 10814 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -13945,7 +13948,7 @@ $as_echo "#define USE_BINARY_FOPEN 1" >>confdefs.h ;; esac -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -13964,6 +13967,21 @@ fi + zstdlib= + +# Check whether --with-system-zstd was given. +if test "${with_system_zstd+set}" = set; then : + withval=$with_system_zstd; if test x$with_system_zstd = xyes ; then + zstdlib=-lzstd + $as_echo "#define HAVE_ZSTD_H 1" >>confdefs.h + + fi + +fi + + + + # Support for VMS timestamps via cross compile if test "$ac_cv_header_time_h" = yes; then diff --git a/gas/configure.ac b/gas/configure.ac index 21795a82bdd..8750ea21381 100644 --- a/gas/configure.ac +++ b/gas/configure.ac @@ -1004,8 +1004,9 @@ AC_CHECK_DECLS([asprintf, mempcpy, stpcpy]) BFD_BINARY_FOPEN -# Link in zlib if we can. This allows us to write compressed debug sections. +# Link in zlib/zstd if we can. This allows us to write compressed debug sections. AM_ZLIB +AM_ZSTD # Support for VMS timestamps via cross compile diff --git a/gas/doc/as.texi b/gas/doc/as.texi index e915893da86..01f49434021 100644 --- a/gas/doc/as.texi +++ b/gas/doc/as.texi @@ -711,16 +711,19 @@ given section @emph{larger} then it is not compressed. @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd These options control how DWARF debug sections are compressed. @option{--compress-debug-sections=none} is equivalent to @option{--nocompress-debug-sections}. @option{--compress-debug-sections=zlib} and @option{--compress-debug-sections=zlib-gabi} are equivalent to @option{--compress-debug-sections}. -@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug -sections using zlib. The debug sections are renamed to begin with -@samp{.zdebug}. Note if compression would make a given section -@emph{larger} then it is not compressed nor renamed. +@option{--compress-debug-sections=zlib-gnu} compresses DWARF debug sections +using the obsoleted zlib-gnu format. The debug sections are renamed to begin +with @samp{.zdebug}. +@option{--compress-debug-sections=zstd} compresses DWARF debug +sections using zstd. Note - if compression would actually make a section +@emph{larger}, then it is not compressed nor renamed. @end ifset diff --git a/gas/write.c b/gas/write.c index f76bbdb706e..0e49df7c03f 100644 --- a/gas/write.c +++ b/gas/write.c @@ -1413,7 +1413,7 @@ write_relocs (bfd *abfd ATTRIBUTE_UNUSED, asection *sec, } static int -compress_frag (struct z_stream_s *strm, const char *contents, int in_size, +compress_frag (bool use_zstd, void *ctx, const char *contents, int in_size, fragS **last_newf, struct obstack *ob) { int out_size; @@ -1442,10 +1442,10 @@ compress_frag (struct z_stream_s *strm, const char *contents, int in_size, as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - out_size = compress_data (strm, &contents, &in_size, - &next_out, &avail_out); + out_size = compress_data (use_zstd, ctx, &contents, &in_size, &next_out, + &avail_out); if (out_size < 0) - return -1; + return -1; f->fr_fix += out_size; total_out_size += out_size; @@ -1471,7 +1471,6 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) const char *section_name; char *compressed_name; char *header; - struct z_stream_s *strm; int x; flagword flags = bfd_section_flags (sec); unsigned int header_size, compression_header_size; @@ -1485,21 +1484,22 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) if (!startswith (section_name, ".debug_")) return; - strm = compress_init (); - if (strm == NULL) + bool use_zstd = abfd->flags & BFD_COMPRESS_ZSTD; + void *ctx = compress_init (use_zstd); + if (ctx == NULL) return; - if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) + if (flag_compress_debug == COMPRESS_DEBUG_GNU_ZLIB) + { + compression_header_size = 0; + header_size = 12; + } + else { compression_header_size = bfd_get_compression_header_size (stdoutput, NULL); header_size = compression_header_size; } - else - { - compression_header_size = 0; - header_size = 12; - } /* Create a new frag to contain the compression header. */ first_newf = frag_alloc (ob); @@ -1531,7 +1531,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) gas_assert (f->fr_type == rs_fill); if (f->fr_fix) { - out_size = compress_frag (strm, f->fr_literal, f->fr_fix, + out_size = compress_frag (use_zstd, ctx, f->fr_literal, f->fr_fix, &last_newf, ob); if (out_size < 0) return; @@ -1545,8 +1545,8 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) { while (count--) { - out_size = compress_frag (strm, fill_literal, (int) fill_size, - &last_newf, ob); + out_size = compress_frag (use_zstd, ctx, fill_literal, + (int)fill_size, &last_newf, ob); if (out_size < 0) return; compressed_size += out_size; @@ -1579,7 +1579,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED) as_fatal (_("can't extend frag")); next_out = obstack_next_free (ob); obstack_blank_fast (ob, avail_out); - x = compress_finish (strm, &next_out, &avail_out, &out_size); + x = compress_finish (use_zstd, ctx, &next_out, &avail_out, &out_size); if (x < 0) return; @@ -2540,6 +2540,8 @@ write_object_file (void) { if (flag_compress_debug == COMPRESS_DEBUG_GABI_ZLIB) stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI; + else if (flag_compress_debug == COMPRESS_DEBUG_ZSTD) + stdoutput->flags |= BFD_COMPRESS | BFD_COMPRESS_GABI | BFD_COMPRESS_ZSTD; else stdoutput->flags |= BFD_COMPRESS; bfd_map_over_sections (stdoutput, compress_debug, (char *) 0); diff --git a/gdb/Makefile.in b/gdb/Makefile.in index 2598b81d205..74194a727ca 100644 --- a/gdb/Makefile.in +++ b/gdb/Makefile.in @@ -173,6 +173,8 @@ BFD_CFLAGS = -I$(BFD_DIR) -I$(BFD_SRC) ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ + # Where is the decnumber library? Typically in ../libdecnumber. LIBDECNUMBER_DIR = ../libdecnumber LIBDECNUMBER = $(LIBDECNUMBER_DIR)/libdecnumber.a @@ -647,7 +649,7 @@ INTERNAL_LDFLAGS = \ # Libraries and corresponding dependencies for compiling gdb. # XM_CLIBS, defined in *config files, have host-dependent libs. # LIBIBERTY appears twice on purpose. -CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \ +CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) $(ZSTD) \ $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \ $(XM_CLIBS) $(GDBTKLIBS) $(LIBBACKTRACE_LIB) \ @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \ @@ -2298,6 +2300,7 @@ aclocal_m4_deps = \ ../config/lcmessage.m4 \ ../config/codeset.m4 \ ../config/zlib.m4 \ + ../config/zstd.m4 \ ../config/ax_pthread.m4 $(srcdir)/aclocal.m4: @MAINTAINER_MODE_TRUE@ $(aclocal_m4_deps) diff --git a/gdb/acinclude.m4 b/gdb/acinclude.m4 index 95ff2b6f35e..242512e81b1 100644 --- a/gdb/acinclude.m4 +++ b/gdb/acinclude.m4 @@ -43,6 +43,7 @@ m4_include([../config/lib-link.m4]) m4_include([../config/iconv.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../gdbsupport/common.m4]) @@ -233,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [ # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $zstdlib $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" AC_CACHE_CHECK( diff --git a/gdb/configure b/gdb/configure index 4dbd0c3b13c..7b80e1b8411 100755 --- a/gdb/configure +++ b/gdb/configure @@ -747,6 +747,7 @@ READLINE_DEPS READLINE LTLIBICONV LIBICONV +zstdlib zlibinc zlibdir MIG @@ -893,6 +894,7 @@ enable_codesign with_pkgversion with_bugurl with_system_zlib +with_system_zstd with_gnu_ld enable_rpath with_libiconv_prefix @@ -1641,6 +1643,7 @@ Optional Packages: --with-pkgversion=PKG Use PKG in the version string in place of "GDB" --with-bugurl=URL Direct users to URL to report a bug --with-system-zlib use installed libz + --with-system-zstd use installed zstd --with-gnu-ld assume the C compiler uses GNU ld default=no --with-libiconv-prefix[=DIR] search for libiconv in DIR/include and DIR/lib --without-libiconv-prefix don't search for libiconv in includedir and libdir @@ -8242,7 +8245,8 @@ if test "$ac_res" != no; then : fi -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -8261,6 +8265,21 @@ fi + zstdlib= + +# Check whether --with-system-zstd was given. +if test "${with_system_zstd+set}" = set; then : + withval=$with_system_zstd; if test x$with_system_zstd = xyes ; then + zstdlib=-lzstd + $as_echo "#define HAVE_ZSTD_H 1" >>confdefs.h + + fi + +fi + + + + if test "X$prefix" = "XNONE"; then acl_final_prefix="$ac_default_prefix" @@ -17281,7 +17300,7 @@ WIN32LIBS="$WIN32LIBS $WIN32APILIBS" # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $zstdlib $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ELF support in BFD" >&5 @@ -17396,7 +17415,7 @@ fi # always want our bfd. CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS" ZLIBDIR=`echo $zlibdir | sed 's,\$(top_builddir)/,,g'` - LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $LDFLAGS" + LDFLAGS="-L../bfd -L../libiberty $ZLIBDIR $zstdlib $LDFLAGS" intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'` LIBS="-lbfd -liberty -lz $intl $LIBS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Mach-O support in BFD" >&5 diff --git a/gdb/configure.ac b/gdb/configure.ac index 215a91c5b27..e8cf808d280 100644 --- a/gdb/configure.ac +++ b/gdb/configure.ac @@ -460,8 +460,10 @@ AC_SEARCH_LIBS(gethostbyname, nsl) # Some systems (e.g. Solaris) have `socketpair' in libsocket. AC_SEARCH_LIBS(socketpair, socket) -# Link in zlib if we can. This allows us to read compressed debug sections. +# Link in zlib/zstd if we can. This allows us to read compressed debug +# sections. AM_ZLIB +AM_ZSTD AM_ICONV diff --git a/ld/Makefile.am b/ld/Makefile.am index d31021c13e2..f037c487214 100644 --- a/ld/Makefile.am +++ b/ld/Makefile.am @@ -40,6 +40,8 @@ am__skipyacc = ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ + ELF_CLFAGS=-DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ @@ -959,7 +961,7 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ldbuildid.c ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD) $(JANSSON_LIBS) # Dependency tracking for the generated emulation files. EXTRA_ld_new_SOURCES += $(ALL_EMULATION_SOURCES) $(ALL_64_EMULATION_SOURCES) @@ -983,7 +985,7 @@ check-DEJAGNU: site.exp CFLAGS_FOR_TARGET="$(CFLAGS_FOR_TARGET)" \ CXX_FOR_TARGET="$(CXX_FOR_TARGET)" \ CXXFLAGS_FOR_TARGET="$(CXXFLAGS_FOR_TARGET)" \ - OFILES="$(OFILES)" BFDLIB="$(TESTBFDLIB)" CTFLIB="$(TESTCTFLIB) $(ZLIB)" \ + OFILES="$(OFILES)" BFDLIB="$(TESTBFDLIB)" CTFLIB="$(TESTCTFLIB) $(ZLIB) $(ZSTD)" \ LIBIBERTY="$(LIBIBERTY) $(LIBINTL)" LIBS="$(LIBS)" \ DO_COMPARE="`echo '$(do_compare)' | sed -e 's,\\$$,,g'`" \ $(RUNTESTFLAGS); \ diff --git a/ld/Makefile.in b/ld/Makefile.in index ee0c98f65b0..dfe8e6d9e33 100644 --- a/ld/Makefile.in +++ b/ld/Makefile.in @@ -126,7 +126,8 @@ am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \ $(top_srcdir)/../config/plugins.m4 \ $(top_srcdir)/../config/po.m4 \ $(top_srcdir)/../config/progtest.m4 \ - $(top_srcdir)/../config/zlib.m4 $(top_srcdir)/../libtool.m4 \ + $(top_srcdir)/../config/zlib.m4 \ + $(top_srcdir)/../config/zstd.m4 $(top_srcdir)/../libtool.m4 \ $(top_srcdir)/../ltoptions.m4 $(top_srcdir)/../ltsugar.m4 \ $(top_srcdir)/../ltversion.m4 $(top_srcdir)/../lt~obsolete.m4 \ $(top_srcdir)/../bfd/version.m4 $(top_srcdir)/configure.ac @@ -544,6 +545,7 @@ top_srcdir = @top_srcdir@ use_sysroot = @use_sysroot@ zlibdir = @zlibdir@ zlibinc = @zlibinc@ +zstdlib = @zstdlib@ AUTOMAKE_OPTIONS = dejagnu no-texinfo.tex no-dist foreign info-in-builddir ACLOCAL_AMFLAGS = -I .. -I ../config -I ../bfd TEXINFO_TEX = ../texinfo/texinfo.tex @@ -560,6 +562,7 @@ am__skipyacc = # case both are empty. ZLIB = @zlibdir@ -lz ZLIBINC = @zlibinc@ +ZSTD = @zstdlib@ ELF_CLFAGS = -DELF_LIST_OPTIONS=@elf_list_options@ \ -DELF_SHLIB_LIST_OPTIONS=@elf_shlib_list_options@ \ -DELF_PLT_UNWIND_LIST_OPTIONS=@elf_plt_unwind_list_options@ @@ -1011,7 +1014,7 @@ ld_new_SOURCES = ldgram.y ldlex-wrapper.c lexsup.c ldlang.c mri.c ldctor.c ldmai ld_new_DEPENDENCIES = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) \ $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL_DEP) $(JANSSON_LIBS) -ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(JANSSON_LIBS) +ld_new_LDADD = $(EMULATION_OFILES) $(EMUL_EXTRA_OFILES) $(BFDLIB) $(LIBCTF) $(LIBIBERTY) $(LIBINTL) $(ZLIB) $(ZSTD) $(JANSSON_LIBS) # # # Build a dummy plugin using libtool. @@ -2630,7 +2633,7 @@ check-DEJAGNU: site.exp CFLAGS_FOR_TARGET="$(CFLAGS_FOR_TARGET)" \ CXX_FOR_TARGET="$(CXX_FOR_TARGET)" \ CXXFLAGS_FOR_TARGET="$(CXXFLAGS_FOR_TARGET)" \ - OFILES="$(OFILES)" BFDLIB="$(TESTBFDLIB)" CTFLIB="$(TESTCTFLIB) $(ZLIB)" \ + OFILES="$(OFILES)" BFDLIB="$(TESTBFDLIB)" CTFLIB="$(TESTCTFLIB) $(ZLIB) $(ZSTD)" \ LIBIBERTY="$(LIBIBERTY) $(LIBINTL)" LIBS="$(LIBS)" \ DO_COMPARE="`echo '$(do_compare)' | sed -e 's,\\$$,,g'`" \ $(RUNTESTFLAGS); \ diff --git a/ld/aclocal.m4 b/ld/aclocal.m4 index d20c542eed5..893e973e4f2 100644 --- a/ld/aclocal.m4 +++ b/ld/aclocal.m4 @@ -1203,6 +1203,7 @@ m4_include([../config/plugins.m4]) m4_include([../config/po.m4]) m4_include([../config/progtest.m4]) m4_include([../config/zlib.m4]) +m4_include([../config/zstd.m4]) m4_include([../libtool.m4]) m4_include([../ltoptions.m4]) m4_include([../ltsugar.m4]) diff --git a/ld/configure b/ld/configure index a1a07005400..fe0853b4bb7 100755 --- a/ld/configure +++ b/ld/configure @@ -646,6 +646,7 @@ elf_plt_unwind_list_options elf_shlib_list_options elf_list_options STRINGIFY +zstdlib zlibinc zlibdir NATIVE_LIB_DIRS @@ -855,6 +856,7 @@ enable_werror enable_build_warnings enable_nls with_system_zlib +with_system_zstd ' ac_precious_vars='build_alias host_alias @@ -1552,6 +1554,7 @@ Optional Packages: --with-lib-path=dir1:dir2... set default LIB_PATH --with-sysroot=DIR Search for usr/lib et al within DIR. --with-system-zlib use installed libz + --with-system-zstd use installed zstd Some influential environment variables: CC C compiler command @@ -11491,7 +11494,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11494 "configure" +#line 11497 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -11597,7 +11600,7 @@ else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF -#line 11600 "configure" +#line 11603 "configure" #include "confdefs.h" #if HAVE_DLFCN_H @@ -17052,8 +17055,8 @@ $as_echo "#define HAVE_DECL_GETOPT 1" >>confdefs.h fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. # Use the system's zlib library. zlibdir="-L\$(top_builddir)/../zlib" @@ -17072,6 +17075,21 @@ fi + zstdlib= + +# Check whether --with-system-zstd was given. +if test "${with_system_zstd+set}" = set; then : + withval=$with_system_zstd; if test x$with_system_zstd = xyes ; then + zstdlib=-lzstd + $as_echo "#define HAVE_ZSTD_H 1" >>confdefs.h + + fi + +fi + + + + # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string # concatenation, or ostring.sed otherwise. This is to support the diff --git a/ld/configure.ac b/ld/configure.ac index eb55904c090..522d5dc0fb7 100644 --- a/ld/configure.ac +++ b/ld/configure.ac @@ -386,9 +386,10 @@ if test $ld_cv_decl_getopt_unistd_h = yes; then [Is the prototype for getopt in in the expected format?]) fi -# Link in zlib if we can. This allows us to read and write -# compressed CTF sections. +# Link in zlib/zstd if we can. This allows us to read and write +# compressed debug sections. AM_ZLIB +AM_ZSTD # When converting linker scripts into strings for use in emulation # files, use astring.sed if the compiler supports ANSI string diff --git a/ld/emultempl/elf.em b/ld/emultempl/elf.em index c52484f35d3..203191f182e 100644 --- a/ld/emultempl/elf.em +++ b/ld/emultempl/elf.em @@ -668,6 +668,15 @@ gld${EMULATION_NAME}_handle_option (int optc) link_info.compress_debug = COMPRESS_DEBUG_GNU_ZLIB; else if (strcasecmp (optarg, "zlib-gabi") == 0) link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB; + else if (strcasecmp (optarg, "zstd") == 0) + { +#ifdef HAVE_ZSTD_H + link_info.compress_debug = COMPRESS_DEBUG_ZSTD; +#else + einfo (_ ("%F%P: --compress-debug-sections=zstd: ld is not built " + "with zstd support\n")); +#endif + } else einfo (_("%F%P: invalid --compress-debug-sections option: \`%s'\n"), optarg); diff --git a/ld/ld.texi b/ld/ld.texi index eabbec8faa9..9daed2e7e9f 100644 --- a/ld/ld.texi +++ b/ld/ld.texi @@ -2863,10 +2863,12 @@ but for most Linux based systems it will be @code{both}. @kindex --compress-debug-sections=zlib @kindex --compress-debug-sections=zlib-gnu @kindex --compress-debug-sections=zlib-gabi +@kindex --compress-debug-sections=zstd @item --compress-debug-sections=none @itemx --compress-debug-sections=zlib @itemx --compress-debug-sections=zlib-gnu @itemx --compress-debug-sections=zlib-gabi +@itemx --compress-debug-sections=zstd On ELF platforms, these options control how DWARF debug sections are compressed using zlib. @@ -2880,6 +2882,9 @@ sets the SHF_COMPRESSED flag in the sections' headers. The @option{--compress-debug-sections=zlib} option is an alias for @option{--compress-debug-sections=zlib-gabi}. +@option{--compress-debug-sections=zstd} compresses DWARF debug sections using +zstd. + Note that this option overrides any compression in input debug sections, so if a binary is linked with @option{--compress-debug-sections=none} for example, then any compressed debug sections in input files will be diff --git a/ld/ldmain.c b/ld/ldmain.c index 1bbddaaad32..71af5935b74 100644 --- a/ld/ldmain.c +++ b/ld/ldmain.c @@ -506,8 +506,12 @@ main (int argc, char **argv) if ((link_info.compress_debug & COMPRESS_DEBUG)) { link_info.output_bfd->flags |= BFD_COMPRESS; - if (link_info.compress_debug == COMPRESS_DEBUG_GABI_ZLIB) - link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug != COMPRESS_DEBUG_GNU_ZLIB) + { + link_info.output_bfd->flags |= BFD_COMPRESS_GABI; + if (link_info.compress_debug == COMPRESS_DEBUG_ZSTD) + link_info.output_bfd->flags |= BFD_COMPRESS_ZSTD; + } } ldwrite (); diff --git a/ld/lexsup.c b/ld/lexsup.c index 9225f71b3ce..299371fb775 100644 --- a/ld/lexsup.c +++ b/ld/lexsup.c @@ -2146,8 +2146,8 @@ elf_static_list_options (FILE *file) fprintf (file, _("\ --package-metadata[=JSON] Generate package metadata note\n")); fprintf (file, _("\ - --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\ - Compress DWARF debug sections using zlib\n")); + --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi|zstd]\n\ + Compress DWARF debug sections\n")); #ifdef DEFAULT_FLAG_COMPRESS_DEBUG fprintf (file, _("\ Default: zlib-gabi\n")); -- 2.37.3.968.ga6b4b080e4-goog