From: Tom de Vries <tdevries@suse.de>
To: Jan Beulich <jbeulich@suse.com>
Cc: binutils@sourceware.org, Michael Matz <matz@suse.de>
Subject: Re: [PATCH] gas: Add --force-compress-debug-sections
Date: Fri, 24 Feb 2023 13:21:06 +0100 [thread overview]
Message-ID: <7cb226d0-1a91-9bad-181c-46f79c4d6eaf@suse.de> (raw)
In-Reply-To: <02dcf47c-4256-c5e5-de9e-814b60da8ce8@suse.com>
[-- Attachment #1: Type: text/plain, Size: 4531 bytes --]
On 2/24/23 12:28, Jan Beulich wrote:
> On 24.02.2023 11:52, Tom de Vries wrote:
>> On 2/23/23 14:44, Jan Beulich wrote:
>>> On 23.02.2023 14:27, Tom de Vries wrote:
>>>> On 2/23/23 14:08, Jan Beulich wrote:
>>>>> On 23.02.2023 13:45, Tom de Vries via Binutils wrote:
>>>>>> Gas has an option --compress-debug-sections that allows it to generate
>>>>>> compressed debug sections.
>>>>>>
>>>>>> That does not guarantee that the debug sections are in fact compressed:
>>>>>> ...
>>>>>> $ gcc ~/hello.c -Wa,-gdwarf-5 -c -Wa,--compress-debug-sections=zstd
>>>>>> $ readelf -S -W hello.o | grep " .debug"
>>>>>> [ 9] .debug_line PROGBITS 0000a8 000053 00 0 0 1
>>>>>> [11] .debug_line_str PROGBITS 0000fb 000025 01 MS 0 0 1
>>>>>> [12] .debug_info PROGBITS 000120 000039 00 0 0 1
>>>>>> [14] .debug_abbrev PROGBITS 000159 000028 00 0 0 1
>>>>>> [15] .debug_aranges PROGBITS 000190 000030 00 0 0 16
>>>>>> [17] .debug_str PROGBITS 0001c0 000039 01 MS 0 0 1
>>>>>> ...
>>>>>>
>>>>>> Sensibly so, they're only compressed if that provides a size benefit.
>>>>>>
>>>>>> However, for the purposes of testing components consuming dwarf
>>>>>> we may want the sections to be compressed regardless.
>>>>>>
>>>>>> Add a new option --force-compress-debug-sections that ignores the size
>>>>>> heuristic, such that we have instead:
>>>>>> ...
>>>>>> $ gcc ~/hello.c -Wa,-gdwarf-5 -c -Wa,--compress-debug-sections=zstd \
>>>>>> -Wa,--force-compress-debug-sections
>>>>>> $ readelf -S -W hello.o | grep " .debug"
>>>>>> [ 9] .debug_line PROGBITS 0000a8 000064 00 C 0 0 8
>>>>>> [11] .debug_line_str PROGBITS 000110 000046 01 MSC 0 0 8
>>>>>> [12] .debug_info PROGBITS 000158 000046 00 C 0 0 8
>>>>>> [14] .debug_abbrev PROGBITS 0001a0 000049 00 C 0 0 8
>>>>>> [15] .debug_aranges PROGBITS 0001f0 000034 00 C 0 0 8
>>>>>> [17] .debug_str PROGBITS 000228 00005a 01 MSC 0 0 8
>>>>>> ...
>>>>>>
>>>>>> Advertised as:
>>>>>> ...
>>>>>> $ as --help 2>&1 | grep compress
>>>>>> --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]
>>>>>> compress DWARF debug sections
>>>>>> --nocompress-debug-sections
>>>>>> don't compress DWARF debug sections
>>>>>> --force-compress-debug-sections
>>>>>> force compression of DWARF debug sections
>>>>>
>>>>> No objection in principle, but have you considered making this a new
>>>>> sub-option to --compress-debug-sections, i.e. compress-debug-sections=force?
>>>>
>>>> I did consider adding a "force-" prefix variant for all the non-none
>>>> sub-options, but decided to go with the simplest solution first.
>>>>
>>>> Your suggestion, --compress-debug-sections=force is more orthogonal,
>>>> though it breaks the pattern that all the sub-options are mutually
>>>> exclusive.
>>>>
>>>> We could have it be standalone, so you'd do:
>>>> --compress-debug-sections=zstd --compress-debug-sections=force.
>>>>
>>>> Or instead combined: --compress-debug-sections=force,zstd. Harder to
>>>> parse though, I suppose.
>>>
>>> I think both should be allowed. In a complex build system it may be
>>> different entities setting "how" and "whether". (To me "none" falls in
>>> the "whether" category together with "force", and it also can be seen
>>> as falling in the "how" category together with "zlib" etc. In Linux
>>> Kconfig, for example, I'd see this being expressed as first a "whether"
>>> choice [yes/maybe/forced] and then a "how" choice dependent upon
>>> "whether != none".)
>>>
>>
>> I gave this approach a try.
>
> Any specific reason you chose + as the separator instead of the more
> conventional , ?
Yes, I initially went for ',', but ran into:
...
$ gcc ~/hello.c -Wa,-gdwarf-5 \
-Wa,--compress-debug-sections=zstd,force -c -v
...
as -v --64 -gdwarf-5 --compress-debug-sections=zstd force -o hello.o \
/tmp/ccOUMqHL.s
...
Assembler messages:
Error: can't open force for reading: No such file or directory
...
> I also wouldn't see anything wrong with something
> like "...=force,zstd,none" - the last one(s) win. That's no different
> from specifying a second instance of the option. And without that it
> looks as if the parsing would end up simpler.
OK, gave that a try.
Thanks,
- Tom
[-- Attachment #2: 0001-gas-Add-compress-debug-sections-force.patch --]
[-- Type: text/x-patch, Size: 10115 bytes --]
From f23c41372cb7d48116ce51f99b9f265248b2d7d4 Mon Sep 17 00:00:00 2001
From: Tom de Vries <tdevries@suse.de>
Date: Thu, 23 Feb 2023 12:53:40 +0100
Subject: [PATCH] gas: Add --compress-debug-sections=force
Gas has an option --compress-debug-sections that allows it to generate
compressed debug sections.
That does not guarantee that the debug sections are in fact compressed:
...
$ gcc hello.c -Wa,-gdwarf-5 -c -Wa,--compress-debug-sections=zstd
$ readelf -S -W hello.o | grep " .debug"
[ 9] .debug_line PROGBITS 0000a8 000053 00 0 0 1
[11] .debug_line_str PROGBITS 0000fb 000025 01 MS 0 0 1
[12] .debug_info PROGBITS 000120 000039 00 0 0 1
[14] .debug_abbrev PROGBITS 000159 000028 00 0 0 1
[15] .debug_aranges PROGBITS 000190 000030 00 0 0 16
[17] .debug_str PROGBITS 0001c0 000039 01 MS 0 0 1
...
Sensibly so, they're only compressed if that provides a size benefit.
However, for the purpose of testing components consuming dwarf
we may want the sections to be compressed regardless.
Add a new suboption --compress-debug-sections=force that ignores the size
heuristic, such that we have instead:
...
$ gcc hello.c -Wa,-gdwarf-5 -c -Wa,--compress-debug-sections=zstd+force
$ readelf -S -W hello.o | grep " .debug"
[ 9] .debug_line PROGBITS 0000a8 000064 00 C 0 0 8
[11] .debug_line_str PROGBITS 000110 000046 01 MSC 0 0 8
[12] .debug_info PROGBITS 000158 000046 00 C 0 0 8
[14] .debug_abbrev PROGBITS 0001a0 000049 00 C 0 0 8
[15] .debug_aranges PROGBITS 0001f0 000034 00 C 0 0 8
[17] .debug_str PROGBITS 000228 00005a 01 MSC 0 0 8
...
Advertised as:
...
$ as --help
...
--compress-debug-sections[={none|<format>|force|force+<format>}]
where <format> is {zlib|zlib-gnu|zlib-gabi|zstd}
compress DWARF debug sections
Default: zstd
...
Tested on x86_64-linux.
---
gas/as.c | 133 +++++++++++++++++++++++++++++++++++++++++-------
gas/as.h | 4 ++
gas/doc/as.texi | 10 +++-
gas/write.c | 4 +-
4 files changed, 128 insertions(+), 23 deletions(-)
diff --git a/gas/as.c b/gas/as.c
index 598bfd56cf5..0f4d43f256d 100644
--- a/gas/as.c
+++ b/gas/as.c
@@ -230,6 +230,18 @@ enum compressed_debug_section_type flag_compress_debug
= DEFAULT_COMPRESSED_DEBUG_ALGORITHM;
#endif
+enum compress_debug_action
+{
+ cda_default,
+ cda_none,
+ cda_force,
+ cda_yes,
+};
+static enum compress_debug_action compress_debug_action
+ = cda_default;
+
+bool force_compress_debug = false;
+
static void
show_usage (FILE * stream)
{
@@ -252,7 +264,8 @@ Options:\n\
fprintf (stream, _("\
--alternate initially turn on alternate macro syntax\n"));
fprintf (stream, _("\
- --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi|zstd}]\n\
+ --compress-debug-sections[={none|<format>|force|force+<format>}]\n\
+ where <format> is {zlib|zlib-gnu|zlib-gabi|zstd}\n\
compress DWARF debug sections\n")),
fprintf (stream, _("\
Default: %s\n"),
@@ -418,6 +431,101 @@ Options:\n\
fprintf (stream, _("Report bugs to %s\n"), REPORT_BUGS_TO);
}
+static void
+parse_compress_debug_optarg_1 (const char *optarg, bool *none, bool *force,
+ enum compressed_debug_section_type *format)
+{
+ gas_assert (optarg != NULL);
+
+ if (strcmp (optarg, "force") == 0)
+ {
+ *force = true;
+ *none = false;
+ return;
+ }
+
+ enum compressed_debug_section_type tmp
+ = bfd_get_compression_algorithm (optarg);
+
+#ifndef HAVE_ZSTD
+ if (tmp == COMPRESS_DEBUG_ZSTD)
+ as_fatal (_ ("--compress-debug-sections=zstd: gas is not "
+ "built with zstd support"));
+#endif
+
+ if (tmp == COMPRESS_UNKNOWN)
+ as_fatal (_("Invalid --compress-debug-sections option: `%s'"),
+ optarg);
+
+ if (tmp == COMPRESS_DEBUG_NONE)
+ {
+ *none = true;
+ *force = false;
+ return;
+ }
+
+ *format = tmp;
+}
+
+static void
+parse_compress_debug_optarg (const char *optarg)
+{
+#if !defined OBJ_ELF && !defined OBJ_MAYBE_ELF
+ as_fatal (_("--compress-debug-sections=%s is unsupported"),
+ optarg);
+#endif
+
+ /* Tokenize subopts seperated by '+' and pass to
+ parse_compress_debug_optarg_1. */
+ bool none = false;
+ bool force = false;
+ enum compressed_debug_section_type format = COMPRESS_UNKNOWN;
+ while (true)
+ {
+ const char *idx = optarg;
+ while (*idx != '\0' && *idx != '+')
+ idx++;
+
+ size_t len = idx - optarg;
+ if (len == 0)
+ {
+ /* Generate error. */
+ parse_compress_debug_optarg_1 ("", NULL, NULL, NULL);
+ break;
+ }
+
+ char *tmp = xstrndup (optarg, len);
+ parse_compress_debug_optarg_1 (tmp, &none, &force, &format);
+ free (tmp);
+
+ if (*idx == '\0')
+ break;
+
+ /* Step over '+' and continue tokenizing. */
+ gas_assert (*idx == '+');
+ optarg = idx + 1;
+ }
+
+ if (none)
+ compress_debug_action = cda_none;
+ else if (force)
+ compress_debug_action = cda_force;
+ else
+ compress_debug_action = cda_yes;
+
+ if (format != COMPRESS_UNKNOWN)
+ flag_compress_debug = format;
+}
+
+static void
+finalize_parse_compress_debug_optarg (void)
+{
+ if (compress_debug_action == cda_none)
+ flag_compress_debug = COMPRESS_DEBUG_NONE;
+ else if (compress_debug_action == cda_force)
+ force_compress_debug = true;
+}
+
/* Since it is easy to do here we interpret the special arg "-"
to mean "use stdin" and we set that argv[] pointing to "".
After we have munged argv[], the only things left are source file
@@ -747,28 +855,13 @@ This program has absolutely no warranty.\n"));
case OPTION_COMPRESS_DEBUG:
if (optarg)
- {
-#if defined OBJ_ELF || defined OBJ_MAYBE_ELF
- flag_compress_debug = bfd_get_compression_algorithm (optarg);
-#ifndef HAVE_ZSTD
- if (flag_compress_debug == COMPRESS_DEBUG_ZSTD)
- as_fatal (_ ("--compress-debug-sections=zstd: gas is not "
- "built with zstd support"));
-#endif
- if (flag_compress_debug == COMPRESS_UNKNOWN)
- as_fatal (_("Invalid --compress-debug-sections option: `%s'"),
- optarg);
-#else
- as_fatal (_("--compress-debug-sections=%s is unsupported"),
- optarg);
-#endif
- }
+ parse_compress_debug_optarg (optarg);
else
- flag_compress_debug = COMPRESS_DEBUG_GABI_ZLIB;
+ parse_compress_debug_optarg ("zlib-gabi");
break;
case OPTION_NOCOMPRESS_DEBUG:
- flag_compress_debug = COMPRESS_DEBUG_NONE;
+ parse_compress_debug_optarg ("none");
break;
case OPTION_DEBUG_PREFIX_MAP:
@@ -1136,6 +1229,8 @@ This program has absolutely no warranty.\n"));
*pargc = new_argc;
*pargv = new_argv;
+ finalize_parse_compress_debug_optarg ();
+
#ifdef md_after_parse_args
md_after_parse_args ();
#endif
diff --git a/gas/as.h b/gas/as.h
index 4c5fa9ecf7d..115af019815 100644
--- a/gas/as.h
+++ b/gas/as.h
@@ -331,6 +331,10 @@ COMMON int flag_traditional_format;
/* Type of compressed debug sections we should generate. */
COMMON enum compressed_debug_section_type flag_compress_debug;
+/* True if we want to generate compressed debug sections, even if it
+ doesn't make them smaller. */
+COMMON bool force_compress_debug;
+
/* TRUE if .note.GNU-stack section with SEC_CODE should be created */
COMMON int flag_execstack;
diff --git a/gas/doc/as.texi b/gas/doc/as.texi
index bbdfa4bfdca..d25559141c7 100644
--- a/gas/doc/as.texi
+++ b/gas/doc/as.texi
@@ -718,7 +718,8 @@ Begin in alternate macro mode.
Compress DWARF debug sections using zlib with SHF_COMPRESSED from the
ELF ABI. The resulting object file may not be compatible with older
linkers and object file utilities. Note if compression would make a
-given section @emph{larger} then it is not compressed.
+given section @emph{larger} then it is not compressed, unless
+@option{--compress-debug-section=force} is used.
@ifset ELF
@cindex @samp{--compress-debug-sections=} option
@@ -727,6 +728,7 @@ given section @emph{larger} then it is not compressed.
@itemx --compress-debug-sections=zlib-gnu
@itemx --compress-debug-sections=zlib-gabi
@itemx --compress-debug-sections=zstd
+@itemx --compress-debug-sections=force
These options control how DWARF debug sections are compressed.
@option{--compress-debug-sections=none} is equivalent to
@option{--nocompress-debug-sections}.
@@ -738,7 +740,11 @@ using the obsoleted zlib-gnu format. The debug sections are renamed to begin
with @samp{.zdebug}.
@option{--compress-debug-sections=zstd} compresses DWARF debug
sections using zstd. Note - if compression would actually make a section
-@emph{larger}, then it is not compressed nor renamed.
+@emph{larger}, then it is not compressed nor renamed, unless
+@option{--compress-debug-section=force} is used.
+@option{--compress-debug-sections=force} compresses DWARF debug sections,
+even if this does not reduce size. It can be used in conjunction with a format
+selection, for instance @option{--compress-debug-section=zstd+force}.
@end ifset
diff --git a/gas/write.c b/gas/write.c
index 8273b7a42f1..39bcea23fac 100644
--- a/gas/write.c
+++ b/gas/write.c
@@ -1465,7 +1465,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED)
flagword flags = bfd_section_flags (sec);
if (seginfo == NULL
- || uncompressed_size < 32
+ || (!force_compress_debug && uncompressed_size < 32)
|| (flags & SEC_HAS_CONTENTS) == 0)
return;
@@ -1582,7 +1582,7 @@ compress_debug (bfd *abfd, asection *sec, void *xxx ATTRIBUTE_UNUSED)
/* PR binutils/18087: If compression didn't make the section smaller,
just keep it uncompressed. */
- if (compressed_size >= uncompressed_size)
+ if (!force_compress_debug && compressed_size >= uncompressed_size)
return;
/* Replace the uncompressed frag list with the compressed frag list. */
--
2.35.3
next prev parent reply other threads:[~2023-02-24 12:20 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-23 12:45 Tom de Vries
2023-02-23 13:08 ` Jan Beulich
2023-02-23 13:27 ` Tom de Vries
2023-02-23 13:44 ` Jan Beulich
2023-02-24 10:52 ` Tom de Vries
2023-02-24 11:28 ` Jan Beulich
2023-02-24 12:21 ` Tom de Vries [this message]
2023-02-24 13:23 ` Jan Beulich
2023-02-24 14:11 ` Tom de Vries
2023-02-24 14:26 ` Jan Beulich
2023-02-24 14:57 ` Tom de Vries
2023-02-27 9:03 ` Jan Beulich
2023-02-27 11:43 ` [PATCH] gas: Add --compress-debug-sections=force Tom de Vries
2023-02-27 11:51 ` Jan Beulich
2023-02-27 13:44 ` [PATCH] gas: Add --force-compress-debug-sections Pedro Alves
2023-02-27 14:07 ` Jan Beulich
2023-02-27 23:24 ` Tom de Vries
2023-02-28 0:19 ` Tom de Vries
2023-02-28 13:21 ` Pedro Alves
2023-02-28 12:49 ` Pedro Alves
2023-02-23 15:23 ` Michael Matz
2023-02-23 15:28 ` Tom de Vries
2023-02-23 15:44 ` Michael Matz
2023-02-23 15:46 ` Tom de Vries
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7cb226d0-1a91-9bad-181c-46f79c4d6eaf@suse.de \
--to=tdevries@suse.de \
--cc=binutils@sourceware.org \
--cc=jbeulich@suse.com \
--cc=matz@suse.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).