Commit: Add support for displaying unicode characters

Commit: Add support for displaying unicode characters

[Nick Clifton]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

Hi Guys,

  I am applying the attached patch to add the ability to display unicode
  characters to the nm, strings, objdump and readelf tools.  This is
  part of a response to the Trojan Source vulnerability that was
  recently disclosed.  I am also working on a separate patch for gas,
  but that is not quite ready yet.

Cheers
  Nick

binutils/ChangeLog
2021-11-09  Nick Clifton  <nickc@redhat.com>

	* nm.c: Add --unicode option to control how unicode characters are
	handled.
	* objdump.c: Likewise.
	* readelf.c: Likewise.
	* strings.c: Likewise.
	* binutils.texi: Document the new feature.
	* NEWS: Document the new feature.
	* testsuite/binutils-all/unicode.exp: New file.
	* testsuite/binutils-all/nm.hex.unicode
	* testsuite/binutils-all/strings.escape.unicode
	* testsuite/binutils-all/objdump.highlight.unicode
	* testsuite/binutils-all/readelf.invalid.unicode


[-- Attachment #2: b.p --]
[-- Type: application/octet-stream, Size: 61203 bytes --]

diff --git a/binutils/NEWS b/binutils/NEWS
index e6977b650db..f948d34f091 100644
--- a/binutils/NEWS
+++ b/binutils/NEWS
@@ -2,6 +2,15 @@
 
 * Add support for the LoongArch instruction set.
 
+* Tools which display symbols or strings (readelf, strings, nm, objdump)
+  have a new command line option which controls how unicode characters are
+  handled.  By default they are treated as normal for the tool.  Using
+  --unicode=locale will display them according to the current locale.
+  Using --unicode=hex will display them as hex byte values, whilst
+  --unicode=escape will display them as escape sequences.  In addition
+  using --unicode=highlight will display them as unicode escape sequences
+  highlighted in red (if supported by the output device).
+
 Changes in 2.37:
 
 * The readelf tool has a new command line option which can be used to specify
diff --git a/binutils/doc/binutils.texi b/binutils/doc/binutils.texi
index 504c3eae177..5de0631f3f4 100644
--- a/binutils/doc/binutils.texi
+++ b/binutils/doc/binutils.texi
@@ -812,6 +812,7 @@ nm [@option{-A}|@option{-o}|@option{--print-file-name}]
    [@option{-s}|@option{--print-armap}]
    [@option{-t} @var{radix}|@option{--radix=}@var{radix}]
    [@option{-u}|@option{--undefined-only}]
+   [@option{-U} @var{method}] [@option{--unicode=}@var{method}]
    [@option{-V}|@option{--version}]
    [@option{-X 32_64}]
    [@option{--defined-only}]
@@ -1132,6 +1133,21 @@ Use @var{radix} as the radix for printing the symbol values.  It must be
 @cindex undefined symbols
 Display only undefined symbols (those external to each object file).
 
+@item -U @var{[d|i|l|e|x|h]}
+@itemx --unicode=@var{[default|invalid|locale|escape|hex|highlight]}
+Controls the display of UTF-8 encoded mulibyte characters in strings.
+The default (@option{--unicode=default}) is to give them no special
+treatment.  The @option{--unicode=locale} option displays the sequence
+in the current locale, which may or may not support them.  The options
+@option{--unicode=hex} and @option{--unicode=invalid} display them as
+hex byte sequences enclosed by either angle brackets or curly braces.
+
+The @option{--unicode=escape} option displays them as escape sequences
+(@var{\uxxxx}) and the @option{--unicode=highlight} option displays
+them as escape sequences highlighted in red (if supported by the
+output device).  The colouring is intended to draw attention to the
+presence of unicode sequences where they might not be expected.
+
 @item -V
 @itemx --version
 Show the version number of @command{nm} and exit.
@@ -2247,6 +2263,7 @@ objdump [@option{-a}|@option{--archive-headers}]
         [@option{--prefix-strip=}@var{level}]
         [@option{--insn-width=}@var{width}]
         [@option{--visualize-jumps[=color|=extended-color|=off]}
+        [@option{-U} @var{method}] [@option{--unicode=}@var{method}]
         [@option{-V}|@option{--version}]
         [@option{-H}|@option{--help}]
         @var{objfile}@dots{}
@@ -2921,6 +2938,21 @@ When displaying symbols include those which the target considers to be
 special in some way and which would not normally be of interest to the
 user.
 
+@item -U @var{[d|i|l|e|x|h]}
+@itemx --unicode=@var{[default|invalid|locale|escape|hex|highlight]}
+Controls the display of UTF-8 encoded mulibyte characters in strings.
+The default (@option{--unicode=default}) is to give them no special
+treatment.  The @option{--unicode=locale} option displays the sequence
+in the current locale, which may or may not support them.  The options
+@option{--unicode=hex} and @option{--unicode=invalid} display them as
+hex byte sequences enclosed by either angle brackets or curly braces.
+
+The @option{--unicode=escape} option displays them as escape sequences
+(@var{\uxxxx}) and the @option{--unicode=highlight} option displays
+them as escape sequences highlighted in red (if supported by the
+output device).  The colouring is intended to draw attention to the
+presence of unicode sequences where they might not be expected.
+
 @item -V
 @itemx --version
 Print the version number of @command{objdump} and exit.
@@ -3197,6 +3229,7 @@ strings [@option{-afovV}] [@option{-}@var{min-len}]
         [@option{-n} @var{min-len}] [@option{--bytes=}@var{min-len}]
         [@option{-t} @var{radix}] [@option{--radix=}@var{radix}]
         [@option{-e} @var{encoding}] [@option{--encoding=}@var{encoding}]
+        [@option{-U} @var{method}] [@option{--unicode=}@var{method}]
         [@option{-}] [@option{--all}] [@option{--print-file-name}]
         [@option{-T} @var{bfdname}] [@option{--target=}@var{bfdname}]
         [@option{-w}] [@option{--include-all-whitespace}]
@@ -3288,6 +3321,28 @@ single-8-bit-byte characters, @samp{b} = 16-bit bigendian, @samp{l} =
 littleendian.  Useful for finding wide character strings. (@samp{l}
 and @samp{b} apply to, for example, Unicode UTF-16/UCS-2 encodings).
 
+@item -U @var{[d|i|l|e|x|h]}
+@itemx --unicode=@var{[default|invalid|locale|escape|hex|highlight]}
+Controls the display of UTF-8 encoded mulibyte characters in strings.
+The default (@option{--unicode=default}) is to give them no special
+treatment, and instead rely upon the setting of the
+@option{--encoding} option.  The other values for this option
+automatically enable @option{--encoding=S}.
+
+The @option{--unicode=invalid} option treats them as non-graphic
+characters and hence not part of a valid string.  All the remaining
+options treat them as valid string characters.
+
+The @option{--unicode=locale} option displays them in the current
+locale, which may or may not support UTF-8 encoding.  The
+@option{--unicode=hex} option displays them as hex byte sequences
+enclosed between @var{<>} characters.  The @option{--unicode=escape}
+option displays them as escape sequences (@var{\uxxxx}) and the
+@option{--unicode=highlight} option displays them as escape sequences
+highlighted in red (if supported by the output device).  The colouring
+is intended to draw attention to the presence of unicode sequences
+where they might not be expected.
+
 @item -T @var{bfdname}
 @itemx --target=@var{bfdname}
 @cindex object code format
@@ -4796,6 +4851,7 @@ readelf [@option{-a}|@option{--all}]
         [@option{--demangle@var{=style}}|@option{--no-demangle}]
         [@option{--quiet}]
         [@option{--recurse-limit}|@option{--no-recurse-limit}]
+        [@option{-U} @var{method}|@option{--unicode=}@var{method}]
         [@option{-n}|@option{--notes}]
         [@option{-r}|@option{--relocs}]
         [@option{-u}|@option{--unwind}]
@@ -4962,6 +5018,28 @@ necessary in order to demangle truly complicated names.  Note however
 that if the recursion limit is disabled then stack exhaustion is
 possible and any bug reports about such an event will be rejected.
 
+@item -U @var{[d|i|l|e|x|h]}
+@itemx --unicode=[default|invalid|locale|escape|hex|highlight]
+Controls the display of non-ASCII characters in identifier names.
+The default (@option{--unicode=locale} or @option{--unicode=default}) is
+to treat them as multibyte characters and display them in the current
+locale.  All other versions of this option treat the bytes as UTF-8
+encoded values and attempt to interpret them.  If they cannot be
+interpreted or if the @option{--unicode=invalid} option is used then
+they are displayed as a sequence of hex bytes, encloses in curly
+parethesis characters.
+
+Using the @option{--unicode=escape} option will display the characters
+as as unicode escape sequences (@var{\uxxxx}).  Using the
+@option{--unicode=hex} will display the characters as hex byte
+sequences enclosed between angle brackets.
+
+Using the @option{--unicode=highlight} will display the characters as 
+unicode escape sequences but it will also highlighted them in red,
+assuming that colouring is supported by the output device.  The
+colouring is intended to draw attention to the presence of unicode
+sequences when they might not be expected.
+
 @item -e
 @itemx --headers
 Display all the headers in the file.  Equivalent to @option{-h -l -S}.
diff --git a/binutils/nm.c b/binutils/nm.c
index 7606956c92a..e7ed6f829af 100644
--- a/binutils/nm.c
+++ b/binutils/nm.c
@@ -38,6 +38,11 @@
 #include "bucomm.h"
 #include "plugin-api.h"
 #include "plugin.h"
+#include "safe-ctype.h"
+
+#ifndef streq
+#define streq(a,b) (strcmp ((a),(b)) == 0)
+#endif
 
 /* When sorting by size, we use this structure to hold the size and a
    pointer to the minisymbol.  */
@@ -216,6 +221,18 @@ static const char *plugin_target = NULL;
 static bfd *lineno_cache_bfd;
 static bfd *lineno_cache_rel_bfd;
 
+typedef enum unicode_display_type
+{
+  unicode_default = 0,
+  unicode_locale,
+  unicode_escape,
+  unicode_hex,
+  unicode_highlight,
+  unicode_invalid
+} unicode_display_type;
+
+static unicode_display_type unicode_display = unicode_default;
+
 enum long_option_values
 {
   OPTION_TARGET = 200,
@@ -260,6 +277,7 @@ static struct option long_options[] =
   {"target", required_argument, 0, OPTION_TARGET},
   {"defined-only", no_argument, &defined_only, 1},
   {"undefined-only", no_argument, &undefined_only, 1},
+  {"unicode", required_argument, NULL, 'U'},
   {"version", no_argument, &show_version, 1},
   {"with-symbol-versions", no_argument, &with_symbol_versions, 1},
   {"without-symbol-versions", no_argument, &with_symbol_versions, 0},
@@ -313,6 +331,8 @@ usage (FILE *stream, int status)
   -t, --radix=RADIX      Use RADIX for printing symbol values\n\
       --target=BFDNAME   Specify the target object format as BFDNAME\n\
   -u, --undefined-only   Display only undefined symbols\n\
+  -U {d|s|i|x|e|h}       Specify how to treat UTF-8 encoded unicode characters\n\
+      --unicode={default|show|invalid|hex|escape|highlight}\n\
       --with-symbol-versions  Display version strings after symbol names\n\
   -X 32_64               (ignored)\n\
   @FILE                  Read options from FILE\n\
@@ -432,6 +452,187 @@ get_coff_symbol_type (const struct internal_syment *sym)
   return bufp;
 }
 \f
+/* Convert a potential UTF-8 encoded sequence in IN into characters in OUT.
+   The conversion format is controlled by the unicode_display variable.
+   Returns the number of characters added to OUT.
+   Returns the number of bytes consumed from IN in CONSUMED.
+   Always consumes at least one byte and displays at least one character.  */
+   
+static unsigned int
+display_utf8 (const unsigned char * in, char * out, unsigned int * consumed)
+{
+  char *        orig_out = out;
+  unsigned int  nchars = 0;
+  unsigned int j;
+
+  if (unicode_display == unicode_default)
+    goto invalid;
+
+  if (in[0] < 0xc0)
+    goto invalid;
+
+  if ((in[1] & 0xc0) != 0x80)
+    goto invalid;
+
+  if ((in[0] & 0x20) == 0)
+    {
+      nchars = 2;
+      goto valid;
+    }
+
+  if ((in[2] & 0xc0) != 0x80)
+    goto invalid;
+
+  if ((in[0] & 0x10) == 0)
+    {
+      nchars = 3;
+      goto valid;
+    }
+
+  if ((in[3] & 0xc0) != 0x80)
+    goto invalid;
+
+  nchars = 4;
+
+ valid:
+  switch (unicode_display)
+    {
+    case unicode_locale:
+      /* Copy the bytes into the output buffer as is.  */
+      memcpy (out, in, nchars);
+      out += nchars;
+      break;
+
+    case unicode_invalid:
+    case unicode_hex:
+      out += sprintf (out, "%c", unicode_display == unicode_hex ? '<' : '{');
+      out += sprintf (out, "0x");
+      for (j = 0; j < nchars; j++)
+	out += sprintf (out, "%02x", in [j]);
+      out += sprintf (out, "%c", unicode_display == unicode_hex ? '>' : '}');
+      break;
+      
+    case unicode_highlight:
+      if (isatty (1))
+	out += sprintf (out, "\x1B[31;47m"); /* Red.  */
+      /* Fall through.  */
+    case unicode_escape:
+      switch (nchars)
+	{
+	case 2:
+	  out += sprintf (out, "\\u%02x%02x",
+		  ((in[0] & 0x1c) >> 2), 
+		  ((in[0] & 0x03) << 6) | (in[1] & 0x3f));
+	  break;
+
+	case 3:
+	  out += sprintf (out, "\\u%02x%02x",
+		  ((in[0] & 0x0f) << 4) | ((in[1] & 0x3c) >> 2),
+		  ((in[1] & 0x03) << 6) | ((in[2] & 0x3f)));
+	  break;
+
+	case 4:
+	  out += sprintf (out, "\\u%02x%02x%02x",
+		  ((in[0] & 0x07) << 6) | ((in[1] & 0x3c) >> 2),
+		  ((in[1] & 0x03) << 6) | ((in[2] & 0x3c) >> 2),
+		  ((in[2] & 0x03) << 6) | ((in[3] & 0x3f)));
+	  break;
+	default:
+	  /* URG.  */
+	  break;
+	}
+
+      if (unicode_display == unicode_highlight && isatty (1))
+	out += sprintf (out, "\033[0m"); /* Default colour.  */
+      break;
+
+    default:
+      /* URG */
+      break;
+    }
+
+  * consumed = nchars;
+  return out - orig_out;
+
+ invalid:
+  /* Not a valid UTF-8 sequence.  */
+  *out = *in;
+  * consumed = 1;
+  return 1;
+}
+
+/* Convert any UTF-8 encoded characters in NAME into the form specified by
+   unicode_display.  Also converts control characters.  Returns a static
+   buffer if conversion was necessary.
+   Code stolen from objdump.c:sanitize_string().  */
+
+static const char *
+convert_utf8 (const char * in)
+{
+  static char *  buffer = NULL;
+  static size_t  buffer_len = 0;
+  const char *   original = in;
+  char *         out;
+
+  /* Paranoia.  */
+  if (in == NULL)
+    return "";
+
+  /* See if any conversion is necessary.
+     In the majority of cases it will not be needed.  */
+  do
+    {
+      unsigned char c = *in++;
+
+      if (c == 0)
+	return original;
+
+      if (ISCNTRL (c))
+	break;
+
+      if (unicode_display != unicode_default && c >= 0xc0)
+	break;
+    }
+  while (1);
+
+  /* Copy the input, translating as needed.  */
+  in = original;
+  if (buffer_len < (strlen (in) * 9))
+    {
+      free ((void *) buffer);
+      buffer_len = strlen (in) * 9;
+      buffer = xmalloc (buffer_len + 1);
+    }
+
+  out = buffer;
+  do
+    {
+      unsigned char c = *in++;
+
+      if (c == 0)
+	break;
+
+      if (ISCNTRL (c))
+	{
+	  *out++ = '^';
+	  *out++ = c + 0x40;
+	}
+      else if (unicode_display != unicode_default && c >= 0xc0)
+	{
+	  unsigned int num_consumed;
+
+	  out += display_utf8 ((const unsigned char *)(in - 1), out, & num_consumed);
+	  in += num_consumed - 1;
+	}
+      else
+	*out++ = c;
+    }
+  while (1);
+
+  *out = 0;
+  return buffer;
+}
+
 /* Print symbol name NAME, read from ABFD, with printf format FORM,
    demangling it if requested.  */
 
@@ -444,6 +645,7 @@ print_symname (const char *form, struct extended_symbol_info *info,
 
   if (name == NULL)
     name = info->sinfo->name;
+
   if (!with_symbol_versions
       && bfd_get_flavour (abfd) == bfd_target_elf_flavour)
     {
@@ -451,6 +653,7 @@ print_symname (const char *form, struct extended_symbol_info *info,
       if (atver)
 	*atver = 0;
     }
+
   if (do_demangle && *name)
     {
       alloc = bfd_demangle (abfd, name, demangle_flags);
@@ -458,6 +661,11 @@ print_symname (const char *form, struct extended_symbol_info *info,
 	name = alloc;
     }
 
+  if (unicode_display != unicode_default)
+    {
+      name = convert_utf8 (name);
+    }
+
   if (info != NULL && info->elfinfo && with_symbol_versions)
     {
       const char *version_string;
@@ -1808,7 +2016,7 @@ main (int argc, char **argv)
     fatal (_("fatal error: libbfd ABI mismatch"));
   set_default_bfd_target ();
 
-  while ((c = getopt_long (argc, argv, "aABCDef:gHhjJlnopPrSst:uvVvX:",
+  while ((c = getopt_long (argc, argv, "aABCDef:gHhjJlnopPrSst:uU:vVvX:",
 			   long_options, (int *) 0)) != EOF)
     {
       switch (c)
@@ -1901,6 +2109,24 @@ main (int argc, char **argv)
 	case 'u':
 	  undefined_only = 1;
 	  break;
+
+	case 'U':
+	  if (streq (optarg, "default") || streq (optarg, "d"))
+	    unicode_display = unicode_default;
+	  else if (streq (optarg, "locale") || streq (optarg, "l"))
+	    unicode_display = unicode_locale;
+	  else if (streq (optarg, "escape") || streq (optarg, "e"))
+	    unicode_display = unicode_escape;
+	  else if (streq (optarg, "invalid") || streq (optarg, "i"))
+	    unicode_display = unicode_invalid;
+	  else if (streq (optarg, "hex") || streq (optarg, "x"))
+	    unicode_display = unicode_hex;
+	  else if (streq (optarg, "highlight") || streq (optarg, "h"))
+	    unicode_display = unicode_highlight;
+	  else
+	    fatal (_("invalid argument to -U/--unicode: %s"), optarg);
+	  break;
+
 	case 'V':
 	  show_version = 1;
 	  break;
diff --git a/binutils/objdump.c b/binutils/objdump.c
index 1bfb5c71e7a..20499aaba0f 100644
--- a/binutils/objdump.c
+++ b/binutils/objdump.c
@@ -204,6 +204,18 @@ static const struct objdump_private_desc * const objdump_private_vectors[] =
 
 /* The list of detected jumps inside a function.  */
 static struct jump_info *detected_jumps = NULL;
+
+typedef enum unicode_display_type
+{
+  unicode_default = 0,
+  unicode_locale,
+  unicode_escape,
+  unicode_hex,
+  unicode_highlight,
+  unicode_invalid
+} unicode_display_type;
+
+static unicode_display_type unicode_display = unicode_default;
 \f
 static void usage (FILE *, int) ATTRIBUTE_NORETURN;
 static void
@@ -330,6 +342,9 @@ usage (FILE *stream, int status)
       fprintf (stream, _("\
   -w, --wide                     Format output for more than 80 columns\n"));
       fprintf (stream, _("\
+  -U[d|l|i|x|e|h]                Controls the display of UTF-8 unicode characters\n\
+  --unicode=[default|locale|invalid|hex|escape|highlight]\n"));
+      fprintf (stream, _("\
   -z, --disassemble-zeroes       Do not skip blocks of zeroes when disassembling\n"));
       fprintf (stream, _("\
       --start-address=ADDR       Only process data whose address is >= ADDR\n"));
@@ -420,17 +435,23 @@ static struct option long_options[]=
 {
   {"adjust-vma", required_argument, NULL, OPTION_ADJUST_VMA},
   {"all-headers", no_argument, NULL, 'x'},
-  {"private-headers", no_argument, NULL, 'p'},
-  {"private", required_argument, NULL, 'P'},
   {"architecture", required_argument, NULL, 'm'},
   {"archive-headers", no_argument, NULL, 'a'},
+#ifdef ENABLE_LIBCTF
+  {"ctf", required_argument, NULL, OPTION_CTF},
+  {"ctf-parent", required_argument, NULL, OPTION_CTF_PARENT},
+#endif
   {"debugging", no_argument, NULL, 'g'},
   {"debugging-tags", no_argument, NULL, 'e'},
   {"demangle", optional_argument, NULL, 'C'},
   {"disassemble", optional_argument, NULL, 'd'},
   {"disassemble-all", no_argument, NULL, 'D'},
-  {"disassembler-options", required_argument, NULL, 'M'},
   {"disassemble-zeroes", no_argument, NULL, 'z'},
+  {"disassembler-options", required_argument, NULL, 'M'},
+  {"dwarf", optional_argument, NULL, OPTION_DWARF},
+  {"dwarf-check", no_argument, 0, OPTION_DWARF_CHECK},
+  {"dwarf-depth", required_argument, 0, OPTION_DWARF_DEPTH},
+  {"dwarf-start", required_argument, 0, OPTION_DWARF_START},
   {"dynamic-reloc", no_argument, NULL, 'R'},
   {"dynamic-syms", no_argument, NULL, 'T'},
   {"endian", required_argument, NULL, OPTION_ENDIAN},
@@ -440,16 +461,23 @@ static struct option long_options[]=
   {"full-contents", no_argument, NULL, 's'},
   {"headers", no_argument, NULL, 'h'},
   {"help", no_argument, NULL, 'H'},
+  {"include", required_argument, NULL, 'I'},
   {"info", no_argument, NULL, 'i'},
+  {"inlines", no_argument, 0, OPTION_INLINES},
+  {"insn-width", required_argument, NULL, OPTION_INSN_WIDTH},
   {"line-numbers", no_argument, NULL, 'l'},
-  {"no-show-raw-insn", no_argument, &show_raw_insn, -1},
   {"no-addresses", no_argument, &no_addresses, 1},
-  {"process-links", no_argument, &process_links, true},
+  {"no-recurse-limit", no_argument, NULL, OPTION_NO_RECURSE_LIMIT},
+  {"no-recursion-limit", no_argument, NULL, OPTION_NO_RECURSE_LIMIT},
+  {"no-show-raw-insn", no_argument, &show_raw_insn, -1},
+  {"prefix", required_argument, NULL, OPTION_PREFIX},
   {"prefix-addresses", no_argument, &prefix_addresses, 1},
+  {"prefix-strip", required_argument, NULL, OPTION_PREFIX_STRIP},
+  {"private", required_argument, NULL, 'P'},
+  {"private-headers", no_argument, NULL, 'p'},
+  {"process-links", no_argument, &process_links, true},
   {"recurse-limit", no_argument, NULL, OPTION_RECURSE_LIMIT},
   {"recursion-limit", no_argument, NULL, OPTION_RECURSE_LIMIT},
-  {"no-recurse-limit", no_argument, NULL, OPTION_NO_RECURSE_LIMIT},
-  {"no-recursion-limit", no_argument, NULL, OPTION_NO_RECURSE_LIMIT},
   {"reloc", no_argument, NULL, 'r'},
   {"section", required_argument, NULL, 'j'},
   {"section-headers", no_argument, NULL, 'h'},
@@ -457,28 +485,16 @@ static struct option long_options[]=
   {"source", no_argument, NULL, 'S'},
   {"source-comment", optional_argument, NULL, OPTION_SOURCE_COMMENT},
   {"special-syms", no_argument, &dump_special_syms, 1},
-  {"include", required_argument, NULL, 'I'},
-  {"dwarf", optional_argument, NULL, OPTION_DWARF},
-#ifdef ENABLE_LIBCTF
-  {"ctf", optional_argument, NULL, OPTION_CTF},
-  {"ctf-parent", required_argument, NULL, OPTION_CTF_PARENT},
-#endif
   {"stabs", no_argument, NULL, 'G'},
   {"start-address", required_argument, NULL, OPTION_START_ADDRESS},
   {"stop-address", required_argument, NULL, OPTION_STOP_ADDRESS},
   {"syms", no_argument, NULL, 't'},
   {"target", required_argument, NULL, 'b'},
+  {"unicode", required_argument, NULL, 'U'},
   {"version", no_argument, NULL, 'V'},
-  {"wide", no_argument, NULL, 'w'},
-  {"prefix", required_argument, NULL, OPTION_PREFIX},
-  {"prefix-strip", required_argument, NULL, OPTION_PREFIX_STRIP},
-  {"insn-width", required_argument, NULL, OPTION_INSN_WIDTH},
-  {"dwarf-depth", required_argument, 0, OPTION_DWARF_DEPTH},
-  {"dwarf-start", required_argument, 0, OPTION_DWARF_START},
-  {"dwarf-check", no_argument, 0, OPTION_DWARF_CHECK},
-  {"inlines", no_argument, 0, OPTION_INLINES},
   {"visualize-jumps", optional_argument, 0, OPTION_VISUALIZE_JUMPS},
-  {0, no_argument, 0, 0}
+  {"wide", no_argument, NULL, 'w'},
+  {NULL, no_argument, NULL, 0}
 };
 \f
 static void
@@ -488,9 +504,121 @@ nonfatal (const char *msg)
   exit_status = 1;
 }
 
+/* Convert a potential UTF-8 encoded sequence in IN into characters in OUT.
+   The conversion format is controlled by the unicode_display variable.
+   Returns the number of characters added to OUT.
+   Returns the number of bytes consumed from IN in CONSUMED.
+   Always consumes at least one byte and displays at least one character.  */
+   
+static unsigned int
+display_utf8 (const unsigned char * in, char * out, unsigned int * consumed)
+{
+  char *        orig_out = out;
+  unsigned int  nchars = 0;
+  unsigned int j;
+
+  if (unicode_display == unicode_default)
+    goto invalid;
+
+  if (in[0] < 0xc0)
+    goto invalid;
+
+  if ((in[1] & 0xc0) != 0x80)
+    goto invalid;
+
+  if ((in[0] & 0x20) == 0)
+    {
+      nchars = 2;
+      goto valid;
+    }
+
+  if ((in[2] & 0xc0) != 0x80)
+    goto invalid;
+
+  if ((in[0] & 0x10) == 0)
+    {
+      nchars = 3;
+      goto valid;
+    }
+
+  if ((in[3] & 0xc0) != 0x80)
+    goto invalid;
+
+  nchars = 4;
+
+ valid:
+  switch (unicode_display)
+    {
+    case unicode_locale:
+      /* Copy the bytes into the output buffer as is.  */
+      memcpy (out, in, nchars);
+      out += nchars;
+      break;
+
+    case unicode_invalid:
+    case unicode_hex:
+      out += sprintf (out, "%c", unicode_display == unicode_hex ? '<' : '{');
+      out += sprintf (out, "0x");
+      for (j = 0; j < nchars; j++)
+	out += sprintf (out, "%02x", in [j]);
+      out += sprintf (out, "%c", unicode_display == unicode_hex ? '>' : '}');
+      break;
+      
+    case unicode_highlight:
+      if (isatty (1))
+	out += sprintf (out, "\x1B[31;47m"); /* Red.  */
+      /* Fall through.  */
+    case unicode_escape:
+      switch (nchars)
+	{
+	case 2:
+	  out += sprintf (out, "\\u%02x%02x",
+		  ((in[0] & 0x1c) >> 2), 
+		  ((in[0] & 0x03) << 6) | (in[1] & 0x3f));
+	  break;
+
+	case 3:
+	  out += sprintf (out, "\\u%02x%02x",
+		  ((in[0] & 0x0f) << 4) | ((in[1] & 0x3c) >> 2),
+		  ((in[1] & 0x03) << 6) | ((in[2] & 0x3f)));
+	  break;
+
+	case 4:
+	  out += sprintf (out, "\\u%02x%02x%02x",
+		  ((in[0] & 0x07) << 6) | ((in[1] & 0x3c) >> 2),
+		  ((in[1] & 0x03) << 6) | ((in[2] & 0x3c) >> 2),
+		  ((in[2] & 0x03) << 6) | ((in[3] & 0x3f)));
+	  break;
+	default:
+	  /* URG.  */
+	  break;
+	}
+
+      if (unicode_display == unicode_highlight && isatty (1))
+	out += sprintf (out, "\033[0m"); /* Default colour.  */
+      break;
+
+    default:
+      /* URG */
+      break;
+    }
+
+  * consumed = nchars;
+  return out - orig_out;
+
+ invalid:
+  /* Not a valid UTF-8 sequence.  */
+  *out = *in;
+  * consumed = 1;
+  return 1;
+}
+
 /* Returns a version of IN with any control characters
    replaced by escape sequences.  Uses a static buffer
-   if necessary.  */
+   if necessary.
+
+   If unicode display is enabled, then also handles the
+   conversion of unicode characters.  */
 
 static const char *
 sanitize_string (const char * in)
@@ -508,40 +636,50 @@ sanitize_string (const char * in)
      of cases it will not be needed.  */
   do
     {
-      char c = *in++;
+      unsigned char c = *in++;
 
       if (c == 0)
 	return original;
 
       if (ISCNTRL (c))
 	break;
+
+      if (unicode_display != unicode_default && c >= 0xc0)
+	break;
     }
   while (1);
 
   /* Copy the input, translating as needed.  */
   in = original;
-  if (buffer_len < (strlen (in) * 2))
+  if (buffer_len < (strlen (in) * 9))
     {
       free ((void *) buffer);
-      buffer_len = strlen (in) * 2;
+      buffer_len = strlen (in) * 9;
       buffer = xmalloc (buffer_len + 1);
     }
 
   out = buffer;
   do
     {
-      char c = *in++;
+      unsigned char c = *in++;
 
       if (c == 0)
 	break;
 
-      if (!ISCNTRL (c))
-	*out++ = c;
-      else
+      if (ISCNTRL (c))
 	{
 	  *out++ = '^';
 	  *out++ = c + 0x40;
 	}
+      else if (unicode_display != unicode_default && c >= 0xc0)
+	{
+	  unsigned int num_consumed;
+
+	  out += display_utf8 ((const unsigned char *)(in - 1), out, & num_consumed);
+	  in += num_consumed - 1;
+	}
+      else
+	*out++ = c;
     }
   while (1);
 
@@ -4522,6 +4660,24 @@ dump_symbols (bfd *abfd ATTRIBUTE_UNUSED, bool dynamic)
 		  free (alloc);
 		}
 	    }
+	  else if (unicode_display != unicode_default
+		   && name != NULL && *name != '\0')
+	    {
+	      const char * sanitized_name;
+
+	      /* If we want to sanitize the name, we do it here, and
+		 temporarily clobber it while calling bfd_print_symbol.
+		 FIXME: This is a gross hack.  */
+	      sanitized_name = sanitize_string (name);
+	      if (sanitized_name != name)
+		(*current)->name = sanitized_name;
+	      else
+		sanitized_name = NULL;
+	      bfd_print_symbol (cur_bfd, stdout, *current,
+				bfd_print_symbol_all);
+	      if (sanitized_name != NULL)
+		(*current)->name = name;
+	    }
 	  else
 	    bfd_print_symbol (cur_bfd, stdout, *current,
 			      bfd_print_symbol_all);
@@ -5205,7 +5361,7 @@ main (int argc, char **argv)
   set_default_bfd_target ();
 
   while ((c = getopt_long (argc, argv,
-			   "CDE:FGHI:LM:P:RSTVW::ab:defghij:lm:prstvwxz",
+			   "CDE:FGHI:LM:P:RSTU:VW::ab:defghij:lm:prstvwxz",
 			   long_options, (int *) 0))
 	 != EOF)
     {
@@ -5489,6 +5645,23 @@ main (int argc, char **argv)
 	  seenflag = true;
 	  break;
 
+	case 'U':
+	  if (streq (optarg, "default") || streq (optarg, "d"))
+	    unicode_display = unicode_default;
+	  else if (streq (optarg, "locale") || streq (optarg, "l"))
+	    unicode_display = unicode_locale;
+	  else if (streq (optarg, "escape") || streq (optarg, "e"))
+	    unicode_display = unicode_escape;
+	  else if (streq (optarg, "invalid") || streq (optarg, "i"))
+	    unicode_display = unicode_invalid;
+	  else if (streq (optarg, "hex") || streq (optarg, "x"))
+	    unicode_display = unicode_hex;
+	  else if (streq (optarg, "highlight") || streq (optarg, "h"))
+	    unicode_display = unicode_highlight;
+	  else
+	    fatal (_("invalid argument to -U/--unicode: %s"), optarg);
+	  break;
+
 	case 'H':
 	  usage (stdout, 0);
 	  /* No need to set seenflag or to break - usage() does not return.  */
diff --git a/binutils/readelf.c b/binutils/readelf.c
index cf3168f4a8e..50129f4ab4f 100644
--- a/binutils/readelf.c
+++ b/binutils/readelf.c
@@ -329,6 +329,19 @@ typedef enum print_mode
 }
 print_mode;
 
+typedef enum unicode_display_type
+{
+  unicode_default = 0,
+  unicode_locale,
+  unicode_escape,
+  unicode_hex,
+  unicode_highlight,
+  unicode_invalid
+} unicode_display_type;
+
+static unicode_display_type unicode_display = unicode_default;
+
+  
 /* Versioned symbol info.  */
 enum versioned_symbol_info
 {
@@ -656,11 +669,18 @@ print_symbol (signed int width, const char * symbol)
       if (c == 0)
 	break;
 
-      /* Do not print control characters directly as they can affect terminal
-	 settings.  Such characters usually appear in the names generated
-	 by the assembler for local labels.  */
-      if (ISCNTRL (c))
+      if (ISPRINT (c))
+	{
+	  putchar (c);
+	  width_remaining --;
+	  num_printed ++;
+	}
+      else if (ISCNTRL (c))
 	{
+	  /* Do not print control characters directly as they can affect terminal
+	     settings.  Such characters usually appear in the names generated
+	     by the assembler for local labels.  */
+
 	  if (width_remaining < 2)
 	    break;
 
@@ -668,11 +688,137 @@ print_symbol (signed int width, const char * symbol)
 	  width_remaining -= 2;
 	  num_printed += 2;
 	}
-      else if (ISPRINT (c))
+      else if (c == 0x7f)
 	{
-	  putchar (c);
-	  width_remaining --;
-	  num_printed ++;
+	  if (width_remaining < 5)
+	    break;
+	  printf ("<DEL>");
+	  width_remaining -= 5;
+	  num_printed += 5;
+	}
+      else if (unicode_display != unicode_locale
+	       && unicode_display != unicode_default)
+	{
+	  /* Display unicode characters as something else.  */
+	  unsigned char bytes[4];
+	  bool          is_utf8;
+	  uint          nbytes;
+
+	  bytes[0] = c;
+
+	  if (bytes[0] < 0xc0)
+	    {
+	      nbytes = 1;
+	      is_utf8 = false;
+	    }
+	  else
+	    {
+	      bytes[1] = *symbol++;
+
+	      if ((bytes[1] & 0xc0) != 0x80)
+		{
+		  is_utf8 = false;
+		  /* Do not consume this character.  It may only
+		     be the first byte in the sequence that was
+		     corrupt.  */
+		  --symbol;
+		  nbytes = 1;
+		}
+	      else if ((bytes[0] & 0x20) == 0)
+		{
+		  is_utf8 = true;
+		  nbytes = 2;
+		}
+	      else
+		{
+		  bytes[2] = *symbol++;
+
+		  if ((bytes[2] & 0xc0) != 0x80)
+		    {
+		      is_utf8 = false;
+		      symbol -= 2;
+		      nbytes = 1;
+		    }
+		  else if ((bytes[0] & 0x10) == 0)
+		    {
+		      is_utf8 = true;
+		      nbytes = 3;
+		    }
+		  else
+		    {
+		      bytes[3] = *symbol++;
+
+		      nbytes = 4;
+
+		      if ((bytes[3] & 0xc0) != 0x80)
+			{
+			  is_utf8 = false;
+			  symbol -= 3;
+			  nbytes = 1;
+			}
+		      else
+			is_utf8 = true;
+		    }
+		}
+	    }
+
+	  if (unicode_display == unicode_invalid)
+	    is_utf8 = false;
+
+	  if (unicode_display == unicode_hex || ! is_utf8)
+	    {
+	      uint i;
+
+	      if (width_remaining < (nbytes * 2) + 2)
+		break;
+	  
+	      putchar (is_utf8 ? '<' : '{');
+	      printf ("0x");
+	      for (i = 0; i < nbytes; i++)
+		printf ("%02x", bytes[i]);
+	      putchar (is_utf8 ? '>' : '}');
+	    }
+	  else
+	    {
+	      if (unicode_display == unicode_highlight && isatty (1))
+		printf ("\x1B[31;47m"); /* Red.  */
+	      
+	      switch (nbytes)
+		{
+		case 2:
+		  if (width_remaining < 6)
+		    break;
+		  printf ("\\u%02x%02x",
+			  (bytes[0] & 0x1c) >> 2, 
+			  ((bytes[0] & 0x03) << 6) | (bytes[1] & 0x3f));
+		  break;
+		case 3:
+		  if (width_remaining < 6)
+		    break;
+		  printf ("\\u%02x%02x",
+			  ((bytes[0] & 0x0f) << 4) | ((bytes[1] & 0x3c) >> 2),
+			  ((bytes[1] & 0x03) << 6) | (bytes[2] & 0x3f));
+		  break;
+		case 4:
+		  if (width_remaining < 8)
+		    break;
+		  printf ("\\u%02x%02x%02x",
+			  ((bytes[0] & 0x07) << 6) | ((bytes[1] & 0x3c) >> 2),
+			  ((bytes[1] & 0x03) << 6) | ((bytes[2] & 0x3c) >> 2),
+			  ((bytes[2] & 0x03) << 6) | (bytes[3] & 0x3f));
+		  
+		  break;
+		default:
+		  /* URG.  */
+		  break;
+		}
+
+	      if (unicode_display == unicode_highlight && isatty (1))
+		printf ("\033[0m"); /* Default colour.  */
+	    }
+	  
+	  if (bytes[nbytes - 1] == 0)
+	    break;
 	}
       else
 	{
@@ -4731,6 +4877,7 @@ static struct option options[] =
   {"syms",	       no_argument, 0, 's'},
   {"silent-truncation",no_argument, 0, 'T'},
   {"section-details",  no_argument, 0, 't'},
+  {"unicode",          required_argument, NULL, 'U'},
   {"unwind",	       no_argument, 0, 'u'},
   {"version-info",     no_argument, 0, 'V'},
   {"version",	       no_argument, 0, 'v'},
@@ -4806,6 +4953,12 @@ usage (FILE * stream)
      --recurse-limit     Enable a demangling recursion limit.  (default)\n"));
   fprintf (stream, _("\
      --no-recurse-limit  Disable a demangling recursion limit\n"));
+  fprintf (stream, _("\
+     -U[dlexhi] --unicode=[default|locale|escape|hex|highlight|invalid]\n\
+                         Display unicode characters as determined by the current locale\n\
+                          (default), escape sequences, \"<hex sequences>\", highlighted\n\
+                          escape sequences, or treat them as invalid and display as\n\
+                          \"{hex sequences}\"\n"));
   fprintf (stream, _("\
   -n --notes             Display the core notes (if present)\n"));
   fprintf (stream, _("\
@@ -4990,7 +5143,7 @@ parse_args (struct dump_data *dumpdata, int argc, char ** argv)
     usage (stderr);
 
   while ((c = getopt_long
-	  (argc, argv, "ACDHILNPR:STVWacdeghi:lnp:rstuvw::x:z", options, NULL)) != EOF)
+	  (argc, argv, "ACDHILNPR:STU:VWacdeghi:lnp:rstuvw::x:z", options, NULL)) != EOF)
     {
       switch (c)
 	{
@@ -5192,6 +5345,25 @@ parse_args (struct dump_data *dumpdata, int argc, char ** argv)
 	  /* Ignored for backward compatibility.  */
 	  break;
 
+	case 'U':
+	  if (optarg == NULL)
+	    error (_("Missing arg to -U/--unicode")); /* Can this happen ?  */
+	  else if (streq (optarg, "default") || streq (optarg, "d"))
+	    unicode_display = unicode_default;
+	  else if (streq (optarg, "locale") || streq (optarg, "l"))
+	    unicode_display = unicode_locale;
+	  else if (streq (optarg, "escape") || streq (optarg, "e"))
+	    unicode_display = unicode_escape;
+	  else if (streq (optarg, "invalid") || streq (optarg, "i"))
+	    unicode_display = unicode_invalid;
+	  else if (streq (optarg, "hex") || streq (optarg, "x"))
+	    unicode_display = unicode_hex;
+	  else if (streq (optarg, "highlight") || streq (optarg, "h"))
+	    unicode_display = unicode_highlight;
+	  else
+	    error (_("invalid argument to -U/--unicode: %s"), optarg);
+	  break;
+
 	case OPTION_SYM_BASE:
 	  sym_base = 0;
 	  if (optarg != NULL)
diff --git a/binutils/strings.c b/binutils/strings.c
index 44a8e1da644..f50badf983e 100644
--- a/binutils/strings.c
+++ b/binutils/strings.c
@@ -55,6 +55,19 @@
    -T {bfdname}
 		Specify a non-default object file format.
 
+  --unicode={default|locale|invalid|hex|escape|highlight}
+  -u {d|l|i|x|e|h}
+                Determine how to handle UTF-8 unicode characters.  The default
+		is no special treatment.  All other versions of this option
+		only apply if the encoding is valid and enabling the option
+		implies --encoding=S.
+		The 'locale' option displays the characters according to the
+		current locale.  The 'invalid' option treats them as
+		non-string characters.  The 'hex' option displays them as hex
+		byte sequences.  The 'escape' option displays them as escape
+		sequences and the 'highlight' option displays them as
+		coloured escape sequences.
+
   --output-separator=sep_string
   -s sep_string	String used to separate parsed strings in output.
 		Default is newline.
@@ -76,6 +89,22 @@
 #include "safe-ctype.h"
 #include "bucomm.h"
 
+#ifndef streq
+#define streq(a,b) (strcmp ((a),(b)) == 0)
+#endif
+
+typedef enum unicode_display_type
+{
+  unicode_default = 0,
+  unicode_locale,
+  unicode_escape,
+  unicode_hex,
+  unicode_highlight,
+  unicode_invalid
+} unicode_display_type;
+
+static unicode_display_type unicode_display = unicode_default;
+
 #define STRING_ISGRAPHIC(c) \
       (   (c) >= 0 \
        && (c) <= 255 \
@@ -94,7 +123,7 @@ extern int errno;
 static int address_radix;
 
 /* Minimum length of sequence of graphic chars to trigger output.  */
-static int string_min;
+static uint string_min;
 
 /* Whether or not we include all whitespace as a graphic char.   */
 static bool include_all_whitespace;
@@ -121,21 +150,22 @@ static char *output_separator;
 static struct option long_options[] =
 {
   {"all", no_argument, NULL, 'a'},
+  {"bytes", required_argument, NULL, 'n'},
   {"data", no_argument, NULL, 'd'},
+  {"encoding", required_argument, NULL, 'e'},
+  {"help", no_argument, NULL, 'h'},
+  {"include-all-whitespace", no_argument, NULL, 'w'},
+  {"output-separator", required_argument, NULL, 's'},
   {"print-file-name", no_argument, NULL, 'f'},
-  {"bytes", required_argument, NULL, 'n'},
   {"radix", required_argument, NULL, 't'},
-  {"include-all-whitespace", no_argument, NULL, 'w'},
-  {"encoding", required_argument, NULL, 'e'},
   {"target", required_argument, NULL, 'T'},
-  {"output-separator", required_argument, NULL, 's'},
-  {"help", no_argument, NULL, 'h'},
+  {"unicode", required_argument, NULL, 'U'},
   {"version", no_argument, NULL, 'v'},
   {NULL, 0, NULL, 0}
 };
 
 static bool strings_file (char *);
-static void print_strings (const char *, FILE *, file_ptr, int, int, char *);
+static void print_strings (const char *, FILE *, file_ptr, int, char *);
 static void usage (FILE *, int) ATTRIBUTE_NORETURN;
 \f
 int main (int, char **);
@@ -171,7 +201,7 @@ main (int argc, char **argv)
   encoding = 's';
   output_separator = NULL;
 
-  while ((optc = getopt_long (argc, argv, "adfhHn:wot:e:T:s:Vv0123456789",
+  while ((optc = getopt_long (argc, argv, "adfhHn:wot:e:T:s:U:Vv0123456789",
 			      long_options, (int *) 0)) != EOF)
     {
       switch (optc)
@@ -244,6 +274,23 @@ main (int argc, char **argv)
 	  output_separator = optarg;
           break;
 
+	case 'U':
+	  if (streq (optarg, "default") || streq (optarg, "d"))
+	    unicode_display = unicode_default;
+	  else if (streq (optarg, "locale") || streq (optarg, "l"))
+	    unicode_display = unicode_locale;
+	  else if (streq (optarg, "escape") || streq (optarg, "e"))
+	    unicode_display = unicode_escape;
+	  else if (streq (optarg, "invalid") || streq (optarg, "i"))
+	    unicode_display = unicode_invalid;
+	  else if (streq (optarg, "hex") || streq (optarg, "x"))
+	    unicode_display = unicode_hex;
+	  else if (streq (optarg, "highlight") || streq (optarg, "h"))
+	    unicode_display = unicode_highlight;
+	  else
+	    fatal (_("invalid argument to -U/--unicode: %s"), optarg);
+	  break;
+
 	case 'V':
 	case 'v':
 	  print_version ("strings");
@@ -258,6 +305,9 @@ main (int argc, char **argv)
 	}
     }
 
+  if (unicode_display != unicode_default)
+    encoding = 'S';
+
   if (numeric_opt != 0)
     {
       string_min = (int) strtoul (argv[numeric_opt - 1] + 1, &s, 0);
@@ -293,14 +343,14 @@ main (int argc, char **argv)
     {
       datasection_only = false;
       SET_BINARY (fileno (stdin));
-      print_strings ("{standard input}", stdin, 0, 0, 0, (char *) NULL);
+      print_strings ("{standard input}", stdin, 0, 0, (char *) NULL);
       files_given = true;
     }
   else
     {
       for (; optind < argc; ++optind)
 	{
-	  if (strcmp (argv[optind], "-") == 0)
+	  if (streq (argv[optind], "-"))
 	    datasection_only = false;
 	  else
 	    {
@@ -342,7 +392,7 @@ strings_a_section (bfd *abfd, asection *sect, const char *filename,
     }
 
   *got_a_section = true;
-  print_strings (filename, NULL, sect->filepos, 0, sectsize, (char *) mem);
+  print_strings (filename, NULL, sect->filepos, sectsize, (char *) mem);
   free (mem);
 }
 
@@ -427,7 +477,7 @@ strings_file (char *file)
 	  return false;
 	}
 
-      print_strings (file, stream, (file_ptr) 0, 0, 0, (char *) 0);
+      print_strings (file, stream, (file_ptr) 0, 0, (char *) NULL);
 
       if (fclose (stream) == EOF)
 	{
@@ -551,11 +601,627 @@ unget_part_char (long c, file_ptr *address, int *magiccount, char **magic)
 	}
     }
 }
+
+static void
+print_filename_and_address (const char * filename, file_ptr address)
+{
+  if (print_filenames)
+    printf ("%s: ", filename);
+
+  if (! print_addresses)
+    return;
+
+  switch (address_radix)
+    {
+    case 8:
+      if (sizeof (address) > sizeof (long))
+	{
+#ifndef __MSVCRT__
+	  printf ("%7llo ", (unsigned long long) address);
+#else
+	  printf ("%7I64o ", (unsigned long long) address);
+#endif
+	}
+      else
+	printf ("%7lo ", (unsigned long) address);
+      break;
+
+    case 10:
+      if (sizeof (address) > sizeof (long))
+	{
+#ifndef __MSVCRT__
+	  printf ("%7llu ", (unsigned long long) address);
+#else
+	  printf ("%7I64d ", (unsigned long long) address);
+#endif
+	}
+      else
+	printf ("%7ld ", (long) address);
+      break;
+
+    case 16:
+      if (sizeof (address) > sizeof (long))
+	{
+#ifndef __MSVCRT__
+	  printf ("%7llx ", (unsigned long long) address);
+#else
+	  printf ("%7I64x ", (unsigned long long) address);
+#endif
+	}
+      else
+	printf ("%7lx ", (unsigned long) address);
+      break;
+    }
+}
+
+/* Return non-zero if the bytes starting at BUFFER form a valid UTF-8 encoding.
+   If the encoding is valid then returns the number of bytes it uses.  */
+
+static unsigned int
+is_valid_utf8 (const unsigned char * buffer, unsigned long buflen)
+{
+  if (buffer[0] < 0xc0)
+    return 0;
+
+  if (buflen < 2)
+    return 0;
+
+  if ((buffer[1] & 0xc0) != 0x80)
+    return 0;
+
+  if ((buffer[0] & 0x20) == 0)
+    return 2;
+
+  if (buflen < 3)
+    return 0;
+
+  if ((buffer[2] & 0xc0) != 0x80)
+    return 0;
+  
+  if ((buffer[0] & 0x10) == 0)
+    return 3;
+
+  if (buflen < 4)
+    return 0;
+
+  if ((buffer[3] & 0xc0) != 0x80)
+    return 0;
+
+  return 4;
+}
+
+/* Display a UTF-8 encoded character in BUFFER according to the setting
+   of unicode_display.  The character is known to be valid.
+   Returns the number of bytes consumed.  */
+
+static uint
+display_utf8_char (const unsigned char * buffer)
+{
+  uint j;
+  uint utf8_len;
+
+  switch (buffer[0] & 0x30)
+    {
+    case 0x00:
+    case 0x10:
+      utf8_len = 2;
+      break;
+    case 0x20:
+      utf8_len = 3;
+      break;
+    default:
+      utf8_len = 4;
+    }
+      
+  switch (unicode_display)
+    {
+    default:
+      fprintf (stderr, "ICE: unexpected unicode display type\n");
+      break;
+
+    case unicode_escape:
+    case unicode_highlight:
+      if (unicode_display == unicode_highlight && isatty (1))
+	printf ("\x1B[31;47m"); /* Red.  */
+
+      switch (utf8_len)
+	{
+	case 2:
+	  printf ("\\u%02x%02x",
+		  ((buffer[0] & 0x1c) >> 2), 
+		  ((buffer[0] & 0x03) << 6) | (buffer[1] & 0x3f));
+	  break;
+
+	case 3:
+	  printf ("\\u%02x%02x",
+		  ((buffer[0] & 0x0f) << 4) | ((buffer[1] & 0x3c) >> 2),
+		  ((buffer[1] & 0x03) << 6) | ((buffer[2] & 0x3f)));
+	  break;
+
+	case 4:
+	  printf ("\\u%02x%02x%02x",
+		  ((buffer[0] & 0x07) << 6) | ((buffer[1] & 0x3c) >> 2),
+		  ((buffer[1] & 0x03) << 6) | ((buffer[2] & 0x3c) >> 2),
+		  ((buffer[2] & 0x03) << 6) | ((buffer[3] & 0x3f)));
+	  break;
+	default:
+	  /* URG.  */
+	  break;
+	}
+
+      if (unicode_display == unicode_highlight && isatty (1))
+	printf ("\033[0m"); /* Default colour.  */
+      break;
+
+    case unicode_hex:
+      putchar ('<');
+      printf ("0x");
+      for (j = 0; j < utf8_len; j++)
+	printf ("%02x", buffer [j]);
+      putchar ('>');
+      break;
+
+    case unicode_locale:
+      printf ("%.1s", buffer);
+      break;
+    }
+
+  return utf8_len;
+}
+
+/* Display strings in BUFFER.  Treat any UTF-8 encoded characters encountered
+   according to the setting of the unicode_display variable.  The buffer
+   contains BUFLEN bytes.
+
+   Display the characters as if they started at ADDRESS and are contained in
+   FILENAME.  */
+
+static void
+print_unicode_buffer (const char *            filename,
+		      file_ptr                address,
+		      const unsigned char *   buffer,
+		      unsigned long           buflen)
+{
+  /* Paranoia checks...  */
+  if (filename == NULL
+      || buffer == NULL
+      || unicode_display == unicode_default
+      || encoding != 'S'
+      || encoding_bytes != 1)
+    {
+      fprintf (stderr, "ICE: bad arguments to print_unicode_buffer\n");
+      return;
+    }
+
+  if (buflen == 0)
+    return;
+
+  /* We must only display strings that are at least string_min *characters*
+     long.  So we scan the buffer in two stages.  First we locate the start
+     of a potential string.  Then we walk along it until we have found
+     string_min characters.  Then we go back to the start point and start
+     displaying characters according to the unicode_display setting.  */
+
+  unsigned long start_point = 0;
+  unsigned long i = 0;
+  unsigned int char_len = 1;
+  unsigned int num_found = 0;
+
+  for (i = 0; i < buflen; i += char_len)
+    {
+      int c = buffer[i];
+
+      char_len = 1;
+
+      /* Find the first potential character of a string.  */
+      if (! STRING_ISGRAPHIC (c))
+	{
+	  num_found = 0;
+	  continue;
+	}
+
+      if (c > 126)
+	{
+	  if (c < 0xc0)
+	    {
+	      num_found = 0;
+	      continue;
+	    }
+
+	  if ((char_len = is_valid_utf8 (buffer + i, buflen - i)) == 0)
+	    {
+	      char_len = 1;
+	      num_found = 0;
+	      continue;
+	    }
+
+	  if (unicode_display == unicode_invalid)
+	    {
+	      /* We have found a valid UTF-8 character, but we treat it as non-graphic.  */
+	      num_found = 0;
+	      continue;
+	    }
+	}
+
+      if (num_found == 0)
+	/* We have found a potential starting point for a string.  */
+	start_point = i;
+
+      ++ num_found;
+
+      if (num_found >= string_min)
+	break;
+    }
+
+  if (num_found < string_min)
+    return;
+
+  print_filename_and_address (filename, address + start_point);
+  
+  /* We have found string_min characters.  Display them and any
+     more that follow.  */
+  for (i = start_point; i < buflen; i += char_len)
+    {
+      int c = buffer[i];
+
+      char_len = 1;
+
+      if (! STRING_ISGRAPHIC (c))
+	break;
+      else if (c < 127)
+	putchar (c);
+      else if (! is_valid_utf8 (buffer + i, buflen - i))
+	break;
+      else if (unicode_display == unicode_invalid)
+	break;
+      else
+	char_len = display_utf8_char (buffer + i);
+    }
+
+  if (output_separator)
+    fputs (output_separator, stdout);
+  else
+    putchar ('\n');
+
+  /* FIXME: Using tail recursion here is lazy programming...  */
+  print_unicode_buffer (filename, address + i, buffer + i, buflen - i);
+}
+
+static int
+get_unicode_byte (FILE * stream, unsigned char * putback, uint * num_putback, uint * num_read)
+{
+  if (* num_putback > 0)
+    {
+      * num_putback = * num_putback - 1;
+      return putback [* num_putback];
+    }
+
+  * num_read = * num_read + 1;
+
+#if defined(HAVE_GETC_UNLOCKED) && HAVE_DECL_GETC_UNLOCKED
+  return getc_unlocked (stream);
+#else
+  return getc (stream);
+#endif
+}
+
+/* Helper function for print_unicode_stream.  */
+
+static void
+print_unicode_stream_body (const char *     filename,
+			   file_ptr         address,
+			   FILE *           stream,
+			   unsigned char *  putback_buf,
+			   uint             num_putback,
+			   unsigned char *  print_buf)
+{
+  /* It would be nice if we could just read the stream into a buffer
+     and then process if with print_unicode_buffer.  But the input
+     might be huge or it might time-locked (eg stdin).  So instead
+     we go one byte at a time...  */
+
+  file_ptr start_point = 0;
+  uint num_read = 0;
+  uint num_chars = 0;
+  uint num_print = 0;
+  int c;
+
+  /* Find a series of string_min characters.  Put them into print_buf.  */
+  do
+    {
+      if (num_chars >= string_min)
+	break;
+
+      c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+      if (c == EOF)
+	break;
+
+      if (! STRING_ISGRAPHIC (c))
+	{
+	  num_chars = num_print = 0;
+	  continue;
+	}
+
+      if (num_chars == 0)
+	start_point = num_read - 1;
+
+      if (c < 127)
+	{
+	  print_buf[num_print] = c;
+	  num_chars ++;
+	  num_print ++;
+	  continue;
+	}
+
+      if (c < 0xc0)
+	{
+	  num_chars = num_print = 0;
+	  continue;
+	}
+
+      /* We *might* have a UTF-8 sequence.  Time to start peeking.  */
+      char utf8[4];
+
+      utf8[0] = c;
+      c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+      if (c == EOF)
+	break;
+      utf8[1] = c;
+
+      if ((utf8[1] & 0xc0) != 0x80)
+	{
+	  /* Invalid UTF-8.  */
+	  putback_buf[num_putback++] = utf8[1];
+	  num_chars = num_print = 0;
+	  continue;
+	}
+      else if ((utf8[0] & 0x20) == 0)
+	{
+	  /* A valid 2-byte UTF-8 encoding.  */
+	  if (unicode_display == unicode_invalid)
+	    {
+	      putback_buf[num_putback++] = utf8[1];
+	      num_chars = num_print = 0;
+	    }
+	  else
+	    {
+	      print_buf[num_print ++] = utf8[0];
+	      print_buf[num_print ++] = utf8[1];
+	      num_chars ++;
+	    }
+	  continue;
+	}
+
+      c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+      if (c == EOF)
+	break;
+      utf8[2] = c;
+
+      if ((utf8[2] & 0xc0) != 0x80)
+	{
+	  /* Invalid UTF-8.  */
+	  putback_buf[num_putback++] = utf8[2];
+	  putback_buf[num_putback++] = utf8[1];
+	  num_chars = num_print = 0;
+	  continue;
+	}
+      else if ((utf8[0] & 0x10) == 0)
+	{
+	  /* A valid 3-byte UTF-8 encoding.  */
+	  if (unicode_display == unicode_invalid)
+	    {
+	      putback_buf[num_putback++] = utf8[2];
+	      putback_buf[num_putback++] = utf8[1];
+	      num_chars = num_print = 0;
+	    }
+	  else
+	    {
+	      print_buf[num_print ++] = utf8[0];
+	      print_buf[num_print ++] = utf8[1];
+	      print_buf[num_print ++] = utf8[2];
+	      num_chars ++;
+	    }
+	  continue;
+	}
+
+      c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+      if (c == EOF)
+	break;
+      utf8[3] = c;
+
+      if ((utf8[3] & 0xc0) != 0x80)
+	{
+	  /* Invalid UTF-8.  */
+	  putback_buf[num_putback++] = utf8[3];
+	  putback_buf[num_putback++] = utf8[2];
+	  putback_buf[num_putback++] = utf8[1];
+	  num_chars = num_print = 0;
+	}
+      /* We have a valid 4-byte UTF-8 encoding.  */
+      else if (unicode_display == unicode_invalid)
+	{
+	  putback_buf[num_putback++] = utf8[3];
+	  putback_buf[num_putback++] = utf8[1];
+	  putback_buf[num_putback++] = utf8[2];
+	  num_chars = num_print = 0;
+	}
+      else
+	{
+	  print_buf[num_print ++] = utf8[0];
+	  print_buf[num_print ++] = utf8[1];
+	  print_buf[num_print ++] = utf8[2];
+	  print_buf[num_print ++] = utf8[3];
+	  num_chars ++;
+	}
+    }
+  while (1);
+
+  if (num_chars >= string_min)
+    {
+      /* We know that we have string_min valid characters in print_buf,
+	 and there may be more to come in the stream.  Start displaying
+	 them.  */
+
+      print_filename_and_address (filename, address + start_point);
+
+      uint i;
+      for (i = 0; i < num_print;)
+	{
+	  if (print_buf[i] < 127)
+	    putchar (print_buf[i++]);
+	  else
+	    i += display_utf8_char (print_buf + i);
+	}
+
+      /* OK so now we have to start read unchecked bytes.  */
+
+        /* Find a series of string_min characters.  Put them into print_buf.  */
+      do
+	{
+	  c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+	  if (c == EOF)
+	    break;
+
+	  if (! STRING_ISGRAPHIC (c))
+	    break;
+
+	  if (c < 127)
+	    {
+	      putchar (c);
+	      continue;
+	    }
+
+	  if (c < 0xc0)
+	    break;
+
+	  /* We *might* have a UTF-8 sequence.  Time to start peeking.  */
+	  unsigned char utf8[4];
+
+	  utf8[0] = c;
+	  c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+	  if (c == EOF)
+	    break;
+	  utf8[1] = c;
+
+	  if ((utf8[1] & 0xc0) != 0x80)
+	    {
+	      /* Invalid UTF-8.  */
+	      putback_buf[num_putback++] = utf8[1];
+	      break;
+	    }
+	  else if ((utf8[0] & 0x20) == 0)
+	    {
+	      /* Valid 2-byte UTF-8.  */
+	      if (unicode_display == unicode_invalid)
+		{
+		  putback_buf[num_putback++] = utf8[1];
+		  break;
+		}
+	      else
+		{
+		  (void) display_utf8_char (utf8);
+		  continue;
+		}
+	    }
+
+	  c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+	  if (c == EOF)
+	    break;
+	  utf8[2] = c;
+
+	  if ((utf8[2] & 0xc0) != 0x80)
+	    {
+	      /* Invalid UTF-8.  */
+	      putback_buf[num_putback++] = utf8[2];
+	      putback_buf[num_putback++] = utf8[1];
+	      break;
+	    }
+	  else if ((utf8[0] & 0x10) == 0)
+	    {
+	      /* Valid 3-byte UTF-8.  */
+	      if (unicode_display == unicode_invalid)
+		{
+		  putback_buf[num_putback++] = utf8[2];
+		  putback_buf[num_putback++] = utf8[1];
+		  break;
+		}
+	      else
+		{
+		  (void) display_utf8_char (utf8);
+		  continue;
+		}
+	    }
+
+	  c = get_unicode_byte (stream, putback_buf, & num_putback, & num_read);
+	  if (c == EOF)
+	    break;
+	  utf8[3] = c;
+
+	  if ((utf8[3] & 0xc0) != 0x80)
+	    {
+	      /* Invalid UTF-8.  */
+	      putback_buf[num_putback++] = utf8[3];
+	      putback_buf[num_putback++] = utf8[2];
+	      putback_buf[num_putback++] = utf8[1];
+	      break;
+	    }
+	  else if (unicode_display == unicode_invalid)
+	    {
+	      putback_buf[num_putback++] = utf8[3];
+	      putback_buf[num_putback++] = utf8[2];
+	      putback_buf[num_putback++] = utf8[1];
+	      break;
+	    }
+	  else
+	    /* A valid 4-byte UTF-8 encoding.  */
+	    (void) display_utf8_char (utf8);
+	}
+      while (1);
+
+      if (output_separator)
+	fputs (output_separator, stdout);
+      else
+	putchar ('\n');
+    }
+
+  if (c != EOF)
+    /* FIXME: Using tail recursion here is lazy, but it works.  */
+    print_unicode_stream_body (filename, address + num_read, stream, putback_buf, num_putback, print_buf);
+}
+
+/* Display strings read in from STREAM.  Treat any UTF-8 encoded characters
+   encountered according to the setting of the unicode_display variable.
+   The stream is positioned at ADDRESS and is attached to FILENAME.  */
+
+static void
+print_unicode_stream (const char * filename,
+		      file_ptr     address,
+		      FILE *       stream)
+{
+  /* Paranoia checks...  */
+  if (filename == NULL
+      || stream == NULL
+      || unicode_display == unicode_default
+      || encoding != 'S'
+      || encoding_bytes != 1)
+    {
+      fprintf (stderr, "ICE: bad arguments to print_unicode_stream\n");
+      return;
+    }
+
+  /* Allocate space for string_min 4-byte utf-8 characters.  */
+  unsigned char * print_buf = xmalloc ((4 * string_min) + 1);
+  /* We should never have to put back more than 4 bytes.  */
+  unsigned char putback_buf[5];
+  uint num_putback = 0;
+
+  print_unicode_stream_body (filename, address, stream, putback_buf, num_putback, print_buf);
+  free (print_buf);
+}
 \f
 /* Find the strings in file FILENAME, read from STREAM.
    Assume that STREAM is positioned so that the next byte read
    is at address ADDRESS in the file.
-   Stop reading at address STOP_POINT in the file, if nonzero.
 
    If STREAM is NULL, do not read from it.
    The caller can supply a buffer of characters
@@ -566,20 +1232,29 @@ unget_part_char (long c, file_ptr *address, int *magiccount, char **magic)
 
 static void
 print_strings (const char *filename, FILE *stream, file_ptr address,
-	       int stop_point, int magiccount, char *magic)
+	       int magiccount, char *magic)
 {
+  if (unicode_display != unicode_default)
+    {
+      if (magic != NULL)
+	print_unicode_buffer (filename, address,
+			      (const unsigned char *) magic, magiccount);
+
+      if (stream != NULL)
+	print_unicode_stream (filename, address, stream);
+      return;
+    }
+
   char *buf = (char *) xmalloc (sizeof (char) * (string_min + 1));
 
   while (1)
     {
       file_ptr start;
-      int i;
+      uint i;
       long c;
 
       /* See if the next `string_min' chars are all graphic chars.  */
     tryline:
-      if (stop_point && address >= stop_point)
-	break;
       start = address;
       for (i = 0; i < string_min; i++)
 	{
@@ -601,51 +1276,7 @@ print_strings (const char *filename, FILE *stream, file_ptr address,
 
       /* We found a run of `string_min' graphic characters.  Print up
 	 to the next non-graphic character.  */
-
-      if (print_filenames)
-	printf ("%s: ", filename);
-      if (print_addresses)
-	switch (address_radix)
-	  {
-	  case 8:
-	    if (sizeof (start) > sizeof (long))
-	      {
-#ifndef __MSVCRT__
-		printf ("%7llo ", (unsigned long long) start);
-#else
-		printf ("%7I64o ", (unsigned long long) start);
-#endif
-	      }
-	    else
-	      printf ("%7lo ", (unsigned long) start);
-	    break;
-
-	  case 10:
-	    if (sizeof (start) > sizeof (long))
-	      {
-#ifndef __MSVCRT__
-		printf ("%7llu ", (unsigned long long) start);
-#else
-		printf ("%7I64d ", (unsigned long long) start);
-#endif
-	      }
-	    else
-	      printf ("%7ld ", (long) start);
-	    break;
-
-	  case 16:
-	    if (sizeof (start) > sizeof (long))
-	      {
-#ifndef __MSVCRT__
-		printf ("%7llx ", (unsigned long long) start);
-#else
-		printf ("%7I64x ", (unsigned long long) start);
-#endif
-	      }
-	    else
-	      printf ("%7lx ", (unsigned long) start);
-	    break;
-	  }
+      print_filename_and_address (filename, start);
 
       buf[i] = '\0';
       fputs (buf, stdout);
@@ -697,6 +1328,8 @@ usage (FILE *stream, int status)
   -T --target=<BFDNAME>     Specify the binary file format\n\
   -e --encoding={s,S,b,l,B,L} Select character size and endianness:\n\
                             s = 7-bit, S = 8-bit, {b,l} = 16-bit, {B,L} = 32-bit\n\
+  --unicode={default|show|invalid|hex|escape|highlight}\n\
+  -u {d|s|i|x|e|h}          Specify how to treat UTF-8 encoded unicode characters\n\
   -s --output-separator=<string> String used to separate strings in output.\n\
   @<file>                   Read options from <file>\n\
   -h --help                 Display this information\n\
--- /dev/null	2021-11-09 09:21:02.662310924 +0000
+++ binutils/testsuite/binutils-all/unicode.exp	2021-11-09 12:03:12.702905609 +0000
@@ -0,0 +1,71 @@
+#   Copyright (C) 2021 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA 02110-1301, USA.
+
+# Written by Nick Clifton <nickc@redhat.com>
+
+proc unicode_test { program options binary_file regexp_file } {
+
+    global srcdir
+    global subdir
+
+    set testname "unicode $program $options [file rootname [file tail $binary_file]]"
+
+    send_log "exec $program $options $binary_file > prog.out\n"
+    set got [remote_exec host "$program $options $binary_file" "" "/dev/null" "prog.out"]
+
+    if { [lindex $got 0] != 0 } then {
+	fail "$testname (reason: execution failed)"
+	send_log $got
+	send_log "\n"
+	return
+    }
+
+    if { ![string match "" [lindex $got 1]] } then {
+	fail "$testname (reason: error output)"
+	send_log $got
+	send_log "\n"
+	return
+    }
+
+    if { [regexp_diff prog.out $srcdir/$subdir/$regexp_file] } then {
+	send_log [file_contents prog.out]
+	send_log "\n"
+	fail $testname
+	return
+    }
+
+    pass $testname
+}
+
+# Assemble the test file.
+if {![binutils_assemble $srcdir/$subdir/trick-hello.s tmpdir/trick-hello.o]} then {
+    unsupported "unicode tests (failed to assemble test file)"
+} else {
+    if ![is_remote host] {
+	set tempfile tmpdir/trick-hello.o
+    } else {
+	set tempfile [remote_download host tmpdir/trick-hello.o]
+    }
+
+    global NM STRINGS OBJDUMP READELF
+    unicode_test $NM      {--unicode=hex}    $tempfile nm.hex.unicode
+    unicode_test $STRINGS {--unicode=escape} $tempfile strings.escape.unicode
+    unicode_test $OBJDUMP {-tUh}             $tempfile objdump.highlight.unicode
+
+    if [is_elf_format] {
+	unicode_test $READELF {-sUi}         $tempfile readelf.invalid.unicode
+    }
+}
--- /dev/null	2021-11-09 09:21:02.662310924 +0000
+++ binutils/testsuite/binutils-all/nm.hex.unicode	2021-11-09 12:00:47.999841722 +0000
@@ -0,0 +1,3 @@
+#...
+.*he<0xe280ae>oll<0xe280ac>
+#pass
--- /dev/null	2021-11-09 09:21:02.662310924 +0000
+++ binutils/testsuite/binutils-all/objdump.highlight.unicode	2021-11-09 12:01:03.888739483 +0000
@@ -0,0 +1,3 @@
+#...
+.*he\\u202eoll\\u202c
+#pass
--- /dev/null	2021-11-09 09:21:02.662310924 +0000
+++ binutils/testsuite/binutils-all/strings.escape.unicode	2021-11-09 11:44:10.246262510 +0000
@@ -0,0 +1,3 @@
+#...
+he\\u202eoll\\u202c
+#pass
--- /dev/null	2021-11-09 09:21:02.662310924 +0000
+++ binutils/testsuite/binutils-all/readelf.invalid.unicode	2021-11-09 12:01:27.454587828 +0000
@@ -0,0 +1,3 @@
+#...
+.*he\{0xe280ae\}oll\{0xe280ac\}
+#pass

Re: Commit: Add support for displaying unicode characters

[Alan Modra]

At low optimisation levels gcc may warn.

	* strings.c (print_unicode_stream_body): Avoid bogus "may be
	used unitialised" warning.

diff --git a/binutils/strings.c b/binutils/strings.c
index f50badf983e..e8649a80d6a 100644
--- a/binutils/strings.c
+++ b/binutils/strings.c
@@ -924,7 +924,7 @@ print_unicode_stream_body (const char *     filename,
   uint num_read = 0;
   uint num_chars = 0;
   uint num_print = 0;
-  int c;
+  int c = 0;
 
   /* Find a series of string_min characters.  Put them into print_buf.  */
   do

-- 
Alan Modra
Australia Development Lab, IBM

Re: Commit: Add support for displaying unicode characters

[Nick Clifton]

Hi Alan,

> At low optimisation levels gcc may warn.
> 
> 	* strings.c (print_unicode_stream_body): Avoid bogus "may be
> 	used unitialised" warning.

Doh!  Thanks for fixing this.

Cheers
   Nick

Re: Commit: Add support for displaying unicode characters

[Fangrui Song]

On Tue, Nov 9, 2021 at 5:25 AM Nick Clifton via Binutils
<binutils@sourceware.org> wrote:
>
> Hi Guys,
>
>   I am applying the attached patch to add the ability to display unicode
>   characters to the nm, strings, objdump and readelf tools.  This is
>   part of a response to the Trojan Source vulnerability that was
>   recently disclosed.  I am also working on a separate patch for gas,
>   but that is not quite ready yet.
>
> Cheers
>   Nick
>
> binutils/ChangeLog
> 2021-11-09  Nick Clifton  <nickc@redhat.com>
>
>         * nm.c: Add --unicode option to control how unicode characters are
>         handled.
>         * objdump.c: Likewise.
>         * readelf.c: Likewise.
>         * strings.c: Likewise.
>         * binutils.texi: Document the new feature.
>         * NEWS: Document the new feature.
>         * testsuite/binutils-all/unicode.exp: New file.
>         * testsuite/binutils-all/nm.hex.unicode
>         * testsuite/binutils-all/strings.escape.unicode
>         * testsuite/binutils-all/objdump.highlight.unicode
>         * testsuite/binutils-all/readelf.invalid.unicode
>

I just learned that this patch added -U as an alias for --unicode. In
macOS nm and llvm-nm, -U is an alias for --defined-only.
--defined-only seems much more useful than --unicode. I wonder if it
is too late to redefine -U as --defined-only...

(On the llvm-nm side, I have filed
https://github.com/llvm/llvm-project/issues/55297 .
If it needs to keep compatibility with GNU nm, it will need to drop -U....)

Re: Commit: Add support for displaying unicode characters

[Nick Clifton]

Hi Fangrui,

> I just learned that this patch added -U as an alias for --unicode. In
> macOS nm and llvm-nm, -U is an alias for --defined-only.
> --defined-only seems much more useful than --unicode. I wonder if it
> is too late to redefine -U as --defined-only...

No, it could be changed.  I have to say that I would have thought that -d
was a better choice for the short version.  But anyway.

Before making the change however, I thought that it would be best to check
to see if there are any other inconsistencies and I did find a couple:

   * nm uses -s as the short version of --print-armap, whereas llvm-nm uses -M.
   * nm uses -V as the short version of --version, whereas llvm-nm does not have one.

The support/lack of -V does not strike me as important, and it should be a simple
matter to add -M as another short-version of --print-armap, so unless you have
any objections I will create a patch to change -U and add -M.

Cheers
   Nick

PS. I also saw that llvm-nm has a -W/--no-weak option which is missing from nm.
Is there a binutils PR filed to add this feature ?

Re: Commit: Add support for displaying unicode characters

[Fangrui Song]

Hi Nick,

Thanks for the reply and openness to make tools more compatible!
(even when llvm-nm made some coordination mistakes...)

On 2022-05-09, Nick Clifton wrote:
>Hi Fangrui,
>
>>I just learned that this patch added -U as an alias for --unicode. In
>>macOS nm and llvm-nm, -U is an alias for --defined-only.
>>--defined-only seems much more useful than --unicode. I wonder if it
>>is too late to redefine -U as --defined-only...
>
>No, it could be changed.  I have to say that I would have thought that -d
>was a better choice for the short version.  But anyway.
>
>Before making the change however, I thought that it would be best to check
>to see if there are any other inconsistencies and I did find a couple:
>
>  * nm uses -s as the short version of --print-armap, whereas llvm-nm uses -M.

-M seems more like a llvm-nm problem and I believe it is almost unused.
I think llvm-nm folks should think how to handle the -s conflict with
macOS. nm probably should not be bothered with this llvm-nm problem.
I think -M can be used for possibly more useful options in the future.

>  * nm uses -V as the short version of --version, whereas llvm-nm does not have one.

The latest version of llvm-nm has -V: https://llvm.org/docs/CommandGuide/llvm-nm.html
Having -V as an alias for --version seems a convention for quite a few
utilities.

>The support/lack of -V does not strike me as important, and it should be a simple
>matter to add -M as another short-version of --print-armap, so unless you have
>any objections I will create a patch to change -U and add -M.

Thanks! Having -U for --defined-only will be great.

>Cheers
>  Nick
>
>PS. I also saw that llvm-nm has a -W/--no-weak option which is missing from nm.
>Is there a binutils PR filed to add this feature ?

Thanks for noticing this.  I just filed one: https://sourceware.org/bugzilla/show_bug.cgi?id=29135 :)
Having -W/--no-weak will be great, too.