DWARF5 support for dwz

DWARF5 support for dwz

[Mark Wielaard]

Hi,

At the virtual GNU tools cauldron we had a little session about making
DWARF5 the default for GCC. GCC, binutils and gdb seem ready now, so
next up is DWZ. https://gcc.gnu.org/wiki/LPC2020NotesDWARF

Since nobody volunteered to add DWARF5 support I started
on some simple patches. The following 4 patches handle the easy parts.
Recognize some new DWARF5 .debug sections. Handle DWARF5 headers for
compile and partial units. Handle new DWARF5 attributes. And handle
some new DWARF5 operations as their GNU extension.

Next up is the harder part, parsing the new FORMs. DW_FORM_data16
should be easy. DW_FORM_line_strp is easy in principle, but might need
some thought to see how/if DIEs that reference the new .debug_line_str
can be moved (.debug_line_str itself cannot be moved to the supplemental
file so the string itself, and possibly .debug_line, should be rewritten
to point to the shared supplemental .debug_str section). The hard part
is DW_FORM_implicit_const which has its value in the abbrev, so just
having form plus .debug_info ptr isn't enough to read it. I think I have
a solution to that issue, but have to check how it interacts with the
hashing and writing out the new from.

There are a couple of forms that GCC only emits when generating split-DWARF
that I don't intend to implement righ now, DW_FORM_strx, DW_FORM_addrx,
DW_FORM_loclistx and DW_FORM_rnglistx.

Also there are standardized forms for GNU_ref_alt and GNU_strp_alt
(DW_FORM_ref_sup4, DW_FORM_ref_sup8 and DW_FORM_strp_sup) which I don't
want to use for now since they would require updates to consumers
(just like switching from .gnu_debugaltlink to .debug_sup). So I hold
those to the end (maybe we will introduce a new --dwarf5 flag for that?)

After the new FORMs patches are needed to read the new .debug_loclists
and .debug_rnglists sections. Then parse the .debug_line version 5 variant
(which gcc currently doesn't emit yet, but I have patches to make it
do so through binutils gas).

Finally we need to write out all the new data formats and possibly
optimize for the new forms.

Cheers,

Mark

[PATCH 1/4] Recognize some new DWARF5 .debug sections.

[Mark Wielaard]

Recognize .debug_loclists, .debug_rnglists, .debug_line_str. loclists,
rnglists and line_str cannot be moved into a supplemental file.

We already handle .debug_macro (the replacement for .debug_macinfo).
.debug_pubnames and .debug_pubtypes are no more, but we deleted them
already when found. There is a standardized variant of .gdb_index,
.debug_names, but gdb works fine with .gdb_index even for DWARF5, so
just keep supporting .gdb_index for now.

There are two new sections .debug_addr and .debug_str_offsets which
are only emitted in split-dwarf mode by GCC. For now we don't recognize
those because we don't handle split-dwarf.

Finally there is .debug_sup which is the standardized variant of
.gnu_debugaltlink. Which we should generate when operating in "full"
DWARF5 mode. But that would require changes to some consumers.

	* dwz.c (enum debug_section_kind): Add DEBUG_LOCLISTS,
	DEBUG_RNGLISTS and DEBUG_LINE_STR.
	(debug_sections): Add .debug_loclists, .debug_rnglists and
	.debug_line_str.
---
 dwz.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/dwz.c b/dwz.c
index 7ec0623..6a4c96c 100644
--- a/dwz.c
+++ b/dwz.c
@@ -687,8 +687,11 @@ enum debug_section_kind
   DEBUG_GNU_PUBTYPES,
   DEBUG_MACINFO,
   DEBUG_LOC,
+  DEBUG_LOCLISTS,
   DEBUG_FRAME,
   DEBUG_RANGES,
+  DEBUG_RNGLISTS,
+  DEBUG_LINE_STR,
   DEBUG_GDB_SCRIPTS,
   GDB_INDEX,
   GNU_DEBUGALTLINK,
@@ -720,8 +723,11 @@ static struct
     { ".debug_gnu_pubtypes", NULL, NULL, 0, 0, 0 },
     { ".debug_macinfo", NULL, NULL, 0, 0, 0 },
     { ".debug_loc", NULL, NULL, 0, 0, 0 },
+    { ".debug_loclists", NULL, NULL, 0, 0, 0 },
     { ".debug_frame", NULL, NULL, 0, 0, 0 },
     { ".debug_ranges", NULL, NULL, 0, 0, 0 },
+    { ".debug_rnglists", NULL, NULL, 0, 0, 0 },
+    { ".debug_line_str", NULL, NULL, 0, 0, 0 },
     { ".debug_gdb_scripts", NULL, NULL, 0, 0, 0 },
     { ".gdb_index", NULL, NULL, 0, 0, 0 },
     { ".gnu_debugaltlink", NULL, NULL, 0, 0, 0 },
-- 
2.18.4

[PATCH 2/4] Handle DWARF5 headers for compile and partial units.

[Mark Wielaard]

DWARF5 also has separate unit headers for types (replacing .debug_types)
and unit types for split-DWARF both a skeleton tree as compile and split
trees. Which aren't handled yet.

	* dwz.c (get_DW_UT_str): New function.
	(read_debug_line): Add .debug_line in version message error string.
	(try_debug_info): Parse cu_version 5 header for DW_UT_compile and
	DW_UT_partial. Add explicit error message for failing to read
	abbrev.
	(read_debug_info): Likewise.
---
 dwz.c | 146 +++++++++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 120 insertions(+), 26 deletions(-)

diff --git a/dwz.c b/dwz.c
index 6a4c96c..5d4b004 100644
--- a/dwz.c
+++ b/dwz.c
@@ -670,6 +670,28 @@ get_DW_AT_str (unsigned int at)
   return buf;
 }
 
+/* Retrun a DW_UT_* name.  */
+static const char *
+get_DW_UT_str (unsigned int ut)
+{
+  const char *name;
+  static char buf[7 + 3 * sizeof (int)];
+  switch (ut)
+    {
+    case DW_UT_compile: name = "DW_UT_compile"; break;
+    case DW_UT_type: name = "DW_UT_type"; break;
+    case DW_UT_partial: name = "DW_UT_partial"; break;
+    case DW_UT_skeleton: name = "DW_UT_partial"; break;
+    case DW_UT_split_compile: name = "DW_UT_split_compile"; break;
+    case DW_UT_split_type: name = "DW_UT_split_type"; break;
+    default: name = 0; break;
+    }
+  if (name)
+    return name;
+  sprintf (buf, "DW_UT_%u", ut);
+  return buf;
+}
+
 /* This must match the debug_sections array content
    below.  */
 enum debug_section_kind
@@ -1332,8 +1354,8 @@ read_debug_line (DSO *dso, dw_cu_ref cu, uint32_t off)
   value = read_16 (ptr);
   if (value < 2 || value > 4)
     {
-      error (0, 0, "%s: DWARF version %d unhandled", dso->filename,
-	     value);
+      error (0, 0, "%s: DWARF version %d in .debug_line unhandled",
+	     dso->filename, value);
       return 1;
     }
 
@@ -5572,24 +5594,41 @@ try_debug_info (DSO *dso)
 	}
 
       cu_version = read_16 (ptr);
-      if (cu_version < 2 || cu_version > 4)
+      if (kind == DEBUG_TYPES &&
+	  (cu_version < 2 || cu_version > 4))
 	{
-	  error (0, 0, "%s: DWARF version %d unhandled", dso->filename,
-		 cu_version);
+	  error (0, 0, "%s: DWARF version %d in .debug_types unhandled",
+		 dso->filename, cu_version);
 	  goto fail;
 	}
-
-      value = read_32 (ptr);
-      if (value >= debug_sections[DEBUG_ABBREV].size)
+      else if (cu_version < 2 || cu_version > 5)
 	{
-	  if (debug_sections[DEBUG_ABBREV].data == NULL)
-	    error (0, 0, "%s: .debug_abbrev not present", dso->filename);
-	  else
-	    error (0, 0, "%s: DWARF CU abbrev offset too large",
-		   dso->filename);
+	  error (0, 0, "%s: DWARF version %d in .debug_info unhandled",
+		 dso->filename, cu_version);
 	  goto fail;
 	}
 
+      if (cu_version == 5)
+	{
+	  value = read_8 (ptr);
+	  if (value != DW_UT_compile && value != DW_UT_partial)
+	    error (0, 0, "%s: DWARF CU type %s unhandled", dso->filename,
+		   get_DW_UT_str (value));
+	}
+      else
+	{
+	  value = read_32 (ptr);
+	  if (value >= debug_sections[DEBUG_ABBREV].size)
+	    {
+	      if (debug_sections[DEBUG_ABBREV].data == NULL)
+		error (0, 0, "%s: .debug_abbrev not present", dso->filename);
+	      else
+		error (0, 0, "%s: DWARF CU abbrev offset too large",
+		       dso->filename);
+	      goto fail;
+	    }
+	}
+
       if (ptr_size == 0)
 	{
 	  ptr_size = read_8 (ptr);
@@ -5607,6 +5646,20 @@ try_debug_info (DSO *dso)
 	  goto fail;
 	}
 
+      if (cu_version == 5)
+	{
+	  value = read_32 (ptr);
+	  if (value >= debug_sections[DEBUG_ABBREV].size)
+	    {
+	      if (debug_sections[DEBUG_ABBREV].data == NULL)
+		error (0, 0, "%s: .debug_abbrev not present", dso->filename);
+	      else
+		error (0, 0, "%s: DWARF CU abbrev offset too large",
+		       dso->filename);
+	      goto fail;
+	    }
+	}
+
       if (abbrev == NULL || value != last_abbrev_offset)
 	{
 	  if (abbrev)
@@ -5614,7 +5667,11 @@ try_debug_info (DSO *dso)
 	  abbrev
 	    = read_abbrev (dso, debug_sections[DEBUG_ABBREV].data + value);
 	  if (abbrev == NULL)
-	    goto fail;
+	    {
+	      error (0, 0, "%s: Couldn't read abbrev at offset 0x%x",
+		     dso->filename, value);
+	      goto fail;
+	    }
 	}
       last_abbrev_offset = value;
 
@@ -5800,24 +5857,42 @@ read_debug_info (DSO *dso, int kind, unsigned int *die_count)
 	}
 
       cu_version = read_16 (ptr);
-      if (cu_version < 2 || cu_version > 4)
+      if (kind == DEBUG_TYPES &&
+	  (cu_version < 2 || cu_version > 4))
 	{
-	  error (0, 0, "%s: DWARF version %d unhandled", dso->filename,
-		 cu_version);
+	  error (0, 0, "%s: DWARF version %d in .debug_types unhandled",
+		 dso->filename, cu_version);
 	  goto fail;
 	}
-
-      value = read_32 (ptr);
-      if (value >= debug_sections[DEBUG_ABBREV].size)
+      else if (cu_version < 2 || cu_version > 5)
 	{
-	  if (debug_sections[DEBUG_ABBREV].data == NULL)
-	    error (0, 0, "%s: .debug_abbrev not present", dso->filename);
-	  else
-	    error (0, 0, "%s: DWARF CU abbrev offset too large",
-		   dso->filename);
+	  error (0, 0, "%s: DWARF version %d in .debug_info unhandled",
+		 dso->filename, cu_version);
 	  goto fail;
 	}
 
+      if (cu_version == 5)
+	{
+	  value = read_8 (ptr);
+	  if (value != DW_UT_compile && value != DW_UT_partial)
+	    error (0, 0, "%s: DWARF CU type %s unhandled", dso->filename,
+		   get_DW_UT_str (value));
+
+	}
+      else
+	{
+	  value = read_32 (ptr);
+	  if (value >= debug_sections[DEBUG_ABBREV].size)
+	    {
+	      if (debug_sections[DEBUG_ABBREV].data == NULL)
+		error (0, 0, "%s: .debug_abbrev not present", dso->filename);
+	      else
+		error (0, 0, "%s: DWARF CU abbrev offset too large",
+		       dso->filename);
+	      goto fail;
+	    }
+	}
+
       if (ptr_size == 0)
 	{
 	  ptr_size = read_8 (ptr);
@@ -5835,6 +5910,20 @@ read_debug_info (DSO *dso, int kind, unsigned int *die_count)
 	  goto fail;
 	}
 
+      if (cu_version == 5)
+	{
+	  value = read_32 (ptr);
+	  if (value >= debug_sections[DEBUG_ABBREV].size)
+	    {
+	      if (debug_sections[DEBUG_ABBREV].data == NULL)
+		error (0, 0, "%s: .debug_abbrev not present", dso->filename);
+	      else
+		error (0, 0, "%s: DWARF CU abbrev offset too large",
+		       dso->filename);
+	      goto fail;
+	    }
+	}
+
       if (unlikely (op_multifile))
 	{
 	  if (ptr == endcu)
@@ -5914,7 +6003,12 @@ read_debug_info (DSO *dso, int kind, unsigned int *die_count)
 	  abbrev
 	    = read_abbrev (dso, debug_sections[DEBUG_ABBREV].data + value);
 	  if (abbrev == NULL)
-	    goto fail;
+	    {
+	      error (0, 0, "%s: Couldn't read abbrev at offset 0x%x",
+		     dso->filename, value);
+
+	      goto fail;
+	    }
 	}
       last_abbrev_offset = value;
 
-- 
2.18.4

[PATCH 3/4] Handle new DWARF5 attributes.

[Mark Wielaard]

There are various new attributes that can hold exprlocs (DW_AT_rank,
DW_AT_call_value, DW_AT_call_target, DW_AT_call_target_clobbered,
DW_AT_call_data_location, DW_AT_call_data_value) or hold addresses
(DW_AT_call_return_pc, DW_AT_call_pc).

Note that DW_AT_call_origin is listed as also holding exprlocs,
but that seems to be a bug in the spec:
http://dwarfstd.org/ShowIssue.php?issue=171103.1

There is a new DW_AT_macros that should work the same as DW_AT_GNU_macros.

Currently unhandled are DW_AT_dwo_name, DW_AT_str_offsets_base,
DW_AT_addr_base, DW_AT_rnglists_base and DW_AT_loclists_base which GCC
only emits when generating split-dwarf.

All other new attributes don't seem to need any special handling.

	* dwz.c (add_locexpr_dummy_dies): Handle DW_AT_rank,
	DW_AT_call_value, DW_AT_call_target, DW_AT_call_target_clobbered,
	DW_AT_call_data_location and DW_AT_call_data_value.
	(checksum_die): Handle DW_AT_call_return_pc, DW_AT_call_pc,
	DW_AT_macros, DW_AT_rank, DW_AT_call_value, DW_AT_call_target,
	DW_AT_call_target_clobbered, DW_AT_call_data_location,
	DW_AT_call_data_value, DW_AT_call_return_pc and DW_AT_call_pc.
	(die_eq_1): Handle DW_AT_call_return_pc and DW_AT_call_pc.
	(read_debug_info): Recognize all new DWARF5 attributes.
	(build_abbrevs_for_die): Deal with DW_AT_macros.
	(write_die): Likewise. And handle DW_AT_rank, DW_AT_call_value,
	DW_AT_call_target, DW_AT_call_target_clobbered,
	DW_AT_call_data_location and DW_AT_call_data_value.
---
 dwz.c | 45 ++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 38 insertions(+), 7 deletions(-)

diff --git a/dwz.c b/dwz.c
index 5d4b004..b497c5f 100644
--- a/dwz.c
+++ b/dwz.c
@@ -2370,6 +2370,7 @@ add_locexpr_dummy_dies (DSO *dso, dw_cu_ref cu, dw_die_ref die,
 {
   if (form == DW_FORM_block1)
     {
+      /* Old DWARF uses blocks instead of exprlocs.  */
       switch (attr)
 	{
 	case DW_AT_frame_base:
@@ -2392,6 +2393,12 @@ add_locexpr_dummy_dies (DSO *dso, dw_cu_ref cu, dw_die_ref die,
 	case DW_AT_associated:
 	case DW_AT_data_location:
 	case DW_AT_byte_stride:
+	case DW_AT_rank:
+	case DW_AT_call_value:
+	case DW_AT_call_target:
+	case DW_AT_call_target_clobbered:
+	case DW_AT_call_data_location:
+	case DW_AT_call_data_value:
 	case DW_AT_GNU_call_site_value:
 	case DW_AT_GNU_call_site_data_value:
 	case DW_AT_GNU_call_site_target:
@@ -2748,6 +2755,8 @@ checksum_die (DSO *dso, dw_cu_ref cu, dw_die_ref top_die, dw_die_ref die)
 	case DW_AT_high_pc:
 	case DW_AT_entry_pc:
 	case DW_AT_ranges:
+	case DW_AT_call_return_pc:
+	case DW_AT_call_pc:
 	  die->die_ck_state = CK_BAD;
 	  break;
 	case DW_AT_start_scope:
@@ -2758,6 +2767,7 @@ checksum_die (DSO *dso, dw_cu_ref cu, dw_die_ref top_die, dw_die_ref die)
 	   can't be moved to other files easily.  */
 	case DW_AT_stmt_list:
 	case DW_AT_macro_info:
+	case DW_AT_macros:
 	case DW_AT_GNU_macros:
 	  if (!die->die_root)
 	    die->die_no_multifile = 1;
@@ -3068,6 +3078,7 @@ checksum_die (DSO *dso, dw_cu_ref cu, dw_die_ref top_die, dw_die_ref die)
 
       if (form == DW_FORM_block1)
 	{
+	  /* Old DWARF uses blocks instead of exprlocs.  */
 	  switch (t->attr[i].attr)
 	    {
 	    case DW_AT_frame_base:
@@ -3090,6 +3101,12 @@ checksum_die (DSO *dso, dw_cu_ref cu, dw_die_ref top_die, dw_die_ref die)
 	    case DW_AT_associated:
 	    case DW_AT_data_location:
 	    case DW_AT_byte_stride:
+	    case DW_AT_rank:
+	    case DW_AT_call_value:
+	    case DW_AT_call_target:
+	    case DW_AT_call_target_clobbered:
+	    case DW_AT_call_data_location:
+	    case DW_AT_call_data_value:
 	    case DW_AT_GNU_call_site_value:
 	    case DW_AT_GNU_call_site_data_value:
 	    case DW_AT_GNU_call_site_target:
@@ -4020,6 +4037,10 @@ die_eq_1 (dw_cu_ref cu1, dw_cu_ref cu2,
 	case DW_AT_high_pc:
 	case DW_AT_entry_pc:
 	case DW_AT_ranges:
+	case DW_AT_call_return_pc:
+	case DW_AT_call_pc:
+	  /* We shouldn't be hitting DIEs with attributes referencing
+	     addresses and we should have removed DW_AT_subling.  */
 	  abort ();
 	case DW_AT_decl_file:
 	case DW_AT_call_file:
@@ -6298,7 +6319,7 @@ read_debug_info (DSO *dso, int kind, unsigned int *die_count)
 		      goto fail;
 		    }
 
-		  if (t->attr[i].attr > DW_AT_linkage_name
+		  if (t->attr[i].attr > DW_AT_loclists_base
 		      && (t->attr[i].attr < DW_AT_MIPS_fde
 			  || t->attr[i].attr > DW_AT_MIPS_has_inlines)
 		      && (t->attr[i].attr < DW_AT_sf_names
@@ -9731,7 +9752,8 @@ build_abbrevs_for_die (htab_t h, dw_cu_ref cu, dw_die_ref die,
 		}
 
 	      if (unlikely (fi_multifile)
-		  && reft->attr[i].attr == DW_AT_GNU_macros
+		  && (reft->attr[i].attr == DW_AT_GNU_macros
+		      || reft->attr[i].attr == DW_AT_macros)
 		  && alt_macro_htab != NULL)
 		{
 		  struct macro_entry me, *m;
@@ -9743,8 +9765,9 @@ build_abbrevs_for_die (htab_t h, dw_cu_ref cu, dw_die_ref die,
 		      value = read_32 (ptr);
 		      break;
 		    default:
-		      error (0, 0, "Unhandled %s for DW_AT_GNU_macros",
-			     get_DW_FORM_str (form));
+		      error (0, 0, "Unhandled %s for %s",
+			     get_DW_FORM_str (form),
+			     get_DW_AT_str (reft->attr[i].attr));
 		      return 1;
 		    }
 		  me.ptr = debug_sections[DEBUG_MACRO].data + value;
@@ -9752,8 +9775,8 @@ build_abbrevs_for_die (htab_t h, dw_cu_ref cu, dw_die_ref die,
 		    htab_find_with_hash (macro_htab, &me, value);
 		  if (m->len)
 		    {
-		      error (0, 0, "DW_AT_GNU_macros referencing "
-				   "transparent include");
+		      error (0, 0, "%s referencing transparent include",
+			     get_DW_AT_str (reft->attr[i].attr));
 		      return 1;
 		    }
 		  ptr -= 4;
@@ -11278,7 +11301,8 @@ write_die (unsigned char *ptr, dw_cu_ref cu, dw_die_ref die,
 	    }
 
 	  if (unlikely (fi_multifile)
-	      && reft->attr[i].attr == DW_AT_GNU_macros
+	      && (reft->attr[i].attr == DW_AT_GNU_macros
+		  || reft->attr[i].attr == DW_AT_macros)
 	      && alt_macro_htab != NULL)
 	    {
 	      struct macro_entry me, *m;
@@ -11539,6 +11563,7 @@ write_die (unsigned char *ptr, dw_cu_ref cu, dw_die_ref die,
 	  memcpy (ptr, orig_ptr, inptr - orig_ptr);
 	  ptr += inptr - orig_ptr;
 
+	  /* Old DWARF uses blocks instead of exprlocs.  */
 	  if (form == DW_FORM_block1)
 	    switch (reft->attr[i].attr)
 	      {
@@ -11562,6 +11587,12 @@ write_die (unsigned char *ptr, dw_cu_ref cu, dw_die_ref die,
 	      case DW_AT_associated:
 	      case DW_AT_data_location:
 	      case DW_AT_byte_stride:
+	      case DW_AT_rank:
+	      case DW_AT_call_value:
+	      case DW_AT_call_target:
+	      case DW_AT_call_target_clobbered:
+	      case DW_AT_call_data_location:
+	      case DW_AT_call_data_value:
 	      case DW_AT_GNU_call_site_value:
 	      case DW_AT_GNU_call_site_data_value:
 	      case DW_AT_GNU_call_site_target:
-- 
2.18.4

[PATCH 4/4] Handle new DWARF5 operations as their GNU extension variants.

[Mark Wielaard]

This handles DW_OP_implicit_pointer, DW_OP_entry_value,
DW_OP_const_type, DW_OP_regval_type, DW_OP_deref_type, DW_OP_convert,
DW_OP_reinterpret as the DW_OP_GNU extensions for pre-DWARF5.

This doesn't handle the new DW_OP_addrx and DW_OP_constx which are
used with split-dwarf .debug_addr tables. Nor does it implement
DW_OP_xderef_type which gcc will never emit.

	* dwz.c (read_exprloc): Handle DW_OP_implicit_pointer,
	DW_OP_entry_value, DW_OP_convert, DW_OP_reinterpret,
	DW_OP_regval_type, DW_OP_const_type and DW_OP_deref_type.
	(read_exprloc_low_mem_phase1): Likewise.
	(adjust_exprloc): Likewise.
---
 dwz.c | 39 ++++++++++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 9 deletions(-)

diff --git a/dwz.c b/dwz.c
index b497c5f..02e984e 100644
--- a/dwz.c
+++ b/dwz.c
@@ -1996,6 +1996,7 @@ read_exprloc (DSO *dso, dw_die_ref die, unsigned char *ptr, size_t len,
 	  break;
 	case DW_OP_call_ref:
 	case DW_OP_GNU_implicit_pointer:
+	case DW_OP_implicit_pointer:
 	case DW_OP_GNU_variable_value:
 	  cu = die_cu (die);
 	  addr = read_size (ptr, cu->cu_version == 2 ? ptr_size : 4);
@@ -2023,7 +2024,7 @@ read_exprloc (DSO *dso, dw_die_ref die, unsigned char *ptr, size_t len,
 	  die->die_ck_state = CK_BAD;
 	  if (need_adjust)
 	    *need_adjust = true;
-	  if (op == DW_OP_GNU_implicit_pointer)
+	  if (op == DW_OP_GNU_implicit_pointer || op == DW_OP_implicit_pointer)
 	    skip_leb128 (ptr);
 	  break;
 	case DW_OP_const8u:
@@ -2051,12 +2052,13 @@ read_exprloc (DSO *dso, dw_die_ref die, unsigned char *ptr, size_t len,
 	  }
 	  break;
 	case DW_OP_GNU_entry_value:
+	case DW_OP_entry_value:
 	  {
 	    uint32_t leni = read_uleb128 (ptr);
 	    if ((uint64_t) (end - ptr) < leni)
 	      {
-		error (0, 0, "%s: DWARF DW_OP_GNU_entry_value with too large"
-		       " length", dso->filename);
+		error (0, 0, "%s: %s with too large length",
+		       get_DW_OP_str (op),  dso->filename);
 		return 1;
 	      }
 	    if (read_exprloc (dso, die, ptr, leni, need_adjust))
@@ -2065,20 +2067,25 @@ read_exprloc (DSO *dso, dw_die_ref die, unsigned char *ptr, size_t len,
 	  }
 	  break;
 	case DW_OP_GNU_convert:
+	case DW_OP_convert:
 	case DW_OP_GNU_reinterpret:
+	case DW_OP_reinterpret:
 	  addr = read_uleb128 (ptr);
 	  if (addr == 0)
 	    break;
 	  goto typed_dwarf;
 	case DW_OP_GNU_regval_type:
+	case DW_OP_regval_type:
 	  skip_leb128 (ptr);
 	  addr = read_uleb128 (ptr);
 	  goto typed_dwarf;
 	case DW_OP_GNU_const_type:
+	case DW_OP_const_type:
 	  addr = read_uleb128 (ptr);
 	  ptr += *ptr + 1;
 	  goto typed_dwarf;
 	case DW_OP_GNU_deref_type:
+	case DW_OP_deref_type:
 	  ++ptr;
 	  addr = read_uleb128 (ptr);
 	typed_dwarf:
@@ -2231,6 +2238,7 @@ read_exprloc_low_mem_phase1 (DSO *dso, dw_die_ref die, unsigned char *ptr,
 	  break;
 	case DW_OP_call_ref:
 	case DW_OP_GNU_implicit_pointer:
+	case DW_OP_implicit_pointer:
 	case DW_OP_GNU_variable_value:
 	  cu = die_cu (die);
 	  addr = read_size (ptr, cu->cu_version == 2 ? ptr_size : 4);
@@ -2242,7 +2250,7 @@ read_exprloc_low_mem_phase1 (DSO *dso, dw_die_ref die, unsigned char *ptr,
 	     necessary if die_cu (ref) != cu, but we don't track cu's during
 	     low-mem phase1.  */
 	  add_dummy_die (cu, addr);
-	  if (op == DW_OP_GNU_implicit_pointer)
+	  if (op == DW_OP_GNU_implicit_pointer || op == DW_OP_implicit_pointer)
 	    skip_leb128 (ptr);
 	  break;
 	case DW_OP_const8u:
@@ -2270,12 +2278,13 @@ read_exprloc_low_mem_phase1 (DSO *dso, dw_die_ref die, unsigned char *ptr,
 	  }
 	  break;
 	case DW_OP_GNU_entry_value:
+	case DW_OP_entry_value:
 	  {
 	    uint32_t leni = read_uleb128 (ptr);
 	    if ((uint64_t) (end - ptr) < leni)
 	      {
-		error (0, 0, "%s: DWARF DW_OP_GNU_entry_value with too large"
-		       " length", dso->filename);
+		error (0, 0, "%s: %s with too large length",
+		       get_DW_OP_str (op), dso->filename);
 		return 1;
 	      }
 	    if (read_exprloc_low_mem_phase1 (dso, die, ptr, leni))
@@ -2284,18 +2293,23 @@ read_exprloc_low_mem_phase1 (DSO *dso, dw_die_ref die, unsigned char *ptr,
 	  }
 	  break;
 	case DW_OP_GNU_convert:
+	case DW_OP_convert:
 	case DW_OP_GNU_reinterpret:
+	case DW_OP_reinterpret:
 	  skip_leb128 (ptr);
 	  break;
 	case DW_OP_GNU_regval_type:
+	case DW_OP_regval_type:
 	  skip_leb128 (ptr);
 	  skip_leb128 (ptr);
 	  break;
 	case DW_OP_GNU_const_type:
-	  skip_leb128 (ptr);
+	case DW_OP_const_type:
+	  read_uleb128 (ptr);
 	  ptr += *ptr + 1;
 	  break;
 	case DW_OP_GNU_deref_type:
+	case DW_OP_deref_type:
 	  ++ptr;
 	  skip_leb128 (ptr);
 	  break;
@@ -11042,6 +11056,7 @@ adjust_exprloc (dw_cu_ref cu, dw_die_ref die, dw_cu_ref refcu,
 	  break;
 	case DW_OP_call_ref:
 	case DW_OP_GNU_implicit_pointer:
+	case DW_OP_implicit_pointer:
 	case DW_OP_GNU_variable_value:
 	  addr = read_size (ptr, refcu->cu_version == 2 ? ptr_size : 4);
 	  assert (cu->cu_version == refcu->cu_version);
@@ -11059,7 +11074,7 @@ adjust_exprloc (dw_cu_ref cu, dw_die_ref die, dw_cu_ref refcu,
 	    ptr += ptr_size;
 	  else
 	    ptr += 4;
-	  if (op == DW_OP_GNU_implicit_pointer)
+	  if (op == DW_OP_GNU_implicit_pointer || op == DW_OP_implicit_pointer)
 	    skip_leb128 (ptr);
 	  break;
 	case DW_OP_const8u:
@@ -11085,28 +11100,34 @@ adjust_exprloc (dw_cu_ref cu, dw_die_ref die, dw_cu_ref refcu,
 	  ptr += leni;
 	  break;
 	case DW_OP_GNU_entry_value:
+	case DW_OP_entry_value:
 	  leni = read_uleb128 (ptr);
 	  assert ((uint64_t) (end - ptr) >= leni);
 	  adjust_exprloc (cu, die, refcu, ref, ptr, leni);
 	  ptr += leni;
 	  break;
 	case DW_OP_GNU_convert:
+	case DW_OP_convert:
 	case DW_OP_GNU_reinterpret:
+	case DW_OP_reinterpret:
 	  orig_ptr = ptr;
 	  addr = read_uleb128 (ptr);
 	  if (addr == 0)
 	    break;
 	  goto typed_dwarf;
 	case DW_OP_GNU_regval_type:
+	case DW_OP_regval_type:
 	  skip_leb128 (ptr);
 	  orig_ptr = ptr;
 	  addr = read_uleb128 (ptr);
 	  goto typed_dwarf;
 	case DW_OP_GNU_const_type:
+	case DW_OP_const_type:
 	  orig_ptr = ptr;
 	  addr = read_uleb128 (ptr);
 	  goto typed_dwarf;
 	case DW_OP_GNU_deref_type:
+	case DW_OP_deref_type:
 	  ++ptr;
 	  orig_ptr = ptr;
 	  addr = read_uleb128 (ptr);
@@ -11129,7 +11150,7 @@ adjust_exprloc (dw_cu_ref cu, dw_die_ref die, dw_cu_ref refcu,
 		*ptr++ = 0x80;
 	      *ptr++ = 0;
 	    }
-	  if (op == DW_OP_GNU_const_type)
+	  if (op == DW_OP_GNU_const_type || op == DW_OP_const_type)
 	    ptr += *ptr + 1;
 	  break;
 	default:
-- 
2.18.4

Re: [PATCH 1/4] Recognize some new DWARF5 .debug sections.

[Jakub Jelinek]

On Mon, Sep 14, 2020 at 12:23:52PM +0200, Mark Wielaard wrote:
> Recognize .debug_loclists, .debug_rnglists, .debug_line_str. loclists,
> rnglists and line_str cannot be moved into a supplemental file.
> 
> We already handle .debug_macro (the replacement for .debug_macinfo).
> .debug_pubnames and .debug_pubtypes are no more, but we deleted them
> already when found. There is a standardized variant of .gdb_index,
> .debug_names, but gdb works fine with .gdb_index even for DWARF5, so
> just keep supporting .gdb_index for now.
> 
> There are two new sections .debug_addr and .debug_str_offsets which
> are only emitted in split-dwarf mode by GCC. For now we don't recognize
> those because we don't handle split-dwarf.
> 
> Finally there is .debug_sup which is the standardized variant of
> .gnu_debugaltlink. Which we should generate when operating in "full"
> DWARF5 mode. But that would require changes to some consumers.
> 
> 	* dwz.c (enum debug_section_kind): Add DEBUG_LOCLISTS,
> 	DEBUG_RNGLISTS and DEBUG_LINE_STR.
> 	(debug_sections): Add .debug_loclists, .debug_rnglists and
> 	.debug_line_str.

LGTM, thanks.

	Jakub

Re: [PATCH 2/4] Handle DWARF5 headers for compile and partial units.

[Jakub Jelinek]

On Mon, Sep 14, 2020 at 12:23:53PM +0200, Mark Wielaard wrote:
> DWARF5 also has separate unit headers for types (replacing .debug_types)
> and unit types for split-DWARF both a skeleton tree as compile and split
> trees. Which aren't handled yet.
> 
> 	* dwz.c (get_DW_UT_str): New function.
> 	(read_debug_line): Add .debug_line in version message error string.
> 	(try_debug_info): Parse cu_version 5 header for DW_UT_compile and
> 	DW_UT_partial. Add explicit error message for failing to read
> 	abbrev.
> 	(read_debug_info): Likewise.
> ---
>  dwz.c | 146 +++++++++++++++++++++++++++++++++++++++++++++++-----------
>  1 file changed, 120 insertions(+), 26 deletions(-)
> 
> diff --git a/dwz.c b/dwz.c
> index 6a4c96c..5d4b004 100644
> --- a/dwz.c
> +++ b/dwz.c
> @@ -670,6 +670,28 @@ get_DW_AT_str (unsigned int at)
>    return buf;
>  }
>  
> +/* Retrun a DW_UT_* name.  */
> +static const char *
> +get_DW_UT_str (unsigned int ut)
> +{
> +  const char *name;
> +  static char buf[7 + 3 * sizeof (int)];
> +  switch (ut)
> +    {
> +    case DW_UT_compile: name = "DW_UT_compile"; break;
> +    case DW_UT_type: name = "DW_UT_type"; break;
> +    case DW_UT_partial: name = "DW_UT_partial"; break;
> +    case DW_UT_skeleton: name = "DW_UT_partial"; break;

Pasto?

> +    case DW_UT_split_compile: name = "DW_UT_split_compile"; break;
> +    case DW_UT_split_type: name = "DW_UT_split_type"; break;
> +    default: name = 0; break;
> +    }

> @@ -5572,24 +5594,41 @@ try_debug_info (DSO *dso)
>  	}
>  
>        cu_version = read_16 (ptr);
> -      if (cu_version < 2 || cu_version > 4)
> +      if (kind == DEBUG_TYPES &&
> +	  (cu_version < 2 || cu_version > 4))

I think dwz follows e.g. the GCC formatting and &&s etc. go
to the start of line rather than end of it.

Ok with the nits fixed.

	Jakub

Re: [PATCH 3/4] Handle new DWARF5 attributes.

[Jakub Jelinek]

On Mon, Sep 14, 2020 at 12:23:54PM +0200, Mark Wielaard wrote:
> There are various new attributes that can hold exprlocs (DW_AT_rank,
> DW_AT_call_value, DW_AT_call_target, DW_AT_call_target_clobbered,
> DW_AT_call_data_location, DW_AT_call_data_value) or hold addresses
> (DW_AT_call_return_pc, DW_AT_call_pc).
> 
> Note that DW_AT_call_origin is listed as also holding exprlocs,
> but that seems to be a bug in the spec:
> http://dwarfstd.org/ShowIssue.php?issue=171103.1
> 
> There is a new DW_AT_macros that should work the same as DW_AT_GNU_macros.
> 
> Currently unhandled are DW_AT_dwo_name, DW_AT_str_offsets_base,
> DW_AT_addr_base, DW_AT_rnglists_base and DW_AT_loclists_base which GCC
> only emits when generating split-dwarf.
> 
> All other new attributes don't seem to need any special handling.
> 
> 	* dwz.c (add_locexpr_dummy_dies): Handle DW_AT_rank,
> 	DW_AT_call_value, DW_AT_call_target, DW_AT_call_target_clobbered,
> 	DW_AT_call_data_location and DW_AT_call_data_value.
> 	(checksum_die): Handle DW_AT_call_return_pc, DW_AT_call_pc,
> 	DW_AT_macros, DW_AT_rank, DW_AT_call_value, DW_AT_call_target,
> 	DW_AT_call_target_clobbered, DW_AT_call_data_location,
> 	DW_AT_call_data_value, DW_AT_call_return_pc and DW_AT_call_pc.
> 	(die_eq_1): Handle DW_AT_call_return_pc and DW_AT_call_pc.
> 	(read_debug_info): Recognize all new DWARF5 attributes.
> 	(build_abbrevs_for_die): Deal with DW_AT_macros.
> 	(write_die): Likewise. And handle DW_AT_rank, DW_AT_call_value,
> 	DW_AT_call_target, DW_AT_call_target_clobbered,
> 	DW_AT_call_data_location and DW_AT_call_data_value.

LGTM.

	Jakub

Re: [PATCH 4/4] Handle new DWARF5 operations as their GNU extension variants.

[Jakub Jelinek]

On Mon, Sep 14, 2020 at 12:23:55PM +0200, Mark Wielaard wrote:
> This handles DW_OP_implicit_pointer, DW_OP_entry_value,
> DW_OP_const_type, DW_OP_regval_type, DW_OP_deref_type, DW_OP_convert,
> DW_OP_reinterpret as the DW_OP_GNU extensions for pre-DWARF5.
> 
> This doesn't handle the new DW_OP_addrx and DW_OP_constx which are
> used with split-dwarf .debug_addr tables. Nor does it implement
> DW_OP_xderef_type which gcc will never emit.
> 
> 	* dwz.c (read_exprloc): Handle DW_OP_implicit_pointer,
> 	DW_OP_entry_value, DW_OP_convert, DW_OP_reinterpret,
> 	DW_OP_regval_type, DW_OP_const_type and DW_OP_deref_type.
> 	(read_exprloc_low_mem_phase1): Likewise.
> 	(adjust_exprloc): Likewise.

Ok.

	Jakub

Re: DWARF5 support for dwz

[Jakub Jelinek]

On Mon, Sep 14, 2020 at 12:23:51PM +0200, Mark Wielaard wrote:
> Since nobody volunteered to add DWARF5 support I started
> on some simple patches. The following 4 patches handle the easy parts.
> Recognize some new DWARF5 .debug sections. Handle DWARF5 headers for

Thanks for working on this.

> compile and partial units. Handle new DWARF5 attributes. And handle
> some new DWARF5 operations as their GNU extension.
> 
> Next up is the harder part, parsing the new FORMs. DW_FORM_data16
> should be easy. DW_FORM_line_strp is easy in principle, but might need
> some thought to see how/if DIEs that reference the new .debug_line_str
> can be moved (.debug_line_str itself cannot be moved to the supplemental
> file so the string itself, and possibly .debug_line, should be rewritten
> to point to the shared supplemental .debug_str section). The hard part
> is DW_FORM_implicit_const which has its value in the abbrev, so just
> having form plus .debug_info ptr isn't enough to read it. I think I have
> a solution to that issue, but have to check how it interacts with the
> hashing and writing out the new from.

For DW_FORM_implicit_const, handling it when reading should be similar
to how DW_FORM_flag_present is handled already.

The harded will be the optimization part, figure out when it is worth to
split an abbreviation to two or more in order to introduce more
DW_FORM_implicit_const forms to make it an overall gain.
On the negative side that means growth in .debug_abbrev section and some
growth in the uleb128s referencing the new abbreviations, so it would need
to do it smartly and only when beneficial.

> Also there are standardized forms for GNU_ref_alt and GNU_strp_alt
> (DW_FORM_ref_sup4, DW_FORM_ref_sup8 and DW_FORM_strp_sup) which I don't
> want to use for now since they would require updates to consumers
> (just like switching from .gnu_debugaltlink to .debug_sup). So I hold
> those to the end (maybe we will introduce a new --dwarf5 flag for that?)

Yeah, some switch at least for the transition period would be worthwhile.
Because the consumers can't be really adjusted to handle it until dwz is
able to emit that, and dwz can't switch to doing it by default until
at least the majority of consumers are adjusted.

My memory is fuzzy, don't we need some adjustments to the expected gain
computations with dwarf5 because the section headers will have now different
sizes?

	Jakub

Re: [PATCH 2/4] Handle DWARF5 headers for compile and partial units.

[Mark Wielaard]

Hi,

On Mon, 2020-09-14 at 12:38 +0200, Jakub Jelinek wrote:
> On Mon, Sep 14, 2020 at 12:23:53PM +0200, Mark Wielaard wrote:
> > +    case DW_UT_compile: name = "DW_UT_compile"; break;
> > +    case DW_UT_type: name = "DW_UT_type"; break;
> > +    case DW_UT_partial: name = "DW_UT_partial"; break;
> > +    case DW_UT_skeleton: name = "DW_UT_partial"; break;
> 
> Pasto?

Yes, fixed.

> > +    case DW_UT_split_compile: name = "DW_UT_split_compile"; break;
> > +    case DW_UT_split_type: name = "DW_UT_split_type"; break;
> > +    default: name = 0; break;
> > +    }
> > @@ -5572,24 +5594,41 @@ try_debug_info (DSO *dso)
> >  	}
> >  
> >        cu_version = read_16 (ptr);
> > -      if (cu_version < 2 || cu_version > 4)
> > +      if (kind == DEBUG_TYPES &&
> > +	  (cu_version < 2 || cu_version > 4))
> 
> I think dwz follows e.g. the GCC formatting and &&s etc. go
> to the start of line rather than end of it.

No idea why I did that. The line isn't too large, so I just kept it all
together (in both places).

> Ok with the nits fixed.

Thanks, pushed,

Mark