public inbox for libabigail@sourceware.org
 help / color / mirror / Atom feed
* [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support
@ 2022-03-14 18:13 Giuliano Procida
  2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida
                   ` (2 more replies)
  0 siblings, 3 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Hi Dodji.

These two patches aim to add kernel symbol namespace support. There
are two bits of this that could be debated (and easily changed).

A. choice of XML attribute name

I've chosen "ns", mostly because "namespace" implies C++ namespace
everywhere else.

B. choice of representation

There are several possible levels of distinction of presence:

1. Non-kernel symbols cannot be exported to a namespace.
2. Kernel symbols can be exported globally and not to namespace.
3. Kernel symbols can be exported to a named namespace.

And there is the option of treating the empty namespace name string
specially. I've chosen to represent namespace internally as
optional<string> and not distinguish between cases 1 and 2.

Regards.

Giuliano Procida (2):
  optional: add operator== and operator!=
  add Linux kernel symbol namespace support

 include/abg-cxx-compat.h | 16 +++++++++++++
 include/abg-ir.h         |  9 ++++++++
 src/abg-comp-filter.cc   | 39 +++++++++++++++++++++++++++++++-
 src/abg-ir.cc            | 27 +++++++++++++++++++++-
 src/abg-reader.cc        |  7 ++++++
 src/abg-reporter-priv.cc | 12 ++++++++++
 src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++
 src/abg-writer.cc        |  4 ++++
 8 files changed, 161 insertions(+), 2 deletions(-)

-- 
2.35.1.723.g4982287a31-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH 1/2] optional: add operator== and operator!=
  2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida
@ 2022-03-14 18:13 ` Giuliano Procida
  2022-03-15 11:02   ` Matthias Maennich
  2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

	* include/abg-cxx-compat.h (optional): Add operator== and
	operator!=.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-cxx-compat.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
index 443905c7..a2cf9095 100644
--- a/include/abg-cxx-compat.h
+++ b/include/abg-cxx-compat.h
@@ -91,6 +91,22 @@ public:
   }
 
   explicit operator bool() const { return has_value_; }
+
+  bool
+  operator==(const optional<T>& other) const
+  {
+    if (!has_value_ && !other.has_value_)
+      return true;
+    if (!has_value_ || !other.has_value_)
+      return false;
+    return value_ == other.value_;
+  }
+
+  bool
+  operator!=(const optional<T>& other) const
+  {
+    return !operator==(other);
+  }
 };
 
 #endif
-- 
2.35.1.723.g4982287a31-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH 2/2] add Linux kernel symbol namespace support
  2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida
  2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida
@ 2022-03-14 18:13 ` Giuliano Procida
  2022-03-15 11:25   ` Matthias Maennich
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Bug 28954 - add Linux Kernel symbol namespace support

This commit adds Linux Kernel symbol namespace support roughly as
discussed.

Each symbol can be exported to a specified named namespace or left in
the global (nameless) namespace. The latter is explicitly represented
as the empty string, at least when it comes to the value of
__kstrtabns_FOO symbols, but the common interpretation is that such
symbols are not exported to a namespace.

I would rather not make this assumption in more than one place in the
code and would also prefer to protect against __kstrtabns_FOO for
globally export FOO disappearing with a future change to the export
macros.

So the code here represents namespace as optional<string> and treats
empty strings found in __ksymtab_strings as missing.

	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
	(elf_symbol::create): Add ns argument.
	(elf_symbol::get_namespace): Declare new function.
	(elf_symbol::set_namespace): Declare new function.
	and set_namespace.
	* src/abg-comp-filter.cc (namespace_changed): Define new
	helper functions.
	(categorize_harmful_diff_node): Also call namespace_changed().
	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
	(elf_symbol::priv::priv): Add namespace_ to initialisers.
	(elf_symbol::elf_symbol): Take new ns argument and pass it to
	priv constructor.
	(elf_symbol::create): Take new ns argument and pass it to
	elf_symbol constructor.
	(elf_symbol::get_namespace): Define new function.
	(elf_symbol::set_namespace): Define new function.
	* src/abg-reader.cc (build_elf_symbol): If ns attribute is
	present, set symbol namespace.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
	symbol namespaces differ, report this.
	* src/abg-symtab-reader.cc (symtab::load): Try to get
	__ksymtab_strings metadata and data. Use this to look up
	__kstrtabns_FOO namespace entries. Set symbol namespaces where
	found.
	* src/abg-writer.cc (write_elf_symbol): Emit ns attribute, if
	symbol has a namespace.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h         |  9 ++++++++
 src/abg-comp-filter.cc   | 39 +++++++++++++++++++++++++++++++-
 src/abg-ir.cc            | 27 +++++++++++++++++++++-
 src/abg-reader.cc        |  7 ++++++
 src/abg-reporter-priv.cc | 12 ++++++++++
 src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++
 src/abg-writer.cc        |  4 ++++
 7 files changed, 145 insertions(+), 2 deletions(-)

diff --git a/include/abg-ir.h b/include/abg-ir.h
index a2f4e1a7..7c42bea7 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -21,6 +21,7 @@
 #include <functional>
 #include <set>
 #include <unordered_map>
+#include "abg-cxx-compat.h"
 #include "abg-fwd.h"
 #include "abg-hash.h"
 #include "abg-traverse.h"
@@ -921,6 +922,7 @@ private:
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
 	     uint64_t		crc = 0,
+	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -946,6 +948,7 @@ public:
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
 	 uint64_t	    crc = 0,
+	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1023,6 +1026,12 @@ public:
   void
   set_crc(uint64_t crc);
 
+  const abg_compat::optional<std::string>&
+  get_namespace() const;
+
+  void
+  set_namespace(const abg_compat::optional<std::string>& ns);
+
   bool
   is_suppressed() const;
 
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 56251274..6ee31e35 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -254,6 +254,42 @@ crc_changed(const diff* diff)
   return false;
 }
 
+/// Test if there was a function or variable namespace change.
+///
+/// @param f the first function or variable to consider.
+///
+/// @param s the second function or variable to consider.
+///
+/// @return true if the test is positive, false otherwise.
+template <typename function_or_var_decl_sptr>
+static bool
+namespace_changed(const function_or_var_decl_sptr& f,
+		  const function_or_var_decl_sptr& s)
+{
+  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
+  if (!symbol_f || !symbol_s)
+    return false;
+  return symbol_f->get_namespace() != symbol_s->get_namespace();
+}
+
+/// test if the current diff tree node carries a namespace change in
+/// either a function or a variable.
+///
+/// @param diff the diff tree node to consider.
+///
+/// @return true if the test is positive, false otherwise.
+static bool
+namespace_changed(const diff* diff)
+{
+  if (const function_decl_diff* d =
+	dynamic_cast<const function_decl_diff*>(diff))
+    return namespace_changed(d->first_function_decl(),
+			     d->second_function_decl());
+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
+    return namespace_changed(d->first_var(), d->second_var());
+  return false;
+}
+
 /// Test if there was a function name change, but there there was no
 /// change in name of the underlying symbol.  IOW, if the name of a
 /// function changed, but the symbol of the new function is equal to
@@ -1781,7 +1817,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
 	      || non_static_data_member_added_or_removed(d)
 	      || base_classes_added_or_removed(d)
 	      || has_harmful_enum_change(d)
-	      || crc_changed(d)))
+	      || crc_changed(d)
+	      || namespace_changed(d)))
 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
 
       if (has_virtual_mem_fn_change(d))
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 0ef5e8b2..c510bc55 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
   bool			is_common_;
   bool			is_in_ksymtab_;
   uint64_t		crc_;
+  abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
       is_common_(false),
       is_in_ksymtab_(false),
       crc_(0),
+      namespace_(),
       is_suppressed_(false)
   {}
 
@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
        uint64_t			  crc,
+       const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
       is_common_(c),
       is_in_ksymtab_(is_in_ksymtab),
       crc_(crc),
+      namespace_(ns),
       is_suppressed_(is_suppressed)
   {
     if (!is_common_)
@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
 /// @param vi the visibility of the symbol.
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
+///
+/// @param ns the namespace of Linux Kernel symbols, if any
 elf_symbol::elf_symbol(const environment* e,
 		       size_t		  i,
 		       size_t		  s,
@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
 		       uint64_t		  crc,
+		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
 		   vi,
 		   is_in_ksymtab,
 		   crc,
+		   ns,
 		   is_suppressed))
 {}
 
@@ -1886,6 +1894,8 @@ elf_symbol::create()
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
 ///
+/// @param ns the namespace of Linux Kernel symbols, if any
+///
 /// @return a (smart) pointer to a newly created instance of @ref
 /// elf_symbol.
 elf_symbol_sptr
@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
 		   uint64_t	      crc,
+		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
-				     is_in_ksymtab, crc, is_suppressed));
+				     is_in_ksymtab, crc, ns, is_suppressed));
   sym->priv_->main_symbol_ = sym;
   return sym;
 }
@@ -2147,6 +2158,20 @@ void
 elf_symbol::set_crc(uint64_t crc)
 {priv_->crc_ = crc;}
 
+/// Getter of the 'namespace' property.
+///
+/// @return the namespace for Linux Kernel symbols, if any
+const abg_compat::optional<std::string>&
+elf_symbol::get_namespace() const
+{return priv_->namespace_;}
+
+/// Setter of the 'namespace' property.
+///
+/// @param ns the new namespace for Linux Kernel symbols, if any
+void
+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
+{priv_->namespace_ = ns;}
+
 /// Getter for the 'is-suppressed' property.
 ///
 /// @return true iff the current symbol has been suppressed by a
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 31885692..55b7348d 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
   if (crc != 0)
     e->set_crc(crc);
 
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "ns"))
+    {
+      std::string ns;
+      xml::xml_char_sptr_to_string(s, ns);
+      e->set_namespace({ns});
+    }
+
   return e;
 }
 
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 7012f5dc..dd40690c 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1158,6 +1158,18 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << std::noshowbase << std::dec
 	  << "\n";
     }
+
+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
+  if (ns1 != ns2)
+    {
+      const std::string none = "(none)";
+      out << indent << "namespace changed from "
+	  << (ns1.has_value() ? ns1.value() : none)
+	  << " to "
+	  << (ns2.has_value() ? ns2.value() : none)
+	  << "\n";
+    }
 }
 
 /// For a given symbol, emit a string made of its name and version.
diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
index b42ce87d..589f3703 100644
--- a/src/abg-symtab-reader.cc
+++ b/src/abg-symtab-reader.cc
@@ -232,9 +232,33 @@ symtab::load_(Elf*	       elf_handle,
       return false;
     }
 
+  // The __kstrtab_strings sections is basically an ELF strtab but does not
+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
+  // section data.
+  //
+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
+  // within the __kstrtab_strings section. To look up the string value, we need
+  // to translate from load address to section offset by subtracting the base
+  // address of the section.
+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
+  size_t strings_offset = 0;
+  const char* strings_data = nullptr;
+  size_t strings_size = 0;
+  if (strings_section)
+    {
+      GElf_Shdr strings_sheader;
+      gelf_getshdr(strings_section, &strings_sheader);
+      strings_offset = strings_sheader.sh_addr;
+      Elf_Data* data = elf_getdata(strings_section, nullptr);
+      ABG_ASSERT(data->d_off == 0);
+      strings_data = reinterpret_cast<const char *>(data->d_buf);
+      strings_size = data->d_size;
+    }
+
   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
   std::unordered_set<std::string> exported_kernel_symbols;
   std::unordered_map<std::string, uint64_t> crc_values;
+  std::unordered_map<std::string, std::string> namespaces;
 
   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
@@ -288,6 +312,20 @@ symtab::load_(Elf*	       elf_handle,
 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
 	  continue;
 	}
+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
+	{
+	  // This symbol lives in the __ksymtab_strings section but st_value is
+	  // a load address so we need to subtract an offset before looking it
+	  // up in that section.
+	  const size_t offset = sym->st_value - strings_offset;
+	  if (offset < strings_size)
+	    {
+	      const char* ns = strings_data + offset;
+	      if (*ns)
+		ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);
+	    }
+	  continue;
+	}
 
       // filter out uninteresting entries and only keep functions/variables for
       // now. The rest might be interesting in the future though.
@@ -402,6 +440,17 @@ symtab::load_(Elf*	       elf_handle,
 	symbol->set_crc(crc_entry.second);
     }
 
+  // Now add the namespaces
+  for (const auto& namespace_entry : namespaces)
+    {
+      const auto r = name_symbol_map_.find(namespace_entry.first);
+      if (r == name_symbol_map_.end())
+	continue;
+
+      for (const auto& symbol : r->second)
+	symbol->set_namespace({namespace_entry.second});
+    }
+
   // sort the symbols for deterministic output
   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 7802128d..4dfa72a6 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3136,6 +3136,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
       << std::hex << std::showbase << sym->get_crc() << "'"
       << std::dec << std::noshowbase;
 
+  if (sym->get_namespace().has_value())
+    o << " ns='"
+      << sym->get_namespace().value() << "'";
+
   o << "/>\n";
 
   return true;
-- 
2.35.1.723.g4982287a31-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH 1/2] optional: add operator== and operator!=
  2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida
@ 2022-03-15 11:02   ` Matthias Maennich
  2022-03-16  9:31     ` Giuliano Procida
  0 siblings, 1 reply; 37+ messages in thread
From: Matthias Maennich @ 2022-03-15 11:02 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Mon, Mar 14, 2022 at 06:13:11PM +0000, Giuliano Procida wrote:
>	* include/abg-cxx-compat.h (optional): Add operator== and
>	operator!=.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>
>---
> include/abg-cxx-compat.h | 16 ++++++++++++++++
> 1 file changed, 16 insertions(+)
>
>diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
>index 443905c7..a2cf9095 100644
>--- a/include/abg-cxx-compat.h
>+++ b/include/abg-cxx-compat.h
>@@ -91,6 +91,22 @@ public:
>   }
>
>   explicit operator bool() const { return has_value_; }

     std signature is (at least make it noexcept):
       > constexpr explicit operator bool() const noexcept;

>+
>+  bool
>+  operator==(const optional<T>& other) const
>+  {
>+    if (!has_value_ && !other.has_value_)
>+      return true;
>+    if (!has_value_ || !other.has_value_)
>+      return false;
>+    return value_ == other.value_;
>+  }
>+
>+  bool
>+  operator!=(const optional<T>& other) const
>+  {
>+    return !operator==(other);
>+  }

I believe C++17 defines those as non-member functions. Should we follow
that for compatibility. Eventually, when switching to C++17 or later we
would want to just replace abg_compat:: with std:: and things should
"just work".

Cheers,
Matthias

> };
>
> #endif
>-- 
>2.35.1.723.g4982287a31-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH 2/2] add Linux kernel symbol namespace support
  2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-15 11:25   ` Matthias Maennich
  0 siblings, 0 replies; 37+ messages in thread
From: Matthias Maennich @ 2022-03-15 11:25 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Mon, Mar 14, 2022 at 06:13:12PM +0000, Giuliano Procida wrote:
>Bug 28954 - add Linux Kernel symbol namespace support
>
>This commit adds Linux Kernel symbol namespace support roughly as
>discussed.
>
>Each symbol can be exported to a specified named namespace or left in
>the global (nameless) namespace. The latter is explicitly represented
>as the empty string, at least when it comes to the value of
>__kstrtabns_FOO symbols, but the common interpretation is that such
>symbols are not exported to a namespace.
>
>I would rather not make this assumption in more than one place in the
>code and would also prefer to protect against __kstrtabns_FOO for
>globally export FOO disappearing with a future change to the export
>macros.
>
>So the code here represents namespace as optional<string> and treats
>empty strings found in __ksymtab_strings as missing.
>
>	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
>	(elf_symbol::create): Add ns argument.
>	(elf_symbol::get_namespace): Declare new function.
>	(elf_symbol::set_namespace): Declare new function.
>	and set_namespace.
>	* src/abg-comp-filter.cc (namespace_changed): Define new
>	helper functions.
>	(categorize_harmful_diff_node): Also call namespace_changed().
>	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
>	(elf_symbol::priv::priv): Add namespace_ to initialisers.
>	(elf_symbol::elf_symbol): Take new ns argument and pass it to
>	priv constructor.
>	(elf_symbol::create): Take new ns argument and pass it to
>	elf_symbol constructor.
>	(elf_symbol::get_namespace): Define new function.
>	(elf_symbol::set_namespace): Define new function.
>	* src/abg-reader.cc (build_elf_symbol): If ns attribute is
>	present, set symbol namespace.
>	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
>	symbol namespaces differ, report this.
>	* src/abg-symtab-reader.cc (symtab::load): Try to get
>	__ksymtab_strings metadata and data. Use this to look up
>	__kstrtabns_FOO namespace entries. Set symbol namespaces where
>	found.
>	* src/abg-writer.cc (write_elf_symbol): Emit ns attribute, if
>	symbol has a namespace.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>

Thanks for working on that! I think this is conceptually the right thing
to do to allow compatibility with different kernel versions and
configurations. I just had some minor comments. With those addressed,
feel free to add:

Reviewed-by: Matthias Maennich <maennich@google.com>

>---
> include/abg-ir.h         |  9 ++++++++
> src/abg-comp-filter.cc   | 39 +++++++++++++++++++++++++++++++-
> src/abg-ir.cc            | 27 +++++++++++++++++++++-
> src/abg-reader.cc        |  7 ++++++
> src/abg-reporter-priv.cc | 12 ++++++++++
> src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++
> src/abg-writer.cc        |  4 ++++
> 7 files changed, 145 insertions(+), 2 deletions(-)
>
>diff --git a/include/abg-ir.h b/include/abg-ir.h
>index a2f4e1a7..7c42bea7 100644
>--- a/include/abg-ir.h
>+++ b/include/abg-ir.h
>@@ -21,6 +21,7 @@
> #include <functional>
> #include <set>
> #include <unordered_map>
>+#include "abg-cxx-compat.h"
> #include "abg-fwd.h"
> #include "abg-hash.h"
> #include "abg-traverse.h"
>@@ -921,6 +922,7 @@ private:
> 	     visibility		vi,
> 	     bool		is_in_ksymtab = false,
> 	     uint64_t		crc = 0,
>+	     const abg_compat::optional<std::string>&	ns = {},

When implementing this in the kernel itself, I also got trapped in
naming inconsistencies, ns ... NS ... namespace. I believe I went with
consistently calling it 'namespace'. I suggest to not abbreviate with
'ns'.

> 	     bool		is_suppressed = false);
>
>   elf_symbol(const elf_symbol&);
>@@ -946,6 +948,7 @@ public:
> 	 visibility	    vi,
> 	 bool		    is_in_ksymtab = false,
> 	 uint64_t	    crc = 0,
>+	 const abg_compat::optional<std::string>&	ns = {},
> 	 bool		    is_suppressed = false);
>
>   const environment*
>@@ -1023,6 +1026,12 @@ public:
>   void
>   set_crc(uint64_t crc);
>
>+  const abg_compat::optional<std::string>&
>+  get_namespace() const;
>+
>+  void
>+  set_namespace(const abg_compat::optional<std::string>& ns);
>+
>   bool
>   is_suppressed() const;
>
>diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
>index 56251274..6ee31e35 100644
>--- a/src/abg-comp-filter.cc
>+++ b/src/abg-comp-filter.cc
>@@ -254,6 +254,42 @@ crc_changed(const diff* diff)
>   return false;
> }
>
>+/// Test if there was a function or variable namespace change.
>+///
>+/// @param f the first function or variable to consider.
>+///
>+/// @param s the second function or variable to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+template <typename function_or_var_decl_sptr>
>+static bool
>+namespace_changed(const function_or_var_decl_sptr& f,
>+		  const function_or_var_decl_sptr& s)
>+{
>+  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();

const auto& to avoid some refcounts on the symbol_sptr.

>+  if (!symbol_f || !symbol_s)
>+    return false;
>+  return symbol_f->get_namespace() != symbol_s->get_namespace();
>+}
>+
>+/// test if the current diff tree node carries a namespace change in
       ^
       Test
>+/// either a function or a variable.
>+///
>+/// @param diff the diff tree node to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+static bool
>+namespace_changed(const diff* diff)
>+{
>+  if (const function_decl_diff* d =
>+	dynamic_cast<const function_decl_diff*>(diff))
>+    return namespace_changed(d->first_function_decl(),
>+			     d->second_function_decl());
>+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
>+    return namespace_changed(d->first_var(), d->second_var());
>+  return false;
>+}
>+
> /// Test if there was a function name change, but there there was no
> /// change in name of the underlying symbol.  IOW, if the name of a
> /// function changed, but the symbol of the new function is equal to
>@@ -1781,7 +1817,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
> 	      || non_static_data_member_added_or_removed(d)
> 	      || base_classes_added_or_removed(d)
> 	      || has_harmful_enum_change(d)
>-	      || crc_changed(d)))
>+	      || crc_changed(d)
>+	      || namespace_changed(d)))
> 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
>
>       if (has_virtual_mem_fn_change(d))
>diff --git a/src/abg-ir.cc b/src/abg-ir.cc
>index 0ef5e8b2..c510bc55 100644
>--- a/src/abg-ir.cc
>+++ b/src/abg-ir.cc
>@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
>   bool			is_common_;
>   bool			is_in_ksymtab_;
>   uint64_t		crc_;
>+  abg_compat::optional<std::string>	namespace_;
>   bool			is_suppressed_;
>   elf_symbol_wptr	main_symbol_;
>   elf_symbol_wptr	next_alias_;
>@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
>       is_common_(false),
>       is_in_ksymtab_(false),
>       crc_(0),
>+      namespace_(),
>       is_suppressed_(false)
>   {}
>
>@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
>        elf_symbol::visibility	  vi,
>        bool			  is_in_ksymtab,
>        uint64_t			  crc,
>+       const abg_compat::optional<std::string>&	ns,
>        bool			  is_suppressed)
>     : env_(e),
>       index_(i),
>@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
>       is_common_(c),
>       is_in_ksymtab_(is_in_ksymtab),
>       crc_(crc),
>+      namespace_(ns),
>       is_suppressed_(is_suppressed)
>   {
>     if (!is_common_)
>@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
> /// @param vi the visibility of the symbol.
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
>+///
>+/// @param ns the namespace of Linux Kernel symbols, if any
> elf_symbol::elf_symbol(const environment* e,
> 		       size_t		  i,
> 		       size_t		  s,
>@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		       visibility	  vi,
> 		       bool		  is_in_ksymtab,
> 		       uint64_t		  crc,
>+		       const abg_compat::optional<std::string>&	ns,
> 		       bool		  is_suppressed)
>   : priv_(new priv(e,
> 		   i,
>@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		   vi,
> 		   is_in_ksymtab,
> 		   crc,
>+		   ns,
> 		   is_suppressed))
> {}
>
>@@ -1886,6 +1894,8 @@ elf_symbol::create()
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
> ///
>+/// @param ns the namespace of Linux Kernel symbols, if any
>+///
> /// @return a (smart) pointer to a newly created instance of @ref
> /// elf_symbol.
> elf_symbol_sptr
>@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
> 		   visibility	      vi,
> 		   bool		      is_in_ksymtab,
> 		   uint64_t	      crc,
>+		   const abg_compat::optional<std::string>&	ns,
> 		   bool		      is_suppressed)
> {
>   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
>-				     is_in_ksymtab, crc, is_suppressed));
>+				     is_in_ksymtab, crc, ns, is_suppressed));
>   sym->priv_->main_symbol_ = sym;
>   return sym;
> }
>@@ -2147,6 +2158,20 @@ void
> elf_symbol::set_crc(uint64_t crc)
> {priv_->crc_ = crc;}
>
>+/// Getter of the 'namespace' property.
>+///
>+/// @return the namespace for Linux Kernel symbols, if any
>+const abg_compat::optional<std::string>&
>+elf_symbol::get_namespace() const
>+{return priv_->namespace_;}
>+
>+/// Setter of the 'namespace' property.
>+///
>+/// @param ns the new namespace for Linux Kernel symbols, if any
>+void
>+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)

While I like the std::optional in the data structure, I am not sure I
like it exposed to the interface. Is there ever a case where
set_namespace is called with an unset optional? And if so, would that
not be a better interface to offer ::unset_namespace ?

>+{priv_->namespace_ = ns;}
>+
> /// Getter for the 'is-suppressed' property.
> ///
> /// @return true iff the current symbol has been suppressed by a
>diff --git a/src/abg-reader.cc b/src/abg-reader.cc
>index 31885692..55b7348d 100644
>--- a/src/abg-reader.cc
>+++ b/src/abg-reader.cc
>@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
>   if (crc != 0)
>     e->set_crc(crc);
>
>+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "ns"))

In particular, I would call the attribute 'namespace' in the XML unless
we want to save bytes here.

>+    {
>+      std::string ns;
>+      xml::xml_char_sptr_to_string(s, ns);
>+      e->set_namespace({ns});
>+    }
>+
>   return e;
> }
>
>diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
>index 7012f5dc..dd40690c 100644
>--- a/src/abg-reporter-priv.cc
>+++ b/src/abg-reporter-priv.cc
>@@ -1158,6 +1158,18 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
> 	  << std::noshowbase << std::dec
> 	  << "\n";
>     }
>+
>+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
>+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
>+  if (ns1 != ns2)
>+    {
>+      const std::string none = "(none)";
>+      out << indent << "namespace changed from "
>+	  << (ns1.has_value() ? ns1.value() : none)
>+	  << " to "
>+	  << (ns2.has_value() ? ns2.value() : none)
>+	  << "\n";
>+    }
> }
>
> /// For a given symbol, emit a string made of its name and version.
>diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
>index b42ce87d..589f3703 100644
>--- a/src/abg-symtab-reader.cc
>+++ b/src/abg-symtab-reader.cc
>@@ -232,9 +232,33 @@ symtab::load_(Elf*	       elf_handle,
>       return false;
>     }
>
>+  // The __kstrtab_strings sections is basically an ELF strtab but does not
>+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
>+  // section data.
>+  //
>+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
>+  // within the __kstrtab_strings section. To look up the string value, we need
>+  // to translate from load address to section offset by subtracting the base
>+  // address of the section.
>+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
>+  size_t strings_offset = 0;
>+  const char* strings_data = nullptr;
>+  size_t strings_size = 0;
>+  if (strings_section)
>+    {
>+      GElf_Shdr strings_sheader;
>+      gelf_getshdr(strings_section, &strings_sheader);
>+      strings_offset = strings_sheader.sh_addr;
>+      Elf_Data* data = elf_getdata(strings_section, nullptr);
>+      ABG_ASSERT(data->d_off == 0);
>+      strings_data = reinterpret_cast<const char *>(data->d_buf);
>+      strings_size = data->d_size;
>+    }
>+
>   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
>   std::unordered_set<std::string> exported_kernel_symbols;
>   std::unordered_map<std::string, uint64_t> crc_values;
>+  std::unordered_map<std::string, std::string> namespaces;
>
>   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
>   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
>@@ -288,6 +312,20 @@ symtab::load_(Elf*	       elf_handle,
> 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
> 	  continue;
> 	}
>+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
>+	{
>+	  // This symbol lives in the __ksymtab_strings section but st_value is
>+	  // a load address so we need to subtract an offset before looking it
>+	  // up in that section.
>+	  const size_t offset = sym->st_value - strings_offset;
>+	  if (offset < strings_size)
>+	    {
>+	      const char* ns = strings_data + offset;
>+	      if (*ns)
>+		ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);

    constexpr auto name_offset = sizeof("__kstrtabns_");
    ABG_ASSERT(namespaces.emplace(name.substr(name_offset), ns).second);

Also, consider limiting the string length to the section end to not
accidentally read beyond.

Cheers,
Matthias


>+	    }
>+	  continue;
>+	}
>
>       // filter out uninteresting entries and only keep functions/variables for
>       // now. The rest might be interesting in the future though.
>@@ -402,6 +440,17 @@ symtab::load_(Elf*	       elf_handle,
> 	symbol->set_crc(crc_entry.second);
>     }
>
>+  // Now add the namespaces
>+  for (const auto& namespace_entry : namespaces)
>+    {
>+      const auto r = name_symbol_map_.find(namespace_entry.first);
>+      if (r == name_symbol_map_.end())
>+	continue;
>+
>+      for (const auto& symbol : r->second)
>+	symbol->set_namespace({namespace_entry.second});
>+    }
>+
>   // sort the symbols for deterministic output
>   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
>
>diff --git a/src/abg-writer.cc b/src/abg-writer.cc
>index 7802128d..4dfa72a6 100644
>--- a/src/abg-writer.cc
>+++ b/src/abg-writer.cc
>@@ -3136,6 +3136,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
>       << std::hex << std::showbase << sym->get_crc() << "'"
>       << std::dec << std::noshowbase;
>
>+  if (sym->get_namespace().has_value())
>+    o << " ns='"
>+      << sym->get_namespace().value() << "'";
>+
>   o << "/>\n";
>
>   return true;
>-- 
>2.35.1.723.g4982287a31-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH 1/2] optional: add operator== and operator!=
  2022-03-15 11:02   ` Matthias Maennich
@ 2022-03-16  9:31     ` Giuliano Procida
  0 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16  9:31 UTC (permalink / raw)
  To: Matthias Maennich; +Cc: libabigail, dodji, kernel-team

Hi.

On Tue, 15 Mar 2022 at 11:02, Matthias Maennich <maennich@google.com> wrote:
>
> On Mon, Mar 14, 2022 at 06:13:11PM +0000, Giuliano Procida wrote:
> >       * include/abg-cxx-compat.h (optional): Add operator== and
> >       operator!=.
> >
> >Signed-off-by: Giuliano Procida <gprocida@google.com>
> >---
> > include/abg-cxx-compat.h | 16 ++++++++++++++++
> > 1 file changed, 16 insertions(+)
> >
> >diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
> >index 443905c7..a2cf9095 100644
> >--- a/include/abg-cxx-compat.h
> >+++ b/include/abg-cxx-compat.h
> >@@ -91,6 +91,22 @@ public:
> >   }
> >
> >   explicit operator bool() const { return has_value_; }
>
>      std signature is (at least make it noexcept):
>        > constexpr explicit operator bool() const noexcept;
>

Will do. I could add some, but not all, the constexpr as C++11
complains about ambiguous overloads.

> >+
> >+  bool
> >+  operator==(const optional<T>& other) const
> >+  {
> >+    if (!has_value_ && !other.has_value_)
> >+      return true;
> >+    if (!has_value_ || !other.has_value_)
> >+      return false;
> >+    return value_ == other.value_;
> >+  }
> >+
> >+  bool
> >+  operator!=(const optional<T>& other) const
> >+  {
> >+    return !operator==(other);
> >+  }
>
> I believe C++17 defines those as non-member functions. Should we follow
> that for compatibility. Eventually, when switching to C++17 or later we
> would want to just replace abg_compat:: with std:: and things should
> "just work".
>

Sure.

> Cheers,
> Matthias
>

v2 will follow.

Giuliano.

> > };
> >
> > #endif
> >--
> >2.35.1.723.g4982287a31-goog
> >

^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v2 0/4] add symbol namespace support, update symbol CRC support
  2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida
  2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida
  2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-16 16:30 ` Giuliano Procida
  2022-03-16 16:30   ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida
                     ` (4 more replies)
  2 siblings, 5 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

New in v2:

* updates following review (includng tweaking a function I copied from)
* better offset checking and working extraction of namespace from .ko
* rework of CRCs to be similar to namespaces (use of optional etc.)
* test case additions / updates

Giuliano Procida (4):
  optional: minor improvements
  crc_changed: eliminate copying of shared_ptr values
  add Linux kernel symbol namespace support
  Linux symbol CRCs: support 0 and report presence changes

 include/abg-cxx-compat.h                      | 30 +++++++--
 include/abg-ir.h                              | 17 ++++--
 src/abg-comp-filter.cc                        | 46 ++++++++++++--
 src/abg-ir.cc                                 | 47 ++++++++++----
 src/abg-reader.cc                             | 15 +++--
 src/abg-reporter-priv.cc                      | 38 ++++++++++--
 src/abg-symtab-reader.cc                      | 61 +++++++++++++++++++
 src/abg-writer.cc                             |  9 ++-
 tests/data/Makefile.am                        |  7 ++-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 ++++++
 tests/test-abidiff.cc                         | 12 +++-
 tests/test-symtab.cc                          |  8 +--
 17 files changed, 322 insertions(+), 47 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v2 1/4] optional: minor improvements
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
@ 2022-03-16 16:30   ` Giuliano Procida
  2022-03-17 10:56     ` Matthias Maennich
  2022-03-16 16:30   ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
                     ` (3 subsequent siblings)
  4 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

This change makes minor improvements to the optional class used with
pre-C++17 compilers.

- adds operator== and operator!=
- adds various missing noexcept (but not constexpr) decorations
- defines operator bool in terms of has_value

Note that some constexpr decorations would require C++17 anyway.

	* include/abg-cxx-compat.h (optional): Add operator== and
	operator!=. Add noexcept decorations. Tweak operator bool.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
index 443905c7..5c5943d0 100644
--- a/include/abg-cxx-compat.h
+++ b/include/abg-cxx-compat.h
@@ -45,7 +45,7 @@ public:
   optional(const T& value) : has_value_(true), value_(value) {}
 
   bool
-  has_value() const
+  has_value() const noexcept
   {
     return has_value_;
   }
@@ -67,19 +67,19 @@ public:
   }
 
   const T&
-  operator*() const
+  operator*() const& noexcept
   { return value_; }
 
   T&
-  operator*()
+  operator*() & noexcept
   { return value_; }
 
   const T*
-  operator->() const
+  operator->() const noexcept
   { return &value_; }
 
   T*
-  operator->()
+  operator->() noexcept
   { return &value_; }
 
   optional&
@@ -90,9 +90,27 @@ public:
     return *this;
   }
 
-  explicit operator bool() const { return has_value_; }
+  explicit operator bool() const noexcept { return has_value(); }
 };
 
+template <typename T, typename U>
+bool
+operator==(const optional<T>& lhs, const optional<U>& rhs)
+{
+  if (!lhs.has_value() && !rhs.has_value())
+    return true;
+  if (!lhs.has_value() || !rhs.has_value())
+    return false;
+  return lhs.value() == rhs.value();
+}
+
+template <typename T, typename U>
+bool
+operator!=(const optional<T>& lhs, const optional<U>& rhs)
+{
+  return !(lhs == rhs);
+}
+
 #endif
 }
 
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-16 16:30   ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida
@ 2022-03-16 16:30   ` Giuliano Procida
  2022-03-17 11:01     ` Matthias Maennich
  2022-03-16 16:30   ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida
                     ` (2 subsequent siblings)
  4 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

As pointed out in a review of similar code, it is possible to avoid
copying a couple of shared pointers in this function, by taking
references instead.

This commit also splits declarations to one per line and removes the
unnecessary parentheses around the return expression.

	* src/abg-comp-filter.cc (crc_changed): Take references to
	avoid std::shared_ptr copying. Split declarations into one per
	line. Remove unnecessary return expression parentheses.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 src/abg-comp-filter.cc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 56251274..f90fdc78 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -230,11 +230,13 @@ static bool
 crc_changed(const function_or_var_decl_sptr& f,
 	    const function_or_var_decl_sptr& s)
 {
-  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc();
-  return (crc_f != 0 && crc_s != 0 && crc_f != crc_s);
+  const auto crc_f = symbol_f->get_crc();
+  const auto crc_s = symbol_s->get_crc();
+  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v2 3/4] add Linux kernel symbol namespace support
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-16 16:30   ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida
  2022-03-16 16:30   ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-03-16 16:30   ` Giuliano Procida
  2022-03-17 11:57     ` Matthias Maennich
  2022-03-16 16:30   ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  4 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Bug 28954 - add Linux Kernel symbol namespace support

Each Linux kernel symbol can be exported to a specified named
namespace or left in the global (nameless) namespace.

One complexity is that the offset symbol values require slightly
different interpretation for vmlinux and .ko loadable modules as the
former has a fixed load address but the latter are relocatable.

The global namespace is explicitly represented as the empty string, at
least when it comes to the value of __kstrtabns_FOO symbols, but the
common interpretation is that such symbols lack an export namespace.

I would rather not have to make use of "empty implies missing" in many
places, so the code here represents namespace as optional<string> and
only the symtab reader cares about empty strings in __ksymtab_strings.

	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
	(elf_symbol::create): Add ns argument.
	(elf_symbol::get_namespace): Declare new function.
	(elf_symbol::set_namespace): Declare new function.
	and set_namespace.
	* src/abg-comp-filter.cc (namespace_changed): Define new
	helper functions.
	(categorize_harmful_diff_node): Also call namespace_changed().
	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
	(elf_symbol::priv::priv): Add namespace_ to initialisers.
	(elf_symbol::elf_symbol): Take new ns argument and pass it to
	priv constructor.
	(elf_symbol::create): Take new ns argument and pass it to
	elf_symbol constructor.
	(elf_symbol::get_namespace): Define new function.
	(elf_symbol::set_namespace): Define new function.
	* src/abg-reader.cc (build_elf_symbol): If namespace
	attribute is present, set symbol namespace.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
	symbol namespaces differ, report this.
	* src/abg-symtab-reader.cc (symtab::load): Try to get
	__ksymtab_strings metadata and data. Use these to look up
	__kstrtabns_FOO namespace entries. Set symbol namespace where
	found.
	* src/abg-writer.cc (write_elf_symbol): Emit namespace
	attribute, if symbol has a namespace.
	* tests/data/Makefile.am: Add new test files.
	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
	* tests/test-abidiff.cc: Add new test case.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  9 +++
 src/abg-comp-filter.cc                        | 40 +++++++++++-
 src/abg-ir.cc                                 | 30 ++++++++-
 src/abg-reader.cc                             |  7 +++
 src/abg-reporter-priv.cc                      | 18 ++++++
 src/abg-symtab-reader.cc                      | 61 +++++++++++++++++++
 src/abg-writer.cc                             |  3 +
 tests/data/Makefile.am                        |  3 +
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 ++++++
 tests/test-abidiff.cc                         |  6 ++
 12 files changed, 221 insertions(+), 3 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

diff --git a/include/abg-ir.h b/include/abg-ir.h
index a2f4e1a7..7c42bea7 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -21,6 +21,7 @@
 #include <functional>
 #include <set>
 #include <unordered_map>
+#include "abg-cxx-compat.h"
 #include "abg-fwd.h"
 #include "abg-hash.h"
 #include "abg-traverse.h"
@@ -921,6 +922,7 @@ private:
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
 	     uint64_t		crc = 0,
+	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -946,6 +948,7 @@ public:
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
 	 uint64_t	    crc = 0,
+	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1023,6 +1026,12 @@ public:
   void
   set_crc(uint64_t crc);
 
+  const abg_compat::optional<std::string>&
+  get_namespace() const;
+
+  void
+  set_namespace(const abg_compat::optional<std::string>& ns);
+
   bool
   is_suppressed() const;
 
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index f90fdc78..c179b6bd 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -256,6 +256,43 @@ crc_changed(const diff* diff)
   return false;
 }
 
+/// Test if there was a function or variable namespace change.
+///
+/// @param f the first function or variable to consider.
+///
+/// @param s the second function or variable to consider.
+///
+/// @return true if the test is positive, false otherwise.
+template <typename function_or_var_decl_sptr>
+static bool
+namespace_changed(const function_or_var_decl_sptr& f,
+		  const function_or_var_decl_sptr& s)
+{
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
+  if (!symbol_f || !symbol_s)
+    return false;
+  return symbol_f->get_namespace() != symbol_s->get_namespace();
+}
+
+/// Test if the current diff tree node carries a namespace change in
+/// either a function or a variable.
+///
+/// @param diff the diff tree node to consider.
+///
+/// @return true if the test is positive, false otherwise.
+static bool
+namespace_changed(const diff* diff)
+{
+  if (const function_decl_diff* d =
+	dynamic_cast<const function_decl_diff*>(diff))
+    return namespace_changed(d->first_function_decl(),
+			     d->second_function_decl());
+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
+    return namespace_changed(d->first_var(), d->second_var());
+  return false;
+}
+
 /// Test if there was a function name change, but there there was no
 /// change in name of the underlying symbol.  IOW, if the name of a
 /// function changed, but the symbol of the new function is equal to
@@ -1783,7 +1820,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
 	      || non_static_data_member_added_or_removed(d)
 	      || base_classes_added_or_removed(d)
 	      || has_harmful_enum_change(d)
-	      || crc_changed(d)))
+	      || crc_changed(d)
+	      || namespace_changed(d)))
 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
 
       if (has_virtual_mem_fn_change(d))
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 0ef5e8b2..78154622 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
   bool			is_common_;
   bool			is_in_ksymtab_;
   uint64_t		crc_;
+  abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
       is_common_(false),
       is_in_ksymtab_(false),
       crc_(0),
+      namespace_(),
       is_suppressed_(false)
   {}
 
@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
        uint64_t			  crc,
+       const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
       is_common_(c),
       is_in_ksymtab_(is_in_ksymtab),
       crc_(crc),
+      namespace_(ns),
       is_suppressed_(is_suppressed)
   {
     if (!is_common_)
@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
 /// @param vi the visibility of the symbol.
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
+///
+/// @param ns the namespace of Linux Kernel symbols, if any
 elf_symbol::elf_symbol(const environment* e,
 		       size_t		  i,
 		       size_t		  s,
@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
 		       uint64_t		  crc,
+		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
 		   vi,
 		   is_in_ksymtab,
 		   crc,
+		   ns,
 		   is_suppressed))
 {}
 
@@ -1886,6 +1894,8 @@ elf_symbol::create()
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
 ///
+/// @param ns the namespace of Linux Kernel symbols, if any
+///
 /// @return a (smart) pointer to a newly created instance of @ref
 /// elf_symbol.
 elf_symbol_sptr
@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
 		   uint64_t	      crc,
+		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
-				     is_in_ksymtab, crc, is_suppressed));
+				     is_in_ksymtab, crc, ns, is_suppressed));
   sym->priv_->main_symbol_ = sym;
   return sym;
 }
@@ -1927,7 +1938,8 @@ textually_equals(const elf_symbol&l,
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
 		 && (l.get_crc() == 0 || r.get_crc() == 0
-		     || l.get_crc() == r.get_crc()));
+		     || l.get_crc() == r.get_crc())
+		 && l.get_namespace() == r.get_namespace());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2147,6 +2159,20 @@ void
 elf_symbol::set_crc(uint64_t crc)
 {priv_->crc_ = crc;}
 
+/// Getter of the 'namespace' property.
+///
+/// @return the namespace for Linux Kernel symbols, if any
+const abg_compat::optional<std::string>&
+elf_symbol::get_namespace() const
+{return priv_->namespace_;}
+
+/// Setter of the 'namespace' property.
+///
+/// @param ns the new namespace for Linux Kernel symbols, if any
+void
+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
+{priv_->namespace_ = ns;}
+
 /// Getter for the 'is-suppressed' property.
 ///
 /// @return true iff the current symbol has been suppressed by a
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 31885692..0a8ecd70 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
   if (crc != 0)
     e->set_crc(crc);
 
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
+    {
+      std::string ns;
+      xml::xml_char_sptr_to_string(s, ns);
+      e->set_namespace(ns);
+    }
+
   return e;
 }
 
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 7012f5dc..9ebcdf00 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1158,6 +1158,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << std::noshowbase << std::dec
 	  << "\n";
     }
+
+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
+  if (ns1 != ns2)
+    {
+      const std::string none = "(none)";
+      out << indent << "namespace changed from ";
+      if (ns1.has_value())
+	out << "'" << ns1.value() << "'";
+      else
+	out << none;
+      out << " to ";
+      if (ns2.has_value())
+	out << "'" << ns2.value() << "'";
+      else
+	out << none;
+      out << "\n";
+    }
 }
 
 /// For a given symbol, emit a string made of its name and version.
diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
index b42ce87d..e59bd62c 100644
--- a/src/abg-symtab-reader.cc
+++ b/src/abg-symtab-reader.cc
@@ -232,9 +232,33 @@ symtab::load_(Elf*	       elf_handle,
       return false;
     }
 
+  // The __kstrtab_strings sections is basically an ELF strtab but does not
+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
+  // section data.
+  //
+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
+  // within the __kstrtab_strings section. To look up the string value, we need
+  // to translate from load address to section offset by subtracting the base
+  // address of the section.
+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
+  size_t strings_offset = 0;
+  const char* strings_data = nullptr;
+  size_t strings_size = 0;
+  if (strings_section)
+    {
+      GElf_Shdr strings_sheader;
+      gelf_getshdr(strings_section, &strings_sheader);
+      strings_offset = strings_sheader.sh_addr;
+      Elf_Data* data = elf_getdata(strings_section, nullptr);
+      ABG_ASSERT(data->d_off == 0);
+      strings_data = reinterpret_cast<const char *>(data->d_buf);
+      strings_size = data->d_size;
+    }
+
   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
   std::unordered_set<std::string> exported_kernel_symbols;
   std::unordered_map<std::string, uint64_t> crc_values;
+  std::unordered_map<std::string, std::string> namespaces;
 
   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
@@ -288,6 +312,32 @@ symtab::load_(Elf*	       elf_handle,
 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
 	  continue;
 	}
+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
+	{
+	  // This symbol lives in the __ksymtab_strings section but st_value is
+	  // a load address so, for vmlinux only, we need to subtract an offset
+	  // before looking it up in that section. Kernel module offsets must
+	  // not be adjusted. The simple way to distinguish the cases is to see
+	  // if the adjustment would overflow.
+          const size_t value = sym->st_value;
+	  const size_t offset =
+              value < strings_offset ? value : value - strings_offset;
+	  // check offset
+	  ABG_ASSERT(offset < strings_size);
+	  // find the terminating NULL
+	  const char* first = strings_data + offset;
+	  const char* last = strings_data + strings_size;
+	  const char* limit = std::find(first, last, 0);
+	  // check NULL found
+	  ABG_ASSERT(limit < last);
+	  // interpret the empty namespace name as no namespace name
+	  if (first < limit)
+	    {
+	      const std::string ns(first, limit - first);
+	      ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);
+	    }
+	  continue;
+	}
 
       // filter out uninteresting entries and only keep functions/variables for
       // now. The rest might be interesting in the future though.
@@ -402,6 +452,17 @@ symtab::load_(Elf*	       elf_handle,
 	symbol->set_crc(crc_entry.second);
     }
 
+  // Now add the namespaces
+  for (const auto& namespace_entry : namespaces)
+    {
+      const auto r = name_symbol_map_.find(namespace_entry.first);
+      if (r == name_symbol_map_.end())
+	continue;
+
+      for (const auto& symbol : r->second)
+	symbol->set_namespace(namespace_entry.second);
+    }
+
   // sort the symbols for deterministic output
   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 7802128d..692abc75 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
       << std::hex << std::showbase << sym->get_crc() << "'"
       << std::dec << std::noshowbase;
 
+  if (sym->get_namespace().has_value())
+    o << " namespace='" << sym->get_namespace().value() << "'";
+
   o << "/>\n";
 
   return true;
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index a7eb7ff0..7ea24d58 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -91,6 +91,9 @@ test-abidiff/test-crc-0.xml \
 test-abidiff/test-crc-1.xml \
 test-abidiff/test-crc-2.xml \
 test-abidiff/test-crc-report.txt \
+test-abidiff/test-namespace-0.xml \
+test-abidiff/test-namespace-1.xml \
+test-abidiff/test-namespace-report.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
new file mode 100644
index 00000000..5a9f5cd2
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-0.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
new file mode 100644
index 00000000..9814844b
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-1.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
new file mode 100644
index 00000000..d2c421ed
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-report.txt
@@ -0,0 +1,17 @@
+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
+
+4 Changed variables:
+
+  [C] 'int v1' was changed:
+    namespace changed from (none) to 'this'
+
+  [C] 'int v2' was changed:
+    namespace changed from 'this' to 'that'
+
+  [C] 'int v3' was changed:
+    namespace changed from 'that' to ''
+
+  [C] 'int v4' was changed:
+    namespace changed from '' to (none)
+
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index 32858ece..009f7de5 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -128,6 +128,12 @@ static InOutSpec specs[] =
     "data/test-abidiff/test-crc-report.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
+  {
+    "data/test-abidiff/test-namespace-0.xml",
+    "data/test-abidiff/test-namespace-1.xml",
+    "data/test-abidiff/test-namespace-report.txt",
+    "output/test-abidiff/test-namespace-report-0-1.txt"
+  },
   {
     "data/test-abidiff/test-PR27616-v0.xml",
     "data/test-abidiff/test-PR27616-v1.xml",
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                     ` (2 preceding siblings ...)
  2022-03-16 16:30   ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-16 16:30   ` Giuliano Procida
  2022-03-17 12:01     ` Matthias Maennich
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  4 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

The CRC with value zero was used to mean "absent". This can be better
modelled using optional.

This commit makes this change and also tweaks reporting so that
disappearing / appearing CRCs are noted. This should be essentially
impossible unless CRCs are enabled / disabled altogether but would be
very noteworthy otherwise.

	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
	now an optional defaulted to absent.
	(elf_symbol::create): Likewise.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_src): Now takes an optional uint64_t.
	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
	optional uint64_t.
	(elf_symbol::priv::priv): Argument crc is now an optional
	uint64_t.
	(elf_symbol::elf_symbol): Likewise.
	(elf_symbol::create): Argument crc is now an optional uint64_t
	and defaults to absent.
	(textually_equals): Simplify comparison.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_crc): Now takes an optional uint64_t.
	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
	Treat CRC 0 the same as other CRC values and also report
	changes to CRC presence.
	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
	showing additional of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
	showing removal of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
	from tests/data/test-abidiff/test-crc-report.txt.
	* tests/test-abidiff.cc: Update test cases that no longer
	generate empty reports.
	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  8 ++++----
 src/abg-comp-filter.cc                        |  4 +---
 src/abg-ir.cc                                 | 19 +++++++++---------
 src/abg-reader.cc                             |  8 ++------
 src/abg-reporter-priv.cc                      | 20 ++++++++++++++-----
 src/abg-writer.cc                             |  6 +++---
 tests/data/Makefile.am                        |  4 +++-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/test-abidiff.cc                         |  6 +++---
 tests/test-symtab.cc                          |  8 ++++----
 12 files changed, 76 insertions(+), 39 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)

diff --git a/include/abg-ir.h b/include/abg-ir.h
index 7c42bea7..7c558828 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -921,7 +921,7 @@ private:
 	     const version&	ve,
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
-	     uint64_t		crc = 0,
+	     const abg_compat::optional<uint64_t>&	crc = {},
 	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
@@ -947,7 +947,7 @@ public:
 	 const version&	    ve,
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
-	 uint64_t	    crc = 0,
+	 const abg_compat::optional<uint64_t>&		crc = {},
 	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
@@ -1020,11 +1020,11 @@ public:
   void
   set_is_in_ksymtab(bool is_in_ksymtab);
 
-  uint64_t
+  const abg_compat::optional<uint64_t>&
   get_crc() const;
 
   void
-  set_crc(uint64_t crc);
+  set_crc(const abg_compat::optional<uint64_t>& crc);
 
   const abg_compat::optional<std::string>&
   get_namespace() const;
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index c179b6bd..22da5244 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f,
   const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc();
-  const auto crc_s = symbol_s->get_crc();
-  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
+  return symbol_f->get_crc() != symbol_s->get_crc();
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 78154622..f90be2e4 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1727,7 +1727,7 @@ struct elf_symbol::priv
   //     STT_COMMON definition of that name that has the largest size.
   bool			is_common_;
   bool			is_in_ksymtab_;
-  uint64_t		crc_;
+  abg_compat::optional<uint64_t>	crc_;
   abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
@@ -1745,7 +1745,7 @@ struct elf_symbol::priv
       is_defined_(false),
       is_common_(false),
       is_in_ksymtab_(false),
-      crc_(0),
+      crc_(),
       namespace_(),
       is_suppressed_(false)
   {}
@@ -1761,7 +1761,7 @@ struct elf_symbol::priv
        const elf_symbol::version& ve,
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
-       uint64_t			  crc,
+       const abg_compat::optional<uint64_t>&	crc,
        const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
@@ -1835,7 +1835,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       const version&	  ve,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
-		       uint64_t		  crc,
+		       const abg_compat::optional<uint64_t>&	crc,
 		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
@@ -1910,7 +1910,7 @@ elf_symbol::create(const environment* e,
 		   const version&     ve,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
-		   uint64_t	      crc,
+		   const abg_compat::optional<uint64_t>&	crc,
 		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
@@ -1937,8 +1937,7 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && (l.get_crc() == 0 || r.get_crc() == 0
-		     || l.get_crc() == r.get_crc())
+		 && l.get_crc() == r.get_crc()
 		 && l.get_namespace() == r.get_namespace());
 
   if (equals && l.is_variable())
@@ -2147,8 +2146,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab)
 
 /// Getter of the 'crc' property.
 ///
-/// @return the CRC (modversions) value for Linux Kernel symbols (if present)
-uint64_t
+/// @return the CRC (modversions) value for Linux Kernel symbols, if any
+const abg_compat::optional<uint64_t>&
 elf_symbol::get_crc() const
 {return priv_->crc_;}
 
@@ -2156,7 +2155,7 @@ elf_symbol::get_crc() const
 ///
 /// @param crc the new CRC (modversions) value for Linux Kernel symbols
 void
-elf_symbol::set_crc(uint64_t crc)
+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
 /// Getter of the 'namespace' property.
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 0a8ecd70..f9e420f1 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 	is_default_version = true;
     }
 
-  uint64_t crc = 0;
-  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
-    crc = strtoull(CHAR_STR(s), NULL, 0);
-
   elf_symbol::type type = elf_symbol::NOTYPE_TYPE;
   read_elf_symbol_type(node, type);
 
@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 
   e->set_is_suppressed(is_suppressed);
 
-  if (crc != 0)
-    e->set_crc(crc);
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
+    e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
     {
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 9ebcdf00..9ad733f4 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << "\n";
     }
 
-  if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0
-      && symbol1->get_crc() != symbol2->get_crc())
+  const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc();
+  const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc();
+  if (crc1 != crc2)
     {
+      const std::string none = "(none)";
       out << indent << "CRC (modversions) changed from "
-	  << std::showbase << std::hex
-	  << symbol1->get_crc() << " to " << symbol2->get_crc()
-	  << std::noshowbase << std::dec
+	  << std::showbase << std::hex;
+      if (crc1.has_value())
+	out << crc1.value();
+      else
+	out << none;
+      out << " to ";
+      if (crc2.has_value())
+	out << crc2.value();
+      else
+	out << none;
+      out << std::noshowbase << std::dec
 	  << "\n";
     }
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 692abc75..03fb658b 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
   if (sym->is_common_symbol())
     o << " is-common='yes'";
 
-  if (sym->get_crc() != 0)
+  if (sym->get_crc().has_value())
     o << " crc='"
-      << std::hex << std::showbase << sym->get_crc() << "'"
-      << std::dec << std::noshowbase;
+      << std::hex << std::showbase << sym->get_crc().value()
+      << std::dec << std::noshowbase << "'";
 
   if (sym->get_namespace().has_value())
     o << " namespace='" << sym->get_namespace().value() << "'";
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index 7ea24d58..7388697b 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml		\
 test-abidiff/test-crc-0.xml \
 test-abidiff/test-crc-1.xml \
 test-abidiff/test-crc-2.xml \
-test-abidiff/test-crc-report.txt \
+test-abidiff/test-crc-report-0-1.txt \
+test-abidiff/test-crc-report-1-0.txt \
+test-abidiff/test-crc-report-1-2.txt \
 test-abidiff/test-namespace-0.xml \
 test-abidiff/test-namespace-1.xml \
 test-abidiff/test-namespace-report.txt \
diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt
new file mode 100644
index 00000000..0db42f68
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from (none) to 0xe52d5bcf
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from (none) to 0xee94d699
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from (none) to 0x41336c46
+
diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt
new file mode 100644
index 00000000..e11f29c1
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from 0xe52d5bcf to (none)
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from 0xee94d699 to (none)
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from 0x41336c46 to (none)
+
diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt
similarity index 100%
rename from tests/data/test-abidiff/test-crc-report.txt
rename to tests/data/test-abidiff/test-crc-report-1-2.txt
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index 009f7de5..93e44d6a 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -113,19 +113,19 @@ static InOutSpec specs[] =
   {
     "data/test-abidiff/test-crc-0.xml",
     "data/test-abidiff/test-crc-1.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-0-1.txt",
     "output/test-abidiff/test-crc-report-0-1.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-0.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-1-0.txt",
     "output/test-abidiff/test-crc-report-1-0.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-2.xml",
-    "data/test-abidiff/test-crc-report.txt",
+    "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
   {
diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc
index 7d7a2df0..85303563 100644
--- a/tests/test-symtab.cc
+++ b/tests/test-symtab.cc
@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]")
   {
     const std::string  binary = base_path + "one_of_each.ko";
     const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2);
-    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0);
-    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0);
+    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc());
+    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc());
   }
 }
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v2 1/4] optional: minor improvements
  2022-03-16 16:30   ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida
@ 2022-03-17 10:56     ` Matthias Maennich
  0 siblings, 0 replies; 37+ messages in thread
From: Matthias Maennich @ 2022-03-17 10:56 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Wed, Mar 16, 2022 at 04:30:52PM +0000, Giuliano Procida wrote:
>This change makes minor improvements to the optional class used with
>pre-C++17 compilers.
>
>- adds operator== and operator!=
>- adds various missing noexcept (but not constexpr) decorations
>- defines operator bool in terms of has_value
>
>Note that some constexpr decorations would require C++17 anyway.
>
>	* include/abg-cxx-compat.h (optional): Add operator== and
>	operator!=. Add noexcept decorations. Tweak operator bool.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>

Reviewed-by: Matthias Maennich <maennich@google.com>

Cheers,
Matthias

>---
> include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------
> 1 file changed, 24 insertions(+), 6 deletions(-)
>
>diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
>index 443905c7..5c5943d0 100644
>--- a/include/abg-cxx-compat.h
>+++ b/include/abg-cxx-compat.h
>@@ -45,7 +45,7 @@ public:
>   optional(const T& value) : has_value_(true), value_(value) {}
>
>   bool
>-  has_value() const
>+  has_value() const noexcept
>   {
>     return has_value_;
>   }
>@@ -67,19 +67,19 @@ public:
>   }
>
>   const T&
>-  operator*() const
>+  operator*() const& noexcept
>   { return value_; }
>
>   T&
>-  operator*()
>+  operator*() & noexcept
>   { return value_; }
>
>   const T*
>-  operator->() const
>+  operator->() const noexcept
>   { return &value_; }
>
>   T*
>-  operator->()
>+  operator->() noexcept
>   { return &value_; }
>
>   optional&
>@@ -90,9 +90,27 @@ public:
>     return *this;
>   }
>
>-  explicit operator bool() const { return has_value_; }
>+  explicit operator bool() const noexcept { return has_value(); }
> };
>
>+template <typename T, typename U>
>+bool
>+operator==(const optional<T>& lhs, const optional<U>& rhs)
>+{
>+  if (!lhs.has_value() && !rhs.has_value())
>+    return true;
>+  if (!lhs.has_value() || !rhs.has_value())
>+    return false;
>+  return lhs.value() == rhs.value();
>+}
>+
>+template <typename T, typename U>
>+bool
>+operator!=(const optional<T>& lhs, const optional<U>& rhs)
>+{
>+  return !(lhs == rhs);
>+}
>+
> #endif
> }
>
>-- 
>2.35.1.894.gb6a874cedc-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values
  2022-03-16 16:30   ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-03-17 11:01     ` Matthias Maennich
  0 siblings, 0 replies; 37+ messages in thread
From: Matthias Maennich @ 2022-03-17 11:01 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team

On Wed, Mar 16, 2022 at 04:30:53PM +0000, Giuliano Procida via Libabigail wrote:
>As pointed out in a review of similar code, it is possible to avoid
>copying a couple of shared pointers in this function, by taking
>references instead.
>
>This commit also splits declarations to one per line and removes the
>unnecessary parentheses around the return expression.
>
>	* src/abg-comp-filter.cc (crc_changed): Take references to
>	avoid std::shared_ptr copying. Split declarations into one per
>	line. Remove unnecessary return expression parentheses.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>

Reviewed-by: Matthias Maennich <maennich@google.com>

Cheers,
Matthias

>---
> src/abg-comp-filter.cc | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
>diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
>index 56251274..f90fdc78 100644
>--- a/src/abg-comp-filter.cc
>+++ b/src/abg-comp-filter.cc
>@@ -230,11 +230,13 @@ static bool
> crc_changed(const function_or_var_decl_sptr& f,
> 	    const function_or_var_decl_sptr& s)
> {
>-  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
>+  const auto& symbol_f = f->get_symbol();
>+  const auto& symbol_s = s->get_symbol();
>   if (!symbol_f || !symbol_s)
>     return false;
>-  const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc();
>-  return (crc_f != 0 && crc_s != 0 && crc_f != crc_s);
>+  const auto crc_f = symbol_f->get_crc();
>+  const auto crc_s = symbol_s->get_crc();
>+  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
> }
>
> /// Test if the current diff tree node carries a CRC change in either a
>-- 
>2.35.1.894.gb6a874cedc-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v2 3/4] add Linux kernel symbol namespace support
  2022-03-16 16:30   ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-17 11:57     ` Matthias Maennich
  0 siblings, 0 replies; 37+ messages in thread
From: Matthias Maennich @ 2022-03-17 11:57 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Wed, Mar 16, 2022 at 04:30:54PM +0000, Giuliano Procida wrote:
>Bug 28954 - add Linux Kernel symbol namespace support
>
>Each Linux kernel symbol can be exported to a specified named
>namespace or left in the global (nameless) namespace.
>
>One complexity is that the offset symbol values require slightly
>different interpretation for vmlinux and .ko loadable modules as the
>former has a fixed load address but the latter are relocatable.
>
>The global namespace is explicitly represented as the empty string, at
>least when it comes to the value of __kstrtabns_FOO symbols, but the
>common interpretation is that such symbols lack an export namespace.
>
>I would rather not have to make use of "empty implies missing" in many
>places, so the code here represents namespace as optional<string> and
>only the symtab reader cares about empty strings in __ksymtab_strings.
>
>	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
>	(elf_symbol::create): Add ns argument.
>	(elf_symbol::get_namespace): Declare new function.
>	(elf_symbol::set_namespace): Declare new function.
>	and set_namespace.
>	* src/abg-comp-filter.cc (namespace_changed): Define new
>	helper functions.
>	(categorize_harmful_diff_node): Also call namespace_changed().
>	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
>	(elf_symbol::priv::priv): Add namespace_ to initialisers.
>	(elf_symbol::elf_symbol): Take new ns argument and pass it to
>	priv constructor.
>	(elf_symbol::create): Take new ns argument and pass it to
>	elf_symbol constructor.
>	(elf_symbol::get_namespace): Define new function.
>	(elf_symbol::set_namespace): Define new function.
>	* src/abg-reader.cc (build_elf_symbol): If namespace
>	attribute is present, set symbol namespace.
>	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
>	symbol namespaces differ, report this.
>	* src/abg-symtab-reader.cc (symtab::load): Try to get
>	__ksymtab_strings metadata and data. Use these to look up
>	__kstrtabns_FOO namespace entries. Set symbol namespace where
>	found.
>	* src/abg-writer.cc (write_elf_symbol): Emit namespace
>	attribute, if symbol has a namespace.
>	* tests/data/Makefile.am: Add new test files.
>	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
>	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
>	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
>	* tests/test-abidiff.cc: Add new test case.
>

Perhaps we can add an actual test case for a kernel module at least?
There are already test cases in test-symtab that could be forked for that
case.

>Signed-off-by: Giuliano Procida <gprocida@google.com>
>---
> include/abg-ir.h                              |  9 +++
> src/abg-comp-filter.cc                        | 40 +++++++++++-
> src/abg-ir.cc                                 | 30 ++++++++-
> src/abg-reader.cc                             |  7 +++
> src/abg-reporter-priv.cc                      | 18 ++++++
> src/abg-symtab-reader.cc                      | 61 +++++++++++++++++++
> src/abg-writer.cc                             |  3 +
> tests/data/Makefile.am                        |  3 +
> tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
> tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
> .../test-abidiff/test-namespace-report.txt    | 17 ++++++
> tests/test-abidiff.cc                         |  6 ++
> 12 files changed, 221 insertions(+), 3 deletions(-)
> create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
> create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
> create mode 100644 tests/data/test-abidiff/test-namespace-report.txt
>
>diff --git a/include/abg-ir.h b/include/abg-ir.h
>index a2f4e1a7..7c42bea7 100644
>--- a/include/abg-ir.h
>+++ b/include/abg-ir.h
>@@ -21,6 +21,7 @@
> #include <functional>
> #include <set>
> #include <unordered_map>
>+#include "abg-cxx-compat.h"
> #include "abg-fwd.h"
> #include "abg-hash.h"
> #include "abg-traverse.h"
>@@ -921,6 +922,7 @@ private:
> 	     visibility		vi,
> 	     bool		is_in_ksymtab = false,
> 	     uint64_t		crc = 0,
>+	     const abg_compat::optional<std::string>&	ns = {},
> 	     bool		is_suppressed = false);
>
>   elf_symbol(const elf_symbol&);
>@@ -946,6 +948,7 @@ public:
> 	 visibility	    vi,
> 	 bool		    is_in_ksymtab = false,
> 	 uint64_t	    crc = 0,
>+	 const abg_compat::optional<std::string>&	ns = {},
> 	 bool		    is_suppressed = false);
>
>   const environment*
>@@ -1023,6 +1026,12 @@ public:
>   void
>   set_crc(uint64_t crc);
>
>+  const abg_compat::optional<std::string>&
>+  get_namespace() const;
>+
>+  void
>+  set_namespace(const abg_compat::optional<std::string>& ns);
>+
>   bool
>   is_suppressed() const;
>
>diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
>index f90fdc78..c179b6bd 100644
>--- a/src/abg-comp-filter.cc
>+++ b/src/abg-comp-filter.cc
>@@ -256,6 +256,43 @@ crc_changed(const diff* diff)
>   return false;
> }
>
>+/// Test if there was a function or variable namespace change.
>+///
>+/// @param f the first function or variable to consider.
>+///
>+/// @param s the second function or variable to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+template <typename function_or_var_decl_sptr>
>+static bool
>+namespace_changed(const function_or_var_decl_sptr& f,
>+		  const function_or_var_decl_sptr& s)
>+{
>+  const auto& symbol_f = f->get_symbol();
>+  const auto& symbol_s = s->get_symbol();
>+  if (!symbol_f || !symbol_s)
>+    return false;
>+  return symbol_f->get_namespace() != symbol_s->get_namespace();
>+}
>+
>+/// Test if the current diff tree node carries a namespace change in
>+/// either a function or a variable.
>+///
>+/// @param diff the diff tree node to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+static bool
>+namespace_changed(const diff* diff)
>+{
>+  if (const function_decl_diff* d =
>+	dynamic_cast<const function_decl_diff*>(diff))
>+    return namespace_changed(d->first_function_decl(),
>+			     d->second_function_decl());
>+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
>+    return namespace_changed(d->first_var(), d->second_var());
>+  return false;
>+}
>+
> /// Test if there was a function name change, but there there was no
> /// change in name of the underlying symbol.  IOW, if the name of a
> /// function changed, but the symbol of the new function is equal to
>@@ -1783,7 +1820,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
> 	      || non_static_data_member_added_or_removed(d)
> 	      || base_classes_added_or_removed(d)
> 	      || has_harmful_enum_change(d)
>-	      || crc_changed(d)))
>+	      || crc_changed(d)
>+	      || namespace_changed(d)))
> 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
>
>       if (has_virtual_mem_fn_change(d))
>diff --git a/src/abg-ir.cc b/src/abg-ir.cc
>index 0ef5e8b2..78154622 100644
>--- a/src/abg-ir.cc
>+++ b/src/abg-ir.cc
>@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
>   bool			is_common_;
>   bool			is_in_ksymtab_;
>   uint64_t		crc_;
>+  abg_compat::optional<std::string>	namespace_;
>   bool			is_suppressed_;
>   elf_symbol_wptr	main_symbol_;
>   elf_symbol_wptr	next_alias_;
>@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
>       is_common_(false),
>       is_in_ksymtab_(false),
>       crc_(0),
>+      namespace_(),
>       is_suppressed_(false)
>   {}
>
>@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
>        elf_symbol::visibility	  vi,
>        bool			  is_in_ksymtab,
>        uint64_t			  crc,
>+       const abg_compat::optional<std::string>&	ns,
>        bool			  is_suppressed)
>     : env_(e),
>       index_(i),
>@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
>       is_common_(c),
>       is_in_ksymtab_(is_in_ksymtab),
>       crc_(crc),
>+      namespace_(ns),
>       is_suppressed_(is_suppressed)
>   {
>     if (!is_common_)
>@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
> /// @param vi the visibility of the symbol.
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
>+///
>+/// @param ns the namespace of Linux Kernel symbols, if any
> elf_symbol::elf_symbol(const environment* e,
> 		       size_t		  i,
> 		       size_t		  s,
>@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		       visibility	  vi,
> 		       bool		  is_in_ksymtab,
> 		       uint64_t		  crc,
>+		       const abg_compat::optional<std::string>&	ns,
> 		       bool		  is_suppressed)
>   : priv_(new priv(e,
> 		   i,
>@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		   vi,
> 		   is_in_ksymtab,
> 		   crc,
>+		   ns,
> 		   is_suppressed))
> {}
>
>@@ -1886,6 +1894,8 @@ elf_symbol::create()
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
> ///
>+/// @param ns the namespace of Linux Kernel symbols, if any
>+///
> /// @return a (smart) pointer to a newly created instance of @ref
> /// elf_symbol.
> elf_symbol_sptr
>@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
> 		   visibility	      vi,
> 		   bool		      is_in_ksymtab,
> 		   uint64_t	      crc,
>+		   const abg_compat::optional<std::string>&	ns,
> 		   bool		      is_suppressed)
> {
>   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
>-				     is_in_ksymtab, crc, is_suppressed));
>+				     is_in_ksymtab, crc, ns, is_suppressed));
>   sym->priv_->main_symbol_ = sym;
>   return sym;
> }
>@@ -1927,7 +1938,8 @@ textually_equals(const elf_symbol&l,
> 		 && l.is_common_symbol() == r.is_common_symbol()
> 		 && l.get_version() == r.get_version()
> 		 && (l.get_crc() == 0 || r.get_crc() == 0
>-		     || l.get_crc() == r.get_crc()));
>+		     || l.get_crc() == r.get_crc())
>+		 && l.get_namespace() == r.get_namespace());
>
>   if (equals && l.is_variable())
>     // These are variable symbols.  Let's compare their symbol size.
>@@ -2147,6 +2159,20 @@ void
> elf_symbol::set_crc(uint64_t crc)
> {priv_->crc_ = crc;}
>
>+/// Getter of the 'namespace' property.
>+///
>+/// @return the namespace for Linux Kernel symbols, if any
>+const abg_compat::optional<std::string>&
>+elf_symbol::get_namespace() const
>+{return priv_->namespace_;}
>+
>+/// Setter of the 'namespace' property.
>+///
>+/// @param ns the new namespace for Linux Kernel symbols, if any
>+void
>+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
>+{priv_->namespace_ = ns;}
>+
> /// Getter for the 'is-suppressed' property.
> ///
> /// @return true iff the current symbol has been suppressed by a
>diff --git a/src/abg-reader.cc b/src/abg-reader.cc
>index 31885692..0a8ecd70 100644
>--- a/src/abg-reader.cc
>+++ b/src/abg-reader.cc
>@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
>   if (crc != 0)
>     e->set_crc(crc);
>
>+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
>+    {
>+      std::string ns;
>+      xml::xml_char_sptr_to_string(s, ns);
>+      e->set_namespace(ns);
>+    }
>+
>   return e;
> }
>
>diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
>index 7012f5dc..9ebcdf00 100644
>--- a/src/abg-reporter-priv.cc
>+++ b/src/abg-reporter-priv.cc
>@@ -1158,6 +1158,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
> 	  << std::noshowbase << std::dec
> 	  << "\n";
>     }
>+
>+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
>+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
>+  if (ns1 != ns2)
>+    {
>+      const std::string none = "(none)";
>+      out << indent << "namespace changed from ";
>+      if (ns1.has_value())
>+	out << "'" << ns1.value() << "'";
>+      else
>+	out << none;
>+      out << " to ";
>+      if (ns2.has_value())
>+	out << "'" << ns2.value() << "'";
>+      else
>+	out << none;
>+      out << "\n";
>+    }
> }
>
> /// For a given symbol, emit a string made of its name and version.
>diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
>index b42ce87d..e59bd62c 100644
>--- a/src/abg-symtab-reader.cc
>+++ b/src/abg-symtab-reader.cc
>@@ -232,9 +232,33 @@ symtab::load_(Elf*	       elf_handle,
>       return false;
>     }
>
>+  // The __kstrtab_strings sections is basically an ELF strtab but does not
>+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
>+  // section data.
>+  //
>+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
>+  // within the __kstrtab_strings section. To look up the string value, we need
>+  // to translate from load address to section offset by subtracting the base
>+  // address of the section.
>+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
>+  size_t strings_offset = 0;
>+  const char* strings_data = nullptr;
>+  size_t strings_size = 0;
>+  if (strings_section)
>+    {
>+      GElf_Shdr strings_sheader;
>+      gelf_getshdr(strings_section, &strings_sheader);
>+      strings_offset = strings_sheader.sh_addr;
>+      Elf_Data* data = elf_getdata(strings_section, nullptr);
>+      ABG_ASSERT(data->d_off == 0);
>+      strings_data = reinterpret_cast<const char *>(data->d_buf);
>+      strings_size = data->d_size;
>+    }
>+
>   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
>   std::unordered_set<std::string> exported_kernel_symbols;
>   std::unordered_map<std::string, uint64_t> crc_values;
>+  std::unordered_map<std::string, std::string> namespaces;
>
>   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
>   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
>@@ -288,6 +312,32 @@ symtab::load_(Elf*	       elf_handle,
> 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
> 	  continue;
> 	}
>+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
>+	{
>+	  // This symbol lives in the __ksymtab_strings section but st_value is
>+	  // a load address so, for vmlinux only, we need to subtract an offset
>+	  // before looking it up in that section. Kernel module offsets must
>+	  // not be adjusted. The simple way to distinguish the cases is to see
>+	  // if the adjustment would overflow.

Perhaps we can distinguish that from the ELF type? Kernel modules appear
to be ET_REL. See elf_file_type() in abg-dwarf-reader.cc, a helper that
could also live in abg-elf-helpers.cc actually.

Cheers,
Matthias
>+          const size_t value = sym->st_value;
>+	  const size_t offset =
>+              value < strings_offset ? value : value - strings_offset;
>+	  // check offset
>+	  ABG_ASSERT(offset < strings_size);
>+	  // find the terminating NULL
>+	  const char* first = strings_data + offset;
>+	  const char* last = strings_data + strings_size;
>+	  const char* limit = std::find(first, last, 0);
>+	  // check NULL found
>+	  ABG_ASSERT(limit < last);
>+	  // interpret the empty namespace name as no namespace name
>+	  if (first < limit)
>+	    {
>+	      const std::string ns(first, limit - first);
>+	      ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);
>+	    }
>+	  continue;
>+	}
>
>       // filter out uninteresting entries and only keep functions/variables for
>       // now. The rest might be interesting in the future though.
>@@ -402,6 +452,17 @@ symtab::load_(Elf*	       elf_handle,
> 	symbol->set_crc(crc_entry.second);
>     }
>
>+  // Now add the namespaces
>+  for (const auto& namespace_entry : namespaces)
>+    {
>+      const auto r = name_symbol_map_.find(namespace_entry.first);
>+      if (r == name_symbol_map_.end())
>+	continue;
>+
>+      for (const auto& symbol : r->second)
>+	symbol->set_namespace(namespace_entry.second);
>+    }
>+
>   // sort the symbols for deterministic output
>   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
>
>diff --git a/src/abg-writer.cc b/src/abg-writer.cc
>index 7802128d..692abc75 100644
>--- a/src/abg-writer.cc
>+++ b/src/abg-writer.cc
>@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
>       << std::hex << std::showbase << sym->get_crc() << "'"
>       << std::dec << std::noshowbase;
>
>+  if (sym->get_namespace().has_value())
>+    o << " namespace='" << sym->get_namespace().value() << "'";
>+
>   o << "/>\n";
>
>   return true;
>diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
>index a7eb7ff0..7ea24d58 100644
>--- a/tests/data/Makefile.am
>+++ b/tests/data/Makefile.am
>@@ -91,6 +91,9 @@ test-abidiff/test-crc-0.xml \
> test-abidiff/test-crc-1.xml \
> test-abidiff/test-crc-2.xml \
> test-abidiff/test-crc-report.txt \
>+test-abidiff/test-namespace-0.xml \
>+test-abidiff/test-namespace-1.xml \
>+test-abidiff/test-namespace-report.txt \
> test-abidiff/test-PR27985-report.txt	 \
> test-abidiff/test-PR27985-v0.c		 \
> test-abidiff/test-PR27985-v0.o		 \
>diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
>new file mode 100644
>index 00000000..5a9f5cd2
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-0.xml
>@@ -0,0 +1,15 @@
>+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
>+  <elf-variable-symbols>
>+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
>+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
>+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
>+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
>+  </elf-variable-symbols>
>+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
>+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
>+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
>+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
>+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
>+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
>+  </abi-instr>
>+</abi-corpus>
>diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
>new file mode 100644
>index 00000000..9814844b
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-1.xml
>@@ -0,0 +1,15 @@
>+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
>+  <elf-variable-symbols>
>+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
>+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
>+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
>+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
>+  </elf-variable-symbols>
>+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
>+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
>+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
>+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
>+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
>+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
>+  </abi-instr>
>+</abi-corpus>
>diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
>new file mode 100644
>index 00000000..d2c421ed
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-report.txt
>@@ -0,0 +1,17 @@
>+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
>+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
>+
>+4 Changed variables:
>+
>+  [C] 'int v1' was changed:
>+    namespace changed from (none) to 'this'
>+
>+  [C] 'int v2' was changed:
>+    namespace changed from 'this' to 'that'
>+
>+  [C] 'int v3' was changed:
>+    namespace changed from 'that' to ''
>+
>+  [C] 'int v4' was changed:
>+    namespace changed from '' to (none)
>+
>diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
>index 32858ece..009f7de5 100644
>--- a/tests/test-abidiff.cc
>+++ b/tests/test-abidiff.cc
>@@ -128,6 +128,12 @@ static InOutSpec specs[] =
>     "data/test-abidiff/test-crc-report.txt",
>     "output/test-abidiff/test-crc-report-1-2.txt"
>   },
>+  {
>+    "data/test-abidiff/test-namespace-0.xml",
>+    "data/test-abidiff/test-namespace-1.xml",
>+    "data/test-abidiff/test-namespace-report.txt",
>+    "output/test-abidiff/test-namespace-report-0-1.txt"
>+  },
>   {
>     "data/test-abidiff/test-PR27616-v0.xml",
>     "data/test-abidiff/test-PR27616-v1.xml",
>-- 
>2.35.1.894.gb6a874cedc-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes
  2022-03-16 16:30   ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-03-17 12:01     ` Matthias Maennich
  0 siblings, 0 replies; 37+ messages in thread
From: Matthias Maennich @ 2022-03-17 12:01 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Wed, Mar 16, 2022 at 04:30:55PM +0000, Giuliano Procida wrote:
>The CRC with value zero was used to mean "absent". This can be better
>modelled using optional.
>
>This commit makes this change and also tweaks reporting so that
>disappearing / appearing CRCs are noted. This should be essentially
>impossible unless CRCs are enabled / disabled altogether but would be
>very noteworthy otherwise.
>
>	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
>	now an optional defaulted to absent.
>	(elf_symbol::create): Likewise.
>	(elf_symbol::get_crc): Now returns an optional uint64_t.
>	(elf_symbol::set_src): Now takes an optional uint64_t.
>	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
>	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
>	optional uint64_t.
>	(elf_symbol::priv::priv): Argument crc is now an optional
>	uint64_t.
>	(elf_symbol::elf_symbol): Likewise.
>	(elf_symbol::create): Argument crc is now an optional uint64_t
>	and defaults to absent.
>	(textually_equals): Simplify comparison.
>	(elf_symbol::get_crc): Now returns an optional uint64_t.
>	(elf_symbol::set_crc): Now takes an optional uint64_t.
>	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
>	as other CRC values.
>	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
>	Treat CRC 0 the same as other CRC values and also report
>	changes to CRC presence.
>	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
>	as other CRC values.
>	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
>	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
>	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
>	showing additional of CRCs.
>	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
>	showing removal of CRCs.
>	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
>	from tests/data/test-abidiff/test-crc-report.txt.
>	* tests/test-abidiff.cc: Update test cases that no longer
>	generate empty reports.
>	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>

Reviewed-by: Matthias Maennich <maennich@google.com>

Cheers,
Matthias

>---
> include/abg-ir.h                              |  8 ++++----
> src/abg-comp-filter.cc                        |  4 +---
> src/abg-ir.cc                                 | 19 +++++++++---------
> src/abg-reader.cc                             |  8 ++------
> src/abg-reporter-priv.cc                      | 20 ++++++++++++++-----
> src/abg-writer.cc                             |  6 +++---
> tests/data/Makefile.am                        |  4 +++-
> .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++
> .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++
> ...crc-report.txt => test-crc-report-1-2.txt} |  0
> tests/test-abidiff.cc                         |  6 +++---
> tests/test-symtab.cc                          |  8 ++++----
> 12 files changed, 76 insertions(+), 39 deletions(-)
> create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
> create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
> rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)
>
>diff --git a/include/abg-ir.h b/include/abg-ir.h
>index 7c42bea7..7c558828 100644
>--- a/include/abg-ir.h
>+++ b/include/abg-ir.h
>@@ -921,7 +921,7 @@ private:
> 	     const version&	ve,
> 	     visibility		vi,
> 	     bool		is_in_ksymtab = false,
>-	     uint64_t		crc = 0,
>+	     const abg_compat::optional<uint64_t>&	crc = {},
> 	     const abg_compat::optional<std::string>&	ns = {},
> 	     bool		is_suppressed = false);
>
>@@ -947,7 +947,7 @@ public:
> 	 const version&	    ve,
> 	 visibility	    vi,
> 	 bool		    is_in_ksymtab = false,
>-	 uint64_t	    crc = 0,
>+	 const abg_compat::optional<uint64_t>&		crc = {},
> 	 const abg_compat::optional<std::string>&	ns = {},
> 	 bool		    is_suppressed = false);
>
>@@ -1020,11 +1020,11 @@ public:
>   void
>   set_is_in_ksymtab(bool is_in_ksymtab);
>
>-  uint64_t
>+  const abg_compat::optional<uint64_t>&
>   get_crc() const;
>
>   void
>-  set_crc(uint64_t crc);
>+  set_crc(const abg_compat::optional<uint64_t>& crc);
>
>   const abg_compat::optional<std::string>&
>   get_namespace() const;
>diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
>index c179b6bd..22da5244 100644
>--- a/src/abg-comp-filter.cc
>+++ b/src/abg-comp-filter.cc
>@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f,
>   const auto& symbol_s = s->get_symbol();
>   if (!symbol_f || !symbol_s)
>     return false;
>-  const auto crc_f = symbol_f->get_crc();
>-  const auto crc_s = symbol_s->get_crc();
>-  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
>+  return symbol_f->get_crc() != symbol_s->get_crc();
> }
>
> /// Test if the current diff tree node carries a CRC change in either a
>diff --git a/src/abg-ir.cc b/src/abg-ir.cc
>index 78154622..f90be2e4 100644
>--- a/src/abg-ir.cc
>+++ b/src/abg-ir.cc
>@@ -1727,7 +1727,7 @@ struct elf_symbol::priv
>   //     STT_COMMON definition of that name that has the largest size.
>   bool			is_common_;
>   bool			is_in_ksymtab_;
>-  uint64_t		crc_;
>+  abg_compat::optional<uint64_t>	crc_;
>   abg_compat::optional<std::string>	namespace_;
>   bool			is_suppressed_;
>   elf_symbol_wptr	main_symbol_;
>@@ -1745,7 +1745,7 @@ struct elf_symbol::priv
>       is_defined_(false),
>       is_common_(false),
>       is_in_ksymtab_(false),
>-      crc_(0),
>+      crc_(),
>       namespace_(),
>       is_suppressed_(false)
>   {}
>@@ -1761,7 +1761,7 @@ struct elf_symbol::priv
>        const elf_symbol::version& ve,
>        elf_symbol::visibility	  vi,
>        bool			  is_in_ksymtab,
>-       uint64_t			  crc,
>+       const abg_compat::optional<uint64_t>&	crc,
>        const abg_compat::optional<std::string>&	ns,
>        bool			  is_suppressed)
>     : env_(e),
>@@ -1835,7 +1835,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		       const version&	  ve,
> 		       visibility	  vi,
> 		       bool		  is_in_ksymtab,
>-		       uint64_t		  crc,
>+		       const abg_compat::optional<uint64_t>&	crc,
> 		       const abg_compat::optional<std::string>&	ns,
> 		       bool		  is_suppressed)
>   : priv_(new priv(e,
>@@ -1910,7 +1910,7 @@ elf_symbol::create(const environment* e,
> 		   const version&     ve,
> 		   visibility	      vi,
> 		   bool		      is_in_ksymtab,
>-		   uint64_t	      crc,
>+		   const abg_compat::optional<uint64_t>&	crc,
> 		   const abg_compat::optional<std::string>&	ns,
> 		   bool		      is_suppressed)
> {
>@@ -1937,8 +1937,7 @@ textually_equals(const elf_symbol&l,
> 		 && l.is_defined() == r.is_defined()
> 		 && l.is_common_symbol() == r.is_common_symbol()
> 		 && l.get_version() == r.get_version()
>-		 && (l.get_crc() == 0 || r.get_crc() == 0
>-		     || l.get_crc() == r.get_crc())
>+		 && l.get_crc() == r.get_crc()
> 		 && l.get_namespace() == r.get_namespace());
>
>   if (equals && l.is_variable())
>@@ -2147,8 +2146,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab)
>
> /// Getter of the 'crc' property.
> ///
>-/// @return the CRC (modversions) value for Linux Kernel symbols (if present)
>-uint64_t
>+/// @return the CRC (modversions) value for Linux Kernel symbols, if any
>+const abg_compat::optional<uint64_t>&
> elf_symbol::get_crc() const
> {return priv_->crc_;}
>
>@@ -2156,7 +2155,7 @@ elf_symbol::get_crc() const
> ///
> /// @param crc the new CRC (modversions) value for Linux Kernel symbols
> void
>-elf_symbol::set_crc(uint64_t crc)
>+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
> {priv_->crc_ = crc;}
>
> /// Getter of the 'namespace' property.
>diff --git a/src/abg-reader.cc b/src/abg-reader.cc
>index 0a8ecd70..f9e420f1 100644
>--- a/src/abg-reader.cc
>+++ b/src/abg-reader.cc
>@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
> 	is_default_version = true;
>     }
>
>-  uint64_t crc = 0;
>-  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
>-    crc = strtoull(CHAR_STR(s), NULL, 0);
>-
>   elf_symbol::type type = elf_symbol::NOTYPE_TYPE;
>   read_elf_symbol_type(node, type);
>
>@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
>
>   e->set_is_suppressed(is_suppressed);
>
>-  if (crc != 0)
>-    e->set_crc(crc);
>+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
>+    e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
>
>   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
>     {
>diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
>index 9ebcdf00..9ad733f4 100644
>--- a/src/abg-reporter-priv.cc
>+++ b/src/abg-reporter-priv.cc
>@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
> 	  << "\n";
>     }
>
>-  if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0
>-      && symbol1->get_crc() != symbol2->get_crc())
>+  const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc();
>+  const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc();
>+  if (crc1 != crc2)
>     {
>+      const std::string none = "(none)";
>       out << indent << "CRC (modversions) changed from "
>-	  << std::showbase << std::hex
>-	  << symbol1->get_crc() << " to " << symbol2->get_crc()
>-	  << std::noshowbase << std::dec
>+	  << std::showbase << std::hex;
>+      if (crc1.has_value())
>+	out << crc1.value();
>+      else
>+	out << none;
>+      out << " to ";
>+      if (crc2.has_value())
>+	out << crc2.value();
>+      else
>+	out << none;
>+      out << std::noshowbase << std::dec
> 	  << "\n";
>     }
>
>diff --git a/src/abg-writer.cc b/src/abg-writer.cc
>index 692abc75..03fb658b 100644
>--- a/src/abg-writer.cc
>+++ b/src/abg-writer.cc
>@@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
>   if (sym->is_common_symbol())
>     o << " is-common='yes'";
>
>-  if (sym->get_crc() != 0)
>+  if (sym->get_crc().has_value())
>     o << " crc='"
>-      << std::hex << std::showbase << sym->get_crc() << "'"
>-      << std::dec << std::noshowbase;
>+      << std::hex << std::showbase << sym->get_crc().value()
>+      << std::dec << std::noshowbase << "'";
>
>   if (sym->get_namespace().has_value())
>     o << " namespace='" << sym->get_namespace().value() << "'";
>diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
>index 7ea24d58..7388697b 100644
>--- a/tests/data/Makefile.am
>+++ b/tests/data/Makefile.am
>@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml		\
> test-abidiff/test-crc-0.xml \
> test-abidiff/test-crc-1.xml \
> test-abidiff/test-crc-2.xml \
>-test-abidiff/test-crc-report.txt \
>+test-abidiff/test-crc-report-0-1.txt \
>+test-abidiff/test-crc-report-1-0.txt \
>+test-abidiff/test-crc-report-1-2.txt \
> test-abidiff/test-namespace-0.xml \
> test-abidiff/test-namespace-1.xml \
> test-abidiff/test-namespace-report.txt \
>diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt
>new file mode 100644
>index 00000000..0db42f68
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt
>@@ -0,0 +1,16 @@
>+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
>+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
>+
>+1 function with some indirect sub-type change:
>+
>+  [C] 'function void exported_function()' has some indirect sub-type changes:
>+    CRC (modversions) changed from (none) to 0xe52d5bcf
>+
>+2 Changed variables:
>+
>+  [C] 'int exported_variable' was changed:
>+    CRC (modversions) changed from (none) to 0xee94d699
>+
>+  [C] 'int exported_variable_gpl' was changed:
>+    CRC (modversions) changed from (none) to 0x41336c46
>+
>diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt
>new file mode 100644
>index 00000000..e11f29c1
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt
>@@ -0,0 +1,16 @@
>+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
>+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
>+
>+1 function with some indirect sub-type change:
>+
>+  [C] 'function void exported_function()' has some indirect sub-type changes:
>+    CRC (modversions) changed from 0xe52d5bcf to (none)
>+
>+2 Changed variables:
>+
>+  [C] 'int exported_variable' was changed:
>+    CRC (modversions) changed from 0xee94d699 to (none)
>+
>+  [C] 'int exported_variable_gpl' was changed:
>+    CRC (modversions) changed from 0x41336c46 to (none)
>+
>diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt
>similarity index 100%
>rename from tests/data/test-abidiff/test-crc-report.txt
>rename to tests/data/test-abidiff/test-crc-report-1-2.txt
>diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
>index 009f7de5..93e44d6a 100644
>--- a/tests/test-abidiff.cc
>+++ b/tests/test-abidiff.cc
>@@ -113,19 +113,19 @@ static InOutSpec specs[] =
>   {
>     "data/test-abidiff/test-crc-0.xml",
>     "data/test-abidiff/test-crc-1.xml",
>-    "data/test-abidiff/empty-report.txt",
>+    "data/test-abidiff/test-crc-report-0-1.txt",
>     "output/test-abidiff/test-crc-report-0-1.txt"
>   },
>   {
>     "data/test-abidiff/test-crc-1.xml",
>     "data/test-abidiff/test-crc-0.xml",
>-    "data/test-abidiff/empty-report.txt",
>+    "data/test-abidiff/test-crc-report-1-0.txt",
>     "output/test-abidiff/test-crc-report-1-0.txt"
>   },
>   {
>     "data/test-abidiff/test-crc-1.xml",
>     "data/test-abidiff/test-crc-2.xml",
>-    "data/test-abidiff/test-crc-report.txt",
>+    "data/test-abidiff/test-crc-report-1-2.txt",
>     "output/test-abidiff/test-crc-report-1-2.txt"
>   },
>   {
>diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc
>index 7d7a2df0..85303563 100644
>--- a/tests/test-symtab.cc
>+++ b/tests/test-symtab.cc
>@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]")
>   {
>     const std::string  binary = base_path + "one_of_each.ko";
>     const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2);
>-    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0);
>-    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0);
>-    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0);
>-    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0);
>+    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc());
>+    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc());
>+    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc());
>+    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc());
>   }
> }
>-- 
>2.35.1.894.gb6a874cedc-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v3 0/4] add symbol namespace support, update symbol CRC support
  2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                     ` (3 preceding siblings ...)
  2022-03-16 16:30   ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-03-17 16:38   ` Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
                       ` (4 more replies)
  4 siblings, 5 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

New in v3:

* distinguish vmlinux vs .ko using ET_REL
* more sensible patch order

Giuliano Procida (4):
  crc_changed: eliminate copying of shared_ptr values
  optional: minor improvements
  Linux symbol CRCs: support 0 and report presence changes
  add Linux kernel symbol namespace support

 include/abg-cxx-compat.h                      | 30 +++++++--
 include/abg-ir.h                              | 17 +++--
 src/abg-comp-filter.cc                        | 46 +++++++++++--
 src/abg-ir.cc                                 | 47 +++++++++----
 src/abg-reader.cc                             | 15 +++--
 src/abg-reporter-priv.cc                      | 38 +++++++++--
 src/abg-symtab-reader.cc                      | 66 +++++++++++++++++++
 src/abg-writer.cc                             |  9 ++-
 tests/data/Makefile.am                        |  7 +-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         | 12 +++-
 tests/test-symtab.cc                          |  8 +--
 17 files changed, 327 insertions(+), 47 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
@ 2022-03-17 16:38     ` Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida
                       ` (3 subsequent siblings)
  4 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

As pointed out in a review of similar code, it is possible to avoid
copying a couple of shared pointers in this function, by taking
references instead.

This commit also splits declarations to one per line and removes the
unnecessary parentheses around the return expression.

	* src/abg-comp-filter.cc (crc_changed): Take references to
	avoid std::shared_ptr copying. Split declarations into one per
	line. Remove unnecessary return expression parentheses.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 src/abg-comp-filter.cc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 56251274..f90fdc78 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -230,11 +230,13 @@ static bool
 crc_changed(const function_or_var_decl_sptr& f,
 	    const function_or_var_decl_sptr& s)
 {
-  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc();
-  return (crc_f != 0 && crc_s != 0 && crc_f != crc_s);
+  const auto crc_f = symbol_f->get_crc();
+  const auto crc_s = symbol_s->get_crc();
+  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v3 2/4] optional: minor improvements
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-03-17 16:38     ` Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
                       ` (2 subsequent siblings)
  4 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

This change makes minor improvements to the optional class used with
pre-C++17 compilers.

- adds operator== and operator!=
- adds various missing noexcept (but not constexpr) decorations
- defines operator bool in terms of has_value

Note that some constexpr decorations would require C++17 anyway.

	* include/abg-cxx-compat.h (optional): Add operator== and
	operator!=. Add noexcept decorations. Tweak operator bool.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
index 443905c7..5c5943d0 100644
--- a/include/abg-cxx-compat.h
+++ b/include/abg-cxx-compat.h
@@ -45,7 +45,7 @@ public:
   optional(const T& value) : has_value_(true), value_(value) {}
 
   bool
-  has_value() const
+  has_value() const noexcept
   {
     return has_value_;
   }
@@ -67,19 +67,19 @@ public:
   }
 
   const T&
-  operator*() const
+  operator*() const& noexcept
   { return value_; }
 
   T&
-  operator*()
+  operator*() & noexcept
   { return value_; }
 
   const T*
-  operator->() const
+  operator->() const noexcept
   { return &value_; }
 
   T*
-  operator->()
+  operator->() noexcept
   { return &value_; }
 
   optional&
@@ -90,9 +90,27 @@ public:
     return *this;
   }
 
-  explicit operator bool() const { return has_value_; }
+  explicit operator bool() const noexcept { return has_value(); }
 };
 
+template <typename T, typename U>
+bool
+operator==(const optional<T>& lhs, const optional<U>& rhs)
+{
+  if (!lhs.has_value() && !rhs.has_value())
+    return true;
+  if (!lhs.has_value() || !rhs.has_value())
+    return false;
+  return lhs.value() == rhs.value();
+}
+
+template <typename T, typename U>
+bool
+operator!=(const optional<T>& lhs, const optional<U>& rhs)
+{
+  return !(lhs == rhs);
+}
+
 #endif
 }
 
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida
@ 2022-03-17 16:38     ` Giuliano Procida
  2022-03-17 16:38     ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  4 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

The CRC with value zero was used to mean "absent". This can be better
modelled using optional.

This commit makes this change and also tweaks reporting so that
disappearing / appearing CRCs are noted. This should be essentially
impossible unless CRCs are enabled / disabled altogether but would be
very noteworthy otherwise.

	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
	now an optional defaulted to absent.
	(elf_symbol::create): Likewise.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_src): Now takes an optional uint64_t.
	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
	optional uint64_t.
	(elf_symbol::priv::priv): Argument crc is now an optional
	uint64_t.
	(elf_symbol::elf_symbol): Likewise.
	(elf_symbol::create): Argument crc is now an optional uint64_t
	and defaults to absent.
	(textually_equals): Simplify comparison.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_crc): Now takes an optional uint64_t.
	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
	Treat CRC 0 the same as other CRC values and also report
	changes to CRC presence.
	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
	showing additional of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
	showing removal of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
	from tests/data/test-abidiff/test-crc-report.txt.
	* tests/test-abidiff.cc: Update test cases that no longer
	generate empty reports.
	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  9 +++++----
 src/abg-comp-filter.cc                        |  4 +---
 src/abg-ir.cc                                 | 19 +++++++++---------
 src/abg-reader.cc                             |  8 ++------
 src/abg-reporter-priv.cc                      | 20 ++++++++++++++-----
 src/abg-writer.cc                             |  6 +++---
 tests/data/Makefile.am                        |  4 +++-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/test-abidiff.cc                         |  6 +++---
 tests/test-symtab.cc                          |  8 ++++----
 12 files changed, 77 insertions(+), 39 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)

diff --git a/include/abg-ir.h b/include/abg-ir.h
index a2f4e1a7..b05a8c6f 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -21,6 +21,7 @@
 #include <functional>
 #include <set>
 #include <unordered_map>
+#include "abg-cxx-compat.h"
 #include "abg-fwd.h"
 #include "abg-hash.h"
 #include "abg-traverse.h"
@@ -920,7 +921,7 @@ private:
 	     const version&	ve,
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
-	     uint64_t		crc = 0,
+	     const abg_compat::optional<uint64_t>&	crc = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -945,7 +946,7 @@ public:
 	 const version&	    ve,
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
-	 uint64_t	    crc = 0,
+	 const abg_compat::optional<uint64_t>&		crc = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1017,11 +1018,11 @@ public:
   void
   set_is_in_ksymtab(bool is_in_ksymtab);
 
-  uint64_t
+  const abg_compat::optional<uint64_t>&
   get_crc() const;
 
   void
-  set_crc(uint64_t crc);
+  set_crc(const abg_compat::optional<uint64_t>& crc);
 
   bool
   is_suppressed() const;
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index f90fdc78..31590284 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f,
   const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc();
-  const auto crc_s = symbol_s->get_crc();
-  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
+  return symbol_f->get_crc() != symbol_s->get_crc();
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 0ef5e8b2..853b01ee 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1727,7 +1727,7 @@ struct elf_symbol::priv
   //     STT_COMMON definition of that name that has the largest size.
   bool			is_common_;
   bool			is_in_ksymtab_;
-  uint64_t		crc_;
+  abg_compat::optional<uint64_t>	crc_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1744,7 +1744,7 @@ struct elf_symbol::priv
       is_defined_(false),
       is_common_(false),
       is_in_ksymtab_(false),
-      crc_(0),
+      crc_(),
       is_suppressed_(false)
   {}
 
@@ -1759,7 +1759,7 @@ struct elf_symbol::priv
        const elf_symbol::version& ve,
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
-       uint64_t			  crc,
+       const abg_compat::optional<uint64_t>&	crc,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       const version&	  ve,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
-		       uint64_t		  crc,
+		       const abg_compat::optional<uint64_t>&	crc,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e,
 		   const version&     ve,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
-		   uint64_t	      crc,
+		   const abg_compat::optional<uint64_t>&	crc,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
@@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && (l.get_crc() == 0 || r.get_crc() == 0
-		     || l.get_crc() == r.get_crc()));
+		 && l.get_crc() == r.get_crc());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab)
 
 /// Getter of the 'crc' property.
 ///
-/// @return the CRC (modversions) value for Linux Kernel symbols (if present)
-uint64_t
+/// @return the CRC (modversions) value for Linux Kernel symbols, if any
+const abg_compat::optional<uint64_t>&
 elf_symbol::get_crc() const
 {return priv_->crc_;}
 
@@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const
 ///
 /// @param crc the new CRC (modversions) value for Linux Kernel symbols
 void
-elf_symbol::set_crc(uint64_t crc)
+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
 /// Getter for the 'is-suppressed' property.
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 31885692..7a9ad1f9 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 	is_default_version = true;
     }
 
-  uint64_t crc = 0;
-  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
-    crc = strtoull(CHAR_STR(s), NULL, 0);
-
   elf_symbol::type type = elf_symbol::NOTYPE_TYPE;
   read_elf_symbol_type(node, type);
 
@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 
   e->set_is_suppressed(is_suppressed);
 
-  if (crc != 0)
-    e->set_crc(crc);
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
+    e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
   return e;
 }
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 7012f5dc..f9dd928b 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << "\n";
     }
 
-  if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0
-      && symbol1->get_crc() != symbol2->get_crc())
+  const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc();
+  const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc();
+  if (crc1 != crc2)
     {
+      const std::string none = "(none)";
       out << indent << "CRC (modversions) changed from "
-	  << std::showbase << std::hex
-	  << symbol1->get_crc() << " to " << symbol2->get_crc()
-	  << std::noshowbase << std::dec
+	  << std::showbase << std::hex;
+      if (crc1.has_value())
+	out << crc1.value();
+      else
+	out << none;
+      out << " to ";
+      if (crc2.has_value())
+	out << crc2.value();
+      else
+	out << none;
+      out << std::noshowbase << std::dec
 	  << "\n";
     }
 }
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 7802128d..0a1e7b66 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
   if (sym->is_common_symbol())
     o << " is-common='yes'";
 
-  if (sym->get_crc() != 0)
+  if (sym->get_crc().has_value())
     o << " crc='"
-      << std::hex << std::showbase << sym->get_crc() << "'"
-      << std::dec << std::noshowbase;
+      << std::hex << std::showbase << sym->get_crc().value()
+      << std::dec << std::noshowbase << "'";
 
   o << "/>\n";
 
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index a7eb7ff0..90bd1934 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml		\
 test-abidiff/test-crc-0.xml \
 test-abidiff/test-crc-1.xml \
 test-abidiff/test-crc-2.xml \
-test-abidiff/test-crc-report.txt \
+test-abidiff/test-crc-report-0-1.txt \
+test-abidiff/test-crc-report-1-0.txt \
+test-abidiff/test-crc-report-1-2.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt
new file mode 100644
index 00000000..0db42f68
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from (none) to 0xe52d5bcf
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from (none) to 0xee94d699
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from (none) to 0x41336c46
+
diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt
new file mode 100644
index 00000000..e11f29c1
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from 0xe52d5bcf to (none)
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from 0xee94d699 to (none)
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from 0x41336c46 to (none)
+
diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt
similarity index 100%
rename from tests/data/test-abidiff/test-crc-report.txt
rename to tests/data/test-abidiff/test-crc-report-1-2.txt
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index 32858ece..a02bbe3d 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -113,19 +113,19 @@ static InOutSpec specs[] =
   {
     "data/test-abidiff/test-crc-0.xml",
     "data/test-abidiff/test-crc-1.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-0-1.txt",
     "output/test-abidiff/test-crc-report-0-1.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-0.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-1-0.txt",
     "output/test-abidiff/test-crc-report-1-0.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-2.xml",
-    "data/test-abidiff/test-crc-report.txt",
+    "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
   {
diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc
index 7d7a2df0..85303563 100644
--- a/tests/test-symtab.cc
+++ b/tests/test-symtab.cc
@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]")
   {
     const std::string  binary = base_path + "one_of_each.ko";
     const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2);
-    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0);
-    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0);
+    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc());
+    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc());
   }
 }
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v3 4/4] add Linux kernel symbol namespace support
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                       ` (2 preceding siblings ...)
  2022-03-17 16:38     ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-03-17 16:38     ` Giuliano Procida
  2022-03-21 12:53       ` Matthias Maennich
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  4 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Bug 28954 - add Linux Kernel symbol namespace support

Each Linux kernel symbol can be exported to a specified named
namespace or left in the global (nameless) namespace.

One complexity is that the symbol values which identify a string in
the __ksymtab_strings section must be interpretated differently for
vmlinux and .ko loadable modules as the former has a fixed load
address but the latter are relocatable. For vmlinux, the section base
address needs to be subtracted to obtain a section-relative offset.

The global namespace is explicitly represented as the empty string, at
least when it comes to the value of __kstrtabns_FOO symbols, but the
common interpretation is that such symbols lack an export namespace.

I would rather not have to make use of "empty implies missing" in many
places, so the code here represents namespace as optional<string> and
only the symtab reader cares about empty strings in __ksymtab_strings.

	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
	(elf_symbol::create): Add ns argument.
	(elf_symbol::get_namespace): Declare new function.
	(elf_symbol::set_namespace): Declare new function.
	and set_namespace.
	* src/abg-comp-filter.cc (namespace_changed): Define new
	helper functions.
	(categorize_harmful_diff_node): Also call namespace_changed().
	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
	(elf_symbol::priv::priv): Add namespace_ to initialisers.
	(elf_symbol::elf_symbol): Take new ns argument and pass it to
	priv constructor.
	(elf_symbol::create): Take new ns argument and pass it to
	elf_symbol constructor.
	(elf_symbol::get_namespace): Define new function.
	(elf_symbol::set_namespace): Define new function.
	* src/abg-reader.cc (build_elf_symbol): If namespace
	attribute is present, set symbol namespace.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
	symbol namespaces differ, report this.
	* src/abg-symtab-reader.cc (symtab::load): Get ELF header to
	distinguish vmlinux from .ko. Try to get __ksymtab_strings
	metadata and data. Use these to look up __kstrtabns_FOO
	namespace entries. Set symbol namespace where found.
	* src/abg-writer.cc (write_elf_symbol): Emit namespace
	attribute, if symbol has a namespace.
	* tests/data/Makefile.am: Add new test files.
	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
	* tests/test-abidiff.cc: Add new test case.

Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  8 +++
 src/abg-comp-filter.cc                        | 40 ++++++++++-
 src/abg-ir.cc                                 | 30 ++++++++-
 src/abg-reader.cc                             |  7 ++
 src/abg-reporter-priv.cc                      | 18 +++++
 src/abg-symtab-reader.cc                      | 66 +++++++++++++++++++
 src/abg-writer.cc                             |  3 +
 tests/data/Makefile.am                        |  3 +
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         |  6 ++
 12 files changed, 225 insertions(+), 3 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

diff --git a/include/abg-ir.h b/include/abg-ir.h
index b05a8c6f..7c558828 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -922,6 +922,7 @@ private:
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
 	     const abg_compat::optional<uint64_t>&	crc = {},
+	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -947,6 +948,7 @@ public:
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
 	 const abg_compat::optional<uint64_t>&		crc = {},
+	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1024,6 +1026,12 @@ public:
   void
   set_crc(const abg_compat::optional<uint64_t>& crc);
 
+  const abg_compat::optional<std::string>&
+  get_namespace() const;
+
+  void
+  set_namespace(const abg_compat::optional<std::string>& ns);
+
   bool
   is_suppressed() const;
 
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 31590284..22da5244 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -254,6 +254,43 @@ crc_changed(const diff* diff)
   return false;
 }
 
+/// Test if there was a function or variable namespace change.
+///
+/// @param f the first function or variable to consider.
+///
+/// @param s the second function or variable to consider.
+///
+/// @return true if the test is positive, false otherwise.
+template <typename function_or_var_decl_sptr>
+static bool
+namespace_changed(const function_or_var_decl_sptr& f,
+		  const function_or_var_decl_sptr& s)
+{
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
+  if (!symbol_f || !symbol_s)
+    return false;
+  return symbol_f->get_namespace() != symbol_s->get_namespace();
+}
+
+/// Test if the current diff tree node carries a namespace change in
+/// either a function or a variable.
+///
+/// @param diff the diff tree node to consider.
+///
+/// @return true if the test is positive, false otherwise.
+static bool
+namespace_changed(const diff* diff)
+{
+  if (const function_decl_diff* d =
+	dynamic_cast<const function_decl_diff*>(diff))
+    return namespace_changed(d->first_function_decl(),
+			     d->second_function_decl());
+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
+    return namespace_changed(d->first_var(), d->second_var());
+  return false;
+}
+
 /// Test if there was a function name change, but there there was no
 /// change in name of the underlying symbol.  IOW, if the name of a
 /// function changed, but the symbol of the new function is equal to
@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
 	      || non_static_data_member_added_or_removed(d)
 	      || base_classes_added_or_removed(d)
 	      || has_harmful_enum_change(d)
-	      || crc_changed(d)))
+	      || crc_changed(d)
+	      || namespace_changed(d)))
 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
 
       if (has_virtual_mem_fn_change(d))
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 853b01ee..f90be2e4 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
   bool			is_common_;
   bool			is_in_ksymtab_;
   abg_compat::optional<uint64_t>	crc_;
+  abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
       is_common_(false),
       is_in_ksymtab_(false),
       crc_(),
+      namespace_(),
       is_suppressed_(false)
   {}
 
@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
        const abg_compat::optional<uint64_t>&	crc,
+       const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
       is_common_(c),
       is_in_ksymtab_(is_in_ksymtab),
       crc_(crc),
+      namespace_(ns),
       is_suppressed_(is_suppressed)
   {
     if (!is_common_)
@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
 /// @param vi the visibility of the symbol.
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
+///
+/// @param ns the namespace of Linux Kernel symbols, if any
 elf_symbol::elf_symbol(const environment* e,
 		       size_t		  i,
 		       size_t		  s,
@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
 		       const abg_compat::optional<uint64_t>&	crc,
+		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
 		   vi,
 		   is_in_ksymtab,
 		   crc,
+		   ns,
 		   is_suppressed))
 {}
 
@@ -1886,6 +1894,8 @@ elf_symbol::create()
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
 ///
+/// @param ns the namespace of Linux Kernel symbols, if any
+///
 /// @return a (smart) pointer to a newly created instance of @ref
 /// elf_symbol.
 elf_symbol_sptr
@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
 		   const abg_compat::optional<uint64_t>&	crc,
+		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
-				     is_in_ksymtab, crc, is_suppressed));
+				     is_in_ksymtab, crc, ns, is_suppressed));
   sym->priv_->main_symbol_ = sym;
   return sym;
 }
@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && l.get_crc() == r.get_crc());
+		 && l.get_crc() == r.get_crc()
+		 && l.get_namespace() == r.get_namespace());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2146,6 +2158,20 @@ void
 elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
+/// Getter of the 'namespace' property.
+///
+/// @return the namespace for Linux Kernel symbols, if any
+const abg_compat::optional<std::string>&
+elf_symbol::get_namespace() const
+{return priv_->namespace_;}
+
+/// Setter of the 'namespace' property.
+///
+/// @param ns the new namespace for Linux Kernel symbols, if any
+void
+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
+{priv_->namespace_ = ns;}
+
 /// Getter for the 'is-suppressed' property.
 ///
 /// @return true iff the current symbol has been suppressed by a
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 7a9ad1f9..f9e420f1 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
     e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
+    {
+      std::string ns;
+      xml::xml_char_sptr_to_string(s, ns);
+      e->set_namespace(ns);
+    }
+
   return e;
 }
 
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index f9dd928b..9ad733f4 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
       out << std::noshowbase << std::dec
 	  << "\n";
     }
+
+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
+  if (ns1 != ns2)
+    {
+      const std::string none = "(none)";
+      out << indent << "namespace changed from ";
+      if (ns1.has_value())
+	out << "'" << ns1.value() << "'";
+      else
+	out << none;
+      out << " to ";
+      if (ns2.has_value())
+	out << "'" << ns2.value() << "'";
+      else
+	out << none;
+      out << "\n";
+    }
 }
 
 /// For a given symbol, emit a string made of its name and version.
diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
index b42ce87d..123ef84a 100644
--- a/src/abg-symtab-reader.cc
+++ b/src/abg-symtab-reader.cc
@@ -204,6 +204,13 @@ symtab::load_(Elf*	       elf_handle,
 	      ir::environment* env,
 	      symbol_predicate is_suppressed)
 {
+  GElf_Ehdr ehdr_mem;
+  GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem);
+  if (!header)
+    {
+      std::cerr << "Could not load get ELF header: Skipping symtab load.\n";
+      return false;
+    }
 
   Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle);
   if (!symtab_section)
@@ -232,9 +239,34 @@ symtab::load_(Elf*	       elf_handle,
       return false;
     }
 
+  // The __kstrtab_strings sections is basically an ELF strtab but does not
+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
+  // section data.
+  //
+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
+  // within the __kstrtab_strings section. To look up the string value, we need
+  // to translate from vmlinux load address to section offset by subtracting the
+  // base address of the section. This adjustment is not needed for loadable
+  // modules which are relocatable.
+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
+  size_t strings_offset = 0;
+  const char* strings_data = nullptr;
+  size_t strings_size = 0;
+  if (strings_section)
+    {
+      GElf_Shdr strings_sheader;
+      gelf_getshdr(strings_section, &strings_sheader);
+      strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr;
+      Elf_Data* data = elf_getdata(strings_section, nullptr);
+      ABG_ASSERT(data->d_off == 0);
+      strings_data = reinterpret_cast<const char *>(data->d_buf);
+      strings_size = data->d_size;
+    }
+
   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
   std::unordered_set<std::string> exported_kernel_symbols;
   std::unordered_map<std::string, uint64_t> crc_values;
+  std::unordered_map<std::string, std::string> namespaces;
 
   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
@@ -288,6 +320,29 @@ symtab::load_(Elf*	       elf_handle,
 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
 	  continue;
 	}
+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
+	{
+	  // This symbol lives in the __ksymtab_strings section but st_value may
+	  // be a vmlinux load address so we need to subtract the offset before
+	  // looking it up in that section.
+          const size_t value = sym->st_value;
+	  const size_t offset = value - strings_offset;
+	  // check offset
+	  ABG_ASSERT(offset < strings_size);
+	  // find the terminating NULL
+	  const char* first = strings_data + offset;
+	  const char* last = strings_data + strings_size;
+	  const char* limit = std::find(first, last, 0);
+	  // check NULL found
+	  ABG_ASSERT(limit < last);
+	  // interpret the empty namespace name as no namespace name
+	  if (first < limit)
+	    {
+	      const std::string ns(first, limit - first);
+	      ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);
+	    }
+	  continue;
+	}
 
       // filter out uninteresting entries and only keep functions/variables for
       // now. The rest might be interesting in the future though.
@@ -402,6 +457,17 @@ symtab::load_(Elf*	       elf_handle,
 	symbol->set_crc(crc_entry.second);
     }
 
+  // Now add the namespaces
+  for (const auto& namespace_entry : namespaces)
+    {
+      const auto r = name_symbol_map_.find(namespace_entry.first);
+      if (r == name_symbol_map_.end())
+	continue;
+
+      for (const auto& symbol : r->second)
+	symbol->set_namespace(namespace_entry.second);
+    }
+
   // sort the symbols for deterministic output
   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 0a1e7b66..03fb658b 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
       << std::hex << std::showbase << sym->get_crc().value()
       << std::dec << std::noshowbase << "'";
 
+  if (sym->get_namespace().has_value())
+    o << " namespace='" << sym->get_namespace().value() << "'";
+
   o << "/>\n";
 
   return true;
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index 90bd1934..7388697b 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \
 test-abidiff/test-crc-report-0-1.txt \
 test-abidiff/test-crc-report-1-0.txt \
 test-abidiff/test-crc-report-1-2.txt \
+test-abidiff/test-namespace-0.xml \
+test-abidiff/test-namespace-1.xml \
+test-abidiff/test-namespace-report.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
new file mode 100644
index 00000000..5a9f5cd2
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-0.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
new file mode 100644
index 00000000..9814844b
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-1.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
new file mode 100644
index 00000000..d2c421ed
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-report.txt
@@ -0,0 +1,17 @@
+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
+
+4 Changed variables:
+
+  [C] 'int v1' was changed:
+    namespace changed from (none) to 'this'
+
+  [C] 'int v2' was changed:
+    namespace changed from 'this' to 'that'
+
+  [C] 'int v3' was changed:
+    namespace changed from 'that' to ''
+
+  [C] 'int v4' was changed:
+    namespace changed from '' to (none)
+
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index a02bbe3d..93e44d6a 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -128,6 +128,12 @@ static InOutSpec specs[] =
     "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
+  {
+    "data/test-abidiff/test-namespace-0.xml",
+    "data/test-abidiff/test-namespace-1.xml",
+    "data/test-abidiff/test-namespace-report.txt",
+    "output/test-abidiff/test-namespace-report-0-1.txt"
+  },
   {
     "data/test-abidiff/test-PR27616-v0.xml",
     "data/test-abidiff/test-PR27616-v1.xml",
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v3 4/4] add Linux kernel symbol namespace support
  2022-03-17 16:38     ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-21 12:53       ` Matthias Maennich
  2022-03-21 15:52         ` Giuliano Procida
  0 siblings, 1 reply; 37+ messages in thread
From: Matthias Maennich @ 2022-03-21 12:53 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team

On Thu, Mar 17, 2022 at 04:38:58PM +0000, Giuliano Procida wrote:
>Bug 28954 - add Linux Kernel symbol namespace support
>
>Each Linux kernel symbol can be exported to a specified named
>namespace or left in the global (nameless) namespace.
>
>One complexity is that the symbol values which identify a string in
>the __ksymtab_strings section must be interpretated differently for
>vmlinux and .ko loadable modules as the former has a fixed load
>address but the latter are relocatable. For vmlinux, the section base
>address needs to be subtracted to obtain a section-relative offset.
>
>The global namespace is explicitly represented as the empty string, at
>least when it comes to the value of __kstrtabns_FOO symbols, but the
>common interpretation is that such symbols lack an export namespace.
>
>I would rather not have to make use of "empty implies missing" in many
>places, so the code here represents namespace as optional<string> and
>only the symtab reader cares about empty strings in __ksymtab_strings.
>
>	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
>	(elf_symbol::create): Add ns argument.
>	(elf_symbol::get_namespace): Declare new function.
>	(elf_symbol::set_namespace): Declare new function.
>	and set_namespace.
>	* src/abg-comp-filter.cc (namespace_changed): Define new
>	helper functions.
>	(categorize_harmful_diff_node): Also call namespace_changed().
>	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
>	(elf_symbol::priv::priv): Add namespace_ to initialisers.
>	(elf_symbol::elf_symbol): Take new ns argument and pass it to
>	priv constructor.
>	(elf_symbol::create): Take new ns argument and pass it to
>	elf_symbol constructor.
>	(elf_symbol::get_namespace): Define new function.
>	(elf_symbol::set_namespace): Define new function.
>	* src/abg-reader.cc (build_elf_symbol): If namespace
>	attribute is present, set symbol namespace.
>	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
>	symbol namespaces differ, report this.
>	* src/abg-symtab-reader.cc (symtab::load): Get ELF header to
>	distinguish vmlinux from .ko. Try to get __ksymtab_strings
>	metadata and data. Use these to look up __kstrtabns_FOO
>	namespace entries. Set symbol namespace where found.
>	* src/abg-writer.cc (write_elf_symbol): Emit namespace
>	attribute, if symbol has a namespace.
>	* tests/data/Makefile.am: Add new test files.
>	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
>	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
>	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
>	* tests/test-abidiff.cc: Add new test case.
>
>Signed-off-by: Giuliano Procida <gprocida@google.com>
>---
> include/abg-ir.h                              |  8 +++
> src/abg-comp-filter.cc                        | 40 ++++++++++-
> src/abg-ir.cc                                 | 30 ++++++++-
> src/abg-reader.cc                             |  7 ++
> src/abg-reporter-priv.cc                      | 18 +++++
> src/abg-symtab-reader.cc                      | 66 +++++++++++++++++++
> src/abg-writer.cc                             |  3 +
> tests/data/Makefile.am                        |  3 +
> tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
> tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
> .../test-abidiff/test-namespace-report.txt    | 17 +++++
> tests/test-abidiff.cc                         |  6 ++
> 12 files changed, 225 insertions(+), 3 deletions(-)
> create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
> create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
> create mode 100644 tests/data/test-abidiff/test-namespace-report.txt
>
>diff --git a/include/abg-ir.h b/include/abg-ir.h
>index b05a8c6f..7c558828 100644
>--- a/include/abg-ir.h
>+++ b/include/abg-ir.h
>@@ -922,6 +922,7 @@ private:
> 	     visibility		vi,
> 	     bool		is_in_ksymtab = false,
> 	     const abg_compat::optional<uint64_t>&	crc = {},
>+	     const abg_compat::optional<std::string>&	ns = {},
> 	     bool		is_suppressed = false);
>
>   elf_symbol(const elf_symbol&);
>@@ -947,6 +948,7 @@ public:
> 	 visibility	    vi,
> 	 bool		    is_in_ksymtab = false,
> 	 const abg_compat::optional<uint64_t>&		crc = {},
>+	 const abg_compat::optional<std::string>&	ns = {},
> 	 bool		    is_suppressed = false);
>
>   const environment*
>@@ -1024,6 +1026,12 @@ public:
>   void
>   set_crc(const abg_compat::optional<uint64_t>& crc);
>
>+  const abg_compat::optional<std::string>&
>+  get_namespace() const;
>+
>+  void
>+  set_namespace(const abg_compat::optional<std::string>& ns);
>+
>   bool
>   is_suppressed() const;
>
>diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
>index 31590284..22da5244 100644
>--- a/src/abg-comp-filter.cc
>+++ b/src/abg-comp-filter.cc
>@@ -254,6 +254,43 @@ crc_changed(const diff* diff)
>   return false;
> }
>
>+/// Test if there was a function or variable namespace change.
>+///
>+/// @param f the first function or variable to consider.
>+///
>+/// @param s the second function or variable to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+template <typename function_or_var_decl_sptr>
>+static bool
>+namespace_changed(const function_or_var_decl_sptr& f,
>+		  const function_or_var_decl_sptr& s)
>+{
>+  const auto& symbol_f = f->get_symbol();
>+  const auto& symbol_s = s->get_symbol();
>+  if (!symbol_f || !symbol_s)
>+    return false;
>+  return symbol_f->get_namespace() != symbol_s->get_namespace();
>+}
>+
>+/// Test if the current diff tree node carries a namespace change in
>+/// either a function or a variable.
>+///
>+/// @param diff the diff tree node to consider.
>+///
>+/// @return true if the test is positive, false otherwise.
>+static bool
>+namespace_changed(const diff* diff)
>+{
>+  if (const function_decl_diff* d =
>+	dynamic_cast<const function_decl_diff*>(diff))
>+    return namespace_changed(d->first_function_decl(),
>+			     d->second_function_decl());
>+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
>+    return namespace_changed(d->first_var(), d->second_var());
>+  return false;
>+}
>+
> /// Test if there was a function name change, but there there was no
> /// change in name of the underlying symbol.  IOW, if the name of a
> /// function changed, but the symbol of the new function is equal to
>@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
> 	      || non_static_data_member_added_or_removed(d)
> 	      || base_classes_added_or_removed(d)
> 	      || has_harmful_enum_change(d)
>-	      || crc_changed(d)))
>+	      || crc_changed(d)
>+	      || namespace_changed(d)))
> 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
>
>       if (has_virtual_mem_fn_change(d))
>diff --git a/src/abg-ir.cc b/src/abg-ir.cc
>index 853b01ee..f90be2e4 100644
>--- a/src/abg-ir.cc
>+++ b/src/abg-ir.cc
>@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
>   bool			is_common_;
>   bool			is_in_ksymtab_;
>   abg_compat::optional<uint64_t>	crc_;
>+  abg_compat::optional<std::string>	namespace_;
>   bool			is_suppressed_;
>   elf_symbol_wptr	main_symbol_;
>   elf_symbol_wptr	next_alias_;
>@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
>       is_common_(false),
>       is_in_ksymtab_(false),
>       crc_(),
>+      namespace_(),
>       is_suppressed_(false)
>   {}
>
>@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
>        elf_symbol::visibility	  vi,
>        bool			  is_in_ksymtab,
>        const abg_compat::optional<uint64_t>&	crc,
>+       const abg_compat::optional<std::string>&	ns,
>        bool			  is_suppressed)
>     : env_(e),
>       index_(i),
>@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
>       is_common_(c),
>       is_in_ksymtab_(is_in_ksymtab),
>       crc_(crc),
>+      namespace_(ns),
>       is_suppressed_(is_suppressed)
>   {
>     if (!is_common_)
>@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
> /// @param vi the visibility of the symbol.
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
>+///
>+/// @param ns the namespace of Linux Kernel symbols, if any
> elf_symbol::elf_symbol(const environment* e,
> 		       size_t		  i,
> 		       size_t		  s,
>@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		       visibility	  vi,
> 		       bool		  is_in_ksymtab,
> 		       const abg_compat::optional<uint64_t>&	crc,
>+		       const abg_compat::optional<std::string>&	ns,
> 		       bool		  is_suppressed)
>   : priv_(new priv(e,
> 		   i,
>@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
> 		   vi,
> 		   is_in_ksymtab,
> 		   crc,
>+		   ns,
> 		   is_suppressed))
> {}
>
>@@ -1886,6 +1894,8 @@ elf_symbol::create()
> ///
> /// @param crc the CRC (modversions) value of Linux Kernel symbols
> ///
>+/// @param ns the namespace of Linux Kernel symbols, if any
>+///
> /// @return a (smart) pointer to a newly created instance of @ref
> /// elf_symbol.
> elf_symbol_sptr
>@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
> 		   visibility	      vi,
> 		   bool		      is_in_ksymtab,
> 		   const abg_compat::optional<uint64_t>&	crc,
>+		   const abg_compat::optional<std::string>&	ns,
> 		   bool		      is_suppressed)
> {
>   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
>-				     is_in_ksymtab, crc, is_suppressed));
>+				     is_in_ksymtab, crc, ns, is_suppressed));
>   sym->priv_->main_symbol_ = sym;
>   return sym;
> }
>@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l,
> 		 && l.is_defined() == r.is_defined()
> 		 && l.is_common_symbol() == r.is_common_symbol()
> 		 && l.get_version() == r.get_version()
>-		 && l.get_crc() == r.get_crc());
>+		 && l.get_crc() == r.get_crc()
>+		 && l.get_namespace() == r.get_namespace());
>
>   if (equals && l.is_variable())
>     // These are variable symbols.  Let's compare their symbol size.
>@@ -2146,6 +2158,20 @@ void
> elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
> {priv_->crc_ = crc;}
>
>+/// Getter of the 'namespace' property.
>+///
>+/// @return the namespace for Linux Kernel symbols, if any
>+const abg_compat::optional<std::string>&
>+elf_symbol::get_namespace() const
>+{return priv_->namespace_;}
>+
>+/// Setter of the 'namespace' property.
>+///
>+/// @param ns the new namespace for Linux Kernel symbols, if any
>+void
>+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
>+{priv_->namespace_ = ns;}
>+
> /// Getter for the 'is-suppressed' property.
> ///
> /// @return true iff the current symbol has been suppressed by a
>diff --git a/src/abg-reader.cc b/src/abg-reader.cc
>index 7a9ad1f9..f9e420f1 100644
>--- a/src/abg-reader.cc
>+++ b/src/abg-reader.cc
>@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
>   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
>     e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
>
>+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
>+    {
>+      std::string ns;
>+      xml::xml_char_sptr_to_string(s, ns);
>+      e->set_namespace(ns);
>+    }
>+
>   return e;
> }
>
>diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
>index f9dd928b..9ad733f4 100644
>--- a/src/abg-reporter-priv.cc
>+++ b/src/abg-reporter-priv.cc
>@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
>       out << std::noshowbase << std::dec
> 	  << "\n";
>     }
>+
>+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
>+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
>+  if (ns1 != ns2)
>+    {
>+      const std::string none = "(none)";
>+      out << indent << "namespace changed from ";
>+      if (ns1.has_value())
>+	out << "'" << ns1.value() << "'";
>+      else
>+	out << none;
>+      out << " to ";
>+      if (ns2.has_value())
>+	out << "'" << ns2.value() << "'";
>+      else
>+	out << none;
>+      out << "\n";
>+    }
> }
>
> /// For a given symbol, emit a string made of its name and version.
>diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
>index b42ce87d..123ef84a 100644
>--- a/src/abg-symtab-reader.cc
>+++ b/src/abg-symtab-reader.cc
>@@ -204,6 +204,13 @@ symtab::load_(Elf*	       elf_handle,
> 	      ir::environment* env,
> 	      symbol_predicate is_suppressed)
> {
>+  GElf_Ehdr ehdr_mem;
>+  GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem);
>+  if (!header)
>+    {
>+      std::cerr << "Could not load get ELF header: Skipping symtab load.\n";

nit: "Could not load ELF header: ..."

>+      return false;
>+    }
>
>   Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle);
>   if (!symtab_section)
>@@ -232,9 +239,34 @@ symtab::load_(Elf*	       elf_handle,
>       return false;
>     }
>
>+  // The __kstrtab_strings sections is basically an ELF strtab but does not
>+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
>+  // section data.

raw section data

>+  //
>+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
>+  // within the __kstrtab_strings section. To look up the string value, we need
>+  // to translate from vmlinux load address to section offset by subtracting the
>+  // base address of the section. This adjustment is not needed for loadable
>+  // modules which are relocatable.

... identifiable by ELF type ET_REL.

>+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
>+  size_t strings_offset = 0;
>+  const char* strings_data = nullptr;
>+  size_t strings_size = 0;
>+  if (strings_section)
>+    {
>+      GElf_Shdr strings_sheader;
>+      gelf_getshdr(strings_section, &strings_sheader);
>+      strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr;
>+      Elf_Data* data = elf_getdata(strings_section, nullptr);
>+      ABG_ASSERT(data->d_off == 0);
>+      strings_data = reinterpret_cast<const char *>(data->d_buf);
>+      strings_size = data->d_size;

Since you later assume strings_data and strings_size being valid based
on the existence of strings_section, perhaps assert that the extraction
worked?
        ABG_ASSERT(strings_data);
        ABG_ASSERT(strings_size > 0); // otherwise, why would there be a section?

>+    }
>+
>   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
>   std::unordered_set<std::string> exported_kernel_symbols;
>   std::unordered_map<std::string, uint64_t> crc_values;
>+  std::unordered_map<std::string, std::string> namespaces;
>
>   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
>   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
>@@ -288,6 +320,29 @@ symtab::load_(Elf*	       elf_handle,
> 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
> 	  continue;
> 	}
>+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
>+	{
>+	  // This symbol lives in the __ksymtab_strings section but st_value may
>+	  // be a vmlinux load address so we need to subtract the offset before
>+	  // looking it up in that section.
>+          const size_t value = sym->st_value;

odd indentation

>+	  const size_t offset = value - strings_offset;
>+	  // check offset
>+	  ABG_ASSERT(offset < strings_size);
>+	  // find the terminating NULL
>+	  const char* first = strings_data + offset;
>+	  const char* last = strings_data + strings_size;
>+	  const char* limit = std::find(first, last, 0);
>+	  // check NULL found
>+	  ABG_ASSERT(limit < last);
>+	  // interpret the empty namespace name as no namespace name
>+	  if (first < limit)
>+	    {
>+	      const std::string ns(first, limit - first);
>+	      ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);

           ABG_ASSERT(namespaces.emplace(name.substr(12), std::move(ns)).second);


With the few nits addressed, feel free to add

Reviewed-by: Matthias Maennich <maennich@google.com>

Cheers,
Matthias

>+	    }
>+	  continue;
>+	}
>
>       // filter out uninteresting entries and only keep functions/variables for
>       // now. The rest might be interesting in the future though.
>@@ -402,6 +457,17 @@ symtab::load_(Elf*	       elf_handle,
> 	symbol->set_crc(crc_entry.second);
>     }
>
>+  // Now add the namespaces
>+  for (const auto& namespace_entry : namespaces)
>+    {
>+      const auto r = name_symbol_map_.find(namespace_entry.first);
>+      if (r == name_symbol_map_.end())
>+	continue;
>+
>+      for (const auto& symbol : r->second)
>+	symbol->set_namespace(namespace_entry.second);
>+    }
>+
>   // sort the symbols for deterministic output
>   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
>
>diff --git a/src/abg-writer.cc b/src/abg-writer.cc
>index 0a1e7b66..03fb658b 100644
>--- a/src/abg-writer.cc
>+++ b/src/abg-writer.cc
>@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
>       << std::hex << std::showbase << sym->get_crc().value()
>       << std::dec << std::noshowbase << "'";
>
>+  if (sym->get_namespace().has_value())
>+    o << " namespace='" << sym->get_namespace().value() << "'";
>+
>   o << "/>\n";
>
>   return true;
>diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
>index 90bd1934..7388697b 100644
>--- a/tests/data/Makefile.am
>+++ b/tests/data/Makefile.am
>@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \
> test-abidiff/test-crc-report-0-1.txt \
> test-abidiff/test-crc-report-1-0.txt \
> test-abidiff/test-crc-report-1-2.txt \
>+test-abidiff/test-namespace-0.xml \
>+test-abidiff/test-namespace-1.xml \
>+test-abidiff/test-namespace-report.txt \
> test-abidiff/test-PR27985-report.txt	 \
> test-abidiff/test-PR27985-v0.c		 \
> test-abidiff/test-PR27985-v0.o		 \
>diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
>new file mode 100644
>index 00000000..5a9f5cd2
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-0.xml
>@@ -0,0 +1,15 @@
>+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
>+  <elf-variable-symbols>
>+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
>+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
>+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
>+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
>+  </elf-variable-symbols>
>+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
>+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
>+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
>+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
>+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
>+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
>+  </abi-instr>
>+</abi-corpus>
>diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
>new file mode 100644
>index 00000000..9814844b
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-1.xml
>@@ -0,0 +1,15 @@
>+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
>+  <elf-variable-symbols>
>+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
>+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
>+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
>+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
>+  </elf-variable-symbols>
>+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
>+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
>+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
>+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
>+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
>+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
>+  </abi-instr>
>+</abi-corpus>
>diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
>new file mode 100644
>index 00000000..d2c421ed
>--- /dev/null
>+++ b/tests/data/test-abidiff/test-namespace-report.txt
>@@ -0,0 +1,17 @@
>+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
>+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
>+
>+4 Changed variables:
>+
>+  [C] 'int v1' was changed:
>+    namespace changed from (none) to 'this'
>+
>+  [C] 'int v2' was changed:
>+    namespace changed from 'this' to 'that'
>+
>+  [C] 'int v3' was changed:
>+    namespace changed from 'that' to ''
>+
>+  [C] 'int v4' was changed:
>+    namespace changed from '' to (none)
>+
>diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
>index a02bbe3d..93e44d6a 100644
>--- a/tests/test-abidiff.cc
>+++ b/tests/test-abidiff.cc
>@@ -128,6 +128,12 @@ static InOutSpec specs[] =
>     "data/test-abidiff/test-crc-report-1-2.txt",
>     "output/test-abidiff/test-crc-report-1-2.txt"
>   },
>+  {
>+    "data/test-abidiff/test-namespace-0.xml",
>+    "data/test-abidiff/test-namespace-1.xml",
>+    "data/test-abidiff/test-namespace-report.txt",
>+    "output/test-abidiff/test-namespace-report-0-1.txt"
>+  },
>   {
>     "data/test-abidiff/test-PR27616-v0.xml",
>     "data/test-abidiff/test-PR27616-v1.xml",
>-- 
>2.35.1.894.gb6a874cedc-goog
>

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v3 4/4] add Linux kernel symbol namespace support
  2022-03-21 12:53       ` Matthias Maennich
@ 2022-03-21 15:52         ` Giuliano Procida
  0 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 15:52 UTC (permalink / raw)
  To: Matthias Maennich; +Cc: libabigail, dodji, kernel-team

Hi.

On Mon, 21 Mar 2022 at 12:53, Matthias Maennich <maennich@google.com> wrote:
>
> On Thu, Mar 17, 2022 at 04:38:58PM +0000, Giuliano Procida wrote:
> >Bug 28954 - add Linux Kernel symbol namespace support
> >
> >Each Linux kernel symbol can be exported to a specified named
> >namespace or left in the global (nameless) namespace.
> >
> >One complexity is that the symbol values which identify a string in
> >the __ksymtab_strings section must be interpretated differently for
> >vmlinux and .ko loadable modules as the former has a fixed load
> >address but the latter are relocatable. For vmlinux, the section base
> >address needs to be subtracted to obtain a section-relative offset.
> >
> >The global namespace is explicitly represented as the empty string, at
> >least when it comes to the value of __kstrtabns_FOO symbols, but the
> >common interpretation is that such symbols lack an export namespace.
> >
> >I would rather not have to make use of "empty implies missing" in many
> >places, so the code here represents namespace as optional<string> and
> >only the symtab reader cares about empty strings in __ksymtab_strings.
> >
> >       * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
> >       (elf_symbol::create): Add ns argument.
> >       (elf_symbol::get_namespace): Declare new function.
> >       (elf_symbol::set_namespace): Declare new function.
> >       and set_namespace.
> >       * src/abg-comp-filter.cc (namespace_changed): Define new
> >       helper functions.
> >       (categorize_harmful_diff_node): Also call namespace_changed().
> >       * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
> >       (elf_symbol::priv::priv): Add namespace_ to initialisers.
> >       (elf_symbol::elf_symbol): Take new ns argument and pass it to
> >       priv constructor.
> >       (elf_symbol::create): Take new ns argument and pass it to
> >       elf_symbol constructor.
> >       (elf_symbol::get_namespace): Define new function.
> >       (elf_symbol::set_namespace): Define new function.
> >       * src/abg-reader.cc (build_elf_symbol): If namespace
> >       attribute is present, set symbol namespace.
> >       * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
> >       symbol namespaces differ, report this.
> >       * src/abg-symtab-reader.cc (symtab::load): Get ELF header to
> >       distinguish vmlinux from .ko. Try to get __ksymtab_strings
> >       metadata and data. Use these to look up __kstrtabns_FOO
> >       namespace entries. Set symbol namespace where found.
> >       * src/abg-writer.cc (write_elf_symbol): Emit namespace
> >       attribute, if symbol has a namespace.
> >       * tests/data/Makefile.am: Add new test files.
> >       * tests/data/test-abidiff/test-namespace-0.xml: New test file.
> >       * tests/data/test-abidiff/test-namespace-1.xml: Likewise
> >       * tests/data/test-abidiff/test-namespace-report.txt: Likewise.
> >       * tests/test-abidiff.cc: Add new test case.
> >
> >Signed-off-by: Giuliano Procida <gprocida@google.com>
> >---
> > include/abg-ir.h                              |  8 +++
> > src/abg-comp-filter.cc                        | 40 ++++++++++-
> > src/abg-ir.cc                                 | 30 ++++++++-
> > src/abg-reader.cc                             |  7 ++
> > src/abg-reporter-priv.cc                      | 18 +++++
> > src/abg-symtab-reader.cc                      | 66 +++++++++++++++++++
> > src/abg-writer.cc                             |  3 +
> > tests/data/Makefile.am                        |  3 +
> > tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
> > tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
> > .../test-abidiff/test-namespace-report.txt    | 17 +++++
> > tests/test-abidiff.cc                         |  6 ++
> > 12 files changed, 225 insertions(+), 3 deletions(-)
> > create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
> > create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
> > create mode 100644 tests/data/test-abidiff/test-namespace-report.txt
> >
> >diff --git a/include/abg-ir.h b/include/abg-ir.h
> >index b05a8c6f..7c558828 100644
> >--- a/include/abg-ir.h
> >+++ b/include/abg-ir.h
> >@@ -922,6 +922,7 @@ private:
> >            visibility         vi,
> >            bool               is_in_ksymtab = false,
> >            const abg_compat::optional<uint64_t>&      crc = {},
> >+           const abg_compat::optional<std::string>&   ns = {},
> >            bool               is_suppressed = false);
> >
> >   elf_symbol(const elf_symbol&);
> >@@ -947,6 +948,7 @@ public:
> >        visibility         vi,
> >        bool               is_in_ksymtab = false,
> >        const abg_compat::optional<uint64_t>&          crc = {},
> >+       const abg_compat::optional<std::string>&       ns = {},
> >        bool               is_suppressed = false);
> >
> >   const environment*
> >@@ -1024,6 +1026,12 @@ public:
> >   void
> >   set_crc(const abg_compat::optional<uint64_t>& crc);
> >
> >+  const abg_compat::optional<std::string>&
> >+  get_namespace() const;
> >+
> >+  void
> >+  set_namespace(const abg_compat::optional<std::string>& ns);
> >+
> >   bool
> >   is_suppressed() const;
> >
> >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
> >index 31590284..22da5244 100644
> >--- a/src/abg-comp-filter.cc
> >+++ b/src/abg-comp-filter.cc
> >@@ -254,6 +254,43 @@ crc_changed(const diff* diff)
> >   return false;
> > }
> >
> >+/// Test if there was a function or variable namespace change.
> >+///
> >+/// @param f the first function or variable to consider.
> >+///
> >+/// @param s the second function or variable to consider.
> >+///
> >+/// @return true if the test is positive, false otherwise.
> >+template <typename function_or_var_decl_sptr>
> >+static bool
> >+namespace_changed(const function_or_var_decl_sptr& f,
> >+                const function_or_var_decl_sptr& s)
> >+{
> >+  const auto& symbol_f = f->get_symbol();
> >+  const auto& symbol_s = s->get_symbol();
> >+  if (!symbol_f || !symbol_s)
> >+    return false;
> >+  return symbol_f->get_namespace() != symbol_s->get_namespace();
> >+}
> >+
> >+/// Test if the current diff tree node carries a namespace change in
> >+/// either a function or a variable.
> >+///
> >+/// @param diff the diff tree node to consider.
> >+///
> >+/// @return true if the test is positive, false otherwise.
> >+static bool
> >+namespace_changed(const diff* diff)
> >+{
> >+  if (const function_decl_diff* d =
> >+      dynamic_cast<const function_decl_diff*>(diff))
> >+    return namespace_changed(d->first_function_decl(),
> >+                           d->second_function_decl());
> >+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
> >+    return namespace_changed(d->first_var(), d->second_var());
> >+  return false;
> >+}
> >+
> > /// Test if there was a function name change, but there there was no
> > /// change in name of the underlying symbol.  IOW, if the name of a
> > /// function changed, but the symbol of the new function is equal to
> >@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
> >             || non_static_data_member_added_or_removed(d)
> >             || base_classes_added_or_removed(d)
> >             || has_harmful_enum_change(d)
> >-            || crc_changed(d)))
> >+            || crc_changed(d)
> >+            || namespace_changed(d)))
> >       category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
> >
> >       if (has_virtual_mem_fn_change(d))
> >diff --git a/src/abg-ir.cc b/src/abg-ir.cc
> >index 853b01ee..f90be2e4 100644
> >--- a/src/abg-ir.cc
> >+++ b/src/abg-ir.cc
> >@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
> >   bool                        is_common_;
> >   bool                        is_in_ksymtab_;
> >   abg_compat::optional<uint64_t>      crc_;
> >+  abg_compat::optional<std::string>   namespace_;
> >   bool                        is_suppressed_;
> >   elf_symbol_wptr     main_symbol_;
> >   elf_symbol_wptr     next_alias_;
> >@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
> >       is_common_(false),
> >       is_in_ksymtab_(false),
> >       crc_(),
> >+      namespace_(),
> >       is_suppressed_(false)
> >   {}
> >
> >@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
> >        elf_symbol::visibility   vi,
> >        bool                     is_in_ksymtab,
> >        const abg_compat::optional<uint64_t>&  crc,
> >+       const abg_compat::optional<std::string>&       ns,
> >        bool                     is_suppressed)
> >     : env_(e),
> >       index_(i),
> >@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
> >       is_common_(c),
> >       is_in_ksymtab_(is_in_ksymtab),
> >       crc_(crc),
> >+      namespace_(ns),
> >       is_suppressed_(is_suppressed)
> >   {
> >     if (!is_common_)
> >@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
> > /// @param vi the visibility of the symbol.
> > ///
> > /// @param crc the CRC (modversions) value of Linux Kernel symbols
> >+///
> >+/// @param ns the namespace of Linux Kernel symbols, if any
> > elf_symbol::elf_symbol(const environment* e,
> >                      size_t             i,
> >                      size_t             s,
> >@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
> >                      visibility         vi,
> >                      bool               is_in_ksymtab,
> >                      const abg_compat::optional<uint64_t>&    crc,
> >+                     const abg_compat::optional<std::string>& ns,
> >                      bool               is_suppressed)
> >   : priv_(new priv(e,
> >                  i,
> >@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
> >                  vi,
> >                  is_in_ksymtab,
> >                  crc,
> >+                 ns,
> >                  is_suppressed))
> > {}
> >
> >@@ -1886,6 +1894,8 @@ elf_symbol::create()
> > ///
> > /// @param crc the CRC (modversions) value of Linux Kernel symbols
> > ///
> >+/// @param ns the namespace of Linux Kernel symbols, if any
> >+///
> > /// @return a (smart) pointer to a newly created instance of @ref
> > /// elf_symbol.
> > elf_symbol_sptr
> >@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
> >                  visibility         vi,
> >                  bool               is_in_ksymtab,
> >                  const abg_compat::optional<uint64_t>&        crc,
> >+                 const abg_compat::optional<std::string>&     ns,
> >                  bool               is_suppressed)
> > {
> >   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
> >-                                   is_in_ksymtab, crc, is_suppressed));
> >+                                   is_in_ksymtab, crc, ns, is_suppressed));
> >   sym->priv_->main_symbol_ = sym;
> >   return sym;
> > }
> >@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l,
> >                && l.is_defined() == r.is_defined()
> >                && l.is_common_symbol() == r.is_common_symbol()
> >                && l.get_version() == r.get_version()
> >-               && l.get_crc() == r.get_crc());
> >+               && l.get_crc() == r.get_crc()
> >+               && l.get_namespace() == r.get_namespace());
> >
> >   if (equals && l.is_variable())
> >     // These are variable symbols.  Let's compare their symbol size.
> >@@ -2146,6 +2158,20 @@ void
> > elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
> > {priv_->crc_ = crc;}
> >
> >+/// Getter of the 'namespace' property.
> >+///
> >+/// @return the namespace for Linux Kernel symbols, if any
> >+const abg_compat::optional<std::string>&
> >+elf_symbol::get_namespace() const
> >+{return priv_->namespace_;}
> >+
> >+/// Setter of the 'namespace' property.
> >+///
> >+/// @param ns the new namespace for Linux Kernel symbols, if any
> >+void
> >+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
> >+{priv_->namespace_ = ns;}
> >+
> > /// Getter for the 'is-suppressed' property.
> > ///
> > /// @return true iff the current symbol has been suppressed by a
> >diff --git a/src/abg-reader.cc b/src/abg-reader.cc
> >index 7a9ad1f9..f9e420f1 100644
> >--- a/src/abg-reader.cc
> >+++ b/src/abg-reader.cc
> >@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
> >   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
> >     e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
> >
> >+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
> >+    {
> >+      std::string ns;
> >+      xml::xml_char_sptr_to_string(s, ns);
> >+      e->set_namespace(ns);
> >+    }
> >+
> >   return e;
> > }
> >
> >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
> >index f9dd928b..9ad733f4 100644
> >--- a/src/abg-reporter-priv.cc
> >+++ b/src/abg-reporter-priv.cc
> >@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&     symbol1,
> >       out << std::noshowbase << std::dec
> >         << "\n";
> >     }
> >+
> >+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
> >+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
> >+  if (ns1 != ns2)
> >+    {
> >+      const std::string none = "(none)";
> >+      out << indent << "namespace changed from ";
> >+      if (ns1.has_value())
> >+      out << "'" << ns1.value() << "'";
> >+      else
> >+      out << none;
> >+      out << " to ";
> >+      if (ns2.has_value())
> >+      out << "'" << ns2.value() << "'";
> >+      else
> >+      out << none;
> >+      out << "\n";
> >+    }
> > }
> >
> > /// For a given symbol, emit a string made of its name and version.
> >diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
> >index b42ce87d..123ef84a 100644
> >--- a/src/abg-symtab-reader.cc
> >+++ b/src/abg-symtab-reader.cc
> >@@ -204,6 +204,13 @@ symtab::load_(Elf*               elf_handle,
> >             ir::environment* env,
> >             symbol_predicate is_suppressed)
> > {
> >+  GElf_Ehdr ehdr_mem;
> >+  GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem);
> >+  if (!header)
> >+    {
> >+      std::cerr << "Could not load get ELF header: Skipping symtab load.\n";
>
> nit: "Could not load ELF header: ..."
>
> >+      return false;
> >+    }
> >
> >   Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle);
> >   if (!symtab_section)
> >@@ -232,9 +239,34 @@ symtab::load_(Elf*               elf_handle,
> >       return false;
> >     }
> >
> >+  // The __kstrtab_strings sections is basically an ELF strtab but does not
> >+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
> >+  // section data.
>
> raw section data

Documentation actually refers to "washed" data, so "raw" doesn't sound right.
I'll insert "washed".

>
> >+  //
> >+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
> >+  // within the __kstrtab_strings section. To look up the string value, we need
> >+  // to translate from vmlinux load address to section offset by subtracting the
> >+  // base address of the section. This adjustment is not needed for loadable
> >+  // modules which are relocatable.
>
> ... identifiable by ELF type ET_REL.
>

Done.

> >+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
> >+  size_t strings_offset = 0;
> >+  const char* strings_data = nullptr;
> >+  size_t strings_size = 0;
> >+  if (strings_section)
> >+    {
> >+      GElf_Shdr strings_sheader;
> >+      gelf_getshdr(strings_section, &strings_sheader);
> >+      strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr;
> >+      Elf_Data* data = elf_getdata(strings_section, nullptr);
> >+      ABG_ASSERT(data->d_off == 0);
> >+      strings_data = reinterpret_cast<const char *>(data->d_buf);
> >+      strings_size = data->d_size;
>
> Since you later assume strings_data and strings_size being valid based
> on the existence of strings_section, perhaps assert that the extraction
> worked?
>         ABG_ASSERT(strings_data);
>         ABG_ASSERT(strings_size > 0); // otherwise, why would there be a section?
>

If the section is empty, null and 0 are expected. Later offset bounds
checking will assert if any lookups are actually attempted.
Any checking here is at best redundant and at worst (very
theoretically) premature.

If strings_data is outside mapped memory and strings_size is non-zero
that will SEGV. I don't think it's worth doing special checks for the
various strings_size == 0 possibilities.

> >+    }
> >+
> >   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
> >   std::unordered_set<std::string> exported_kernel_symbols;
> >   std::unordered_map<std::string, uint64_t> crc_values;
> >+  std::unordered_map<std::string, std::string> namespaces;
> >
> >   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
> >   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
> >@@ -288,6 +320,29 @@ symtab::load_(Elf*               elf_handle,
> >         ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
> >         continue;
> >       }
> >+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
> >+      {
> >+        // This symbol lives in the __ksymtab_strings section but st_value may
> >+        // be a vmlinux load address so we need to subtract the offset before
> >+        // looking it up in that section.
> >+          const size_t value = sym->st_value;
>
> odd indentation
>

Done (tabification).

> >+        const size_t offset = value - strings_offset;
> >+        // check offset
> >+        ABG_ASSERT(offset < strings_size);
> >+        // find the terminating NULL
> >+        const char* first = strings_data + offset;
> >+        const char* last = strings_data + strings_size;
> >+        const char* limit = std::find(first, last, 0);
> >+        // check NULL found
> >+        ABG_ASSERT(limit < last);
> >+        // interpret the empty namespace name as no namespace name
> >+        if (first < limit)
> >+          {
> >+            const std::string ns(first, limit - first);
> >+            ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second);
>
>            ABG_ASSERT(namespaces.emplace(name.substr(12), std::move(ns)).second);
>

Done (inlined).

>
> With the few nits addressed, feel free to add
>
> Reviewed-by: Matthias Maennich <maennich@google.com>
>
> Cheers,
> Matthias
>

Thanks!

Giuliano.

> >+          }
> >+        continue;
> >+      }
> >
> >       // filter out uninteresting entries and only keep functions/variables for
> >       // now. The rest might be interesting in the future though.
> >@@ -402,6 +457,17 @@ symtab::load_(Elf*               elf_handle,
> >       symbol->set_crc(crc_entry.second);
> >     }
> >
> >+  // Now add the namespaces
> >+  for (const auto& namespace_entry : namespaces)
> >+    {
> >+      const auto r = name_symbol_map_.find(namespace_entry.first);
> >+      if (r == name_symbol_map_.end())
> >+      continue;
> >+
> >+      for (const auto& symbol : r->second)
> >+      symbol->set_namespace(namespace_entry.second);
> >+    }
> >+
> >   // sort the symbols for deterministic output
> >   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
> >
> >diff --git a/src/abg-writer.cc b/src/abg-writer.cc
> >index 0a1e7b66..03fb658b 100644
> >--- a/src/abg-writer.cc
> >+++ b/src/abg-writer.cc
> >@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&  sym,
> >       << std::hex << std::showbase << sym->get_crc().value()
> >       << std::dec << std::noshowbase << "'";
> >
> >+  if (sym->get_namespace().has_value())
> >+    o << " namespace='" << sym->get_namespace().value() << "'";
> >+
> >   o << "/>\n";
> >
> >   return true;
> >diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
> >index 90bd1934..7388697b 100644
> >--- a/tests/data/Makefile.am
> >+++ b/tests/data/Makefile.am
> >@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \
> > test-abidiff/test-crc-report-0-1.txt \
> > test-abidiff/test-crc-report-1-0.txt \
> > test-abidiff/test-crc-report-1-2.txt \
> >+test-abidiff/test-namespace-0.xml \
> >+test-abidiff/test-namespace-1.xml \
> >+test-abidiff/test-namespace-report.txt \
> > test-abidiff/test-PR27985-report.txt   \
> > test-abidiff/test-PR27985-v0.c                 \
> > test-abidiff/test-PR27985-v0.o                 \
> >diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
> >new file mode 100644
> >index 00000000..5a9f5cd2
> >--- /dev/null
> >+++ b/tests/data/test-abidiff/test-namespace-0.xml
> >@@ -0,0 +1,15 @@
> >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
> >+  <elf-variable-symbols>
> >+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
> >+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
> >+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
> >+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
> >+  </elf-variable-symbols>
> >+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
> >+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
> >+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
> >+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
> >+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
> >+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
> >+  </abi-instr>
> >+</abi-corpus>
> >diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
> >new file mode 100644
> >index 00000000..9814844b
> >--- /dev/null
> >+++ b/tests/data/test-abidiff/test-namespace-1.xml
> >@@ -0,0 +1,15 @@
> >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
> >+  <elf-variable-symbols>
> >+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
> >+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
> >+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
> >+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
> >+  </elf-variable-symbols>
> >+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
> >+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
> >+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
> >+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
> >+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
> >+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
> >+  </abi-instr>
> >+</abi-corpus>
> >diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
> >new file mode 100644
> >index 00000000..d2c421ed
> >--- /dev/null
> >+++ b/tests/data/test-abidiff/test-namespace-report.txt
> >@@ -0,0 +1,17 @@
> >+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
> >+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
> >+
> >+4 Changed variables:
> >+
> >+  [C] 'int v1' was changed:
> >+    namespace changed from (none) to 'this'
> >+
> >+  [C] 'int v2' was changed:
> >+    namespace changed from 'this' to 'that'
> >+
> >+  [C] 'int v3' was changed:
> >+    namespace changed from 'that' to ''
> >+
> >+  [C] 'int v4' was changed:
> >+    namespace changed from '' to (none)
> >+
> >diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
> >index a02bbe3d..93e44d6a 100644
> >--- a/tests/test-abidiff.cc
> >+++ b/tests/test-abidiff.cc
> >@@ -128,6 +128,12 @@ static InOutSpec specs[] =
> >     "data/test-abidiff/test-crc-report-1-2.txt",
> >     "output/test-abidiff/test-crc-report-1-2.txt"
> >   },
> >+  {
> >+    "data/test-abidiff/test-namespace-0.xml",
> >+    "data/test-abidiff/test-namespace-1.xml",
> >+    "data/test-abidiff/test-namespace-report.txt",
> >+    "output/test-abidiff/test-namespace-report-0-1.txt"
> >+  },
> >   {
> >     "data/test-abidiff/test-PR27616-v0.xml",
> >     "data/test-abidiff/test-PR27616-v1.xml",
> >--
> >2.35.1.894.gb6a874cedc-goog
> >

^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v4 0/4] add symbol namespace support, update symbol CRC support
  2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                       ` (3 preceding siblings ...)
  2022-03-17 16:38     ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-03-21 16:02     ` Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
                         ` (8 more replies)
  4 siblings, 9 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

New in v4:

* address review comments on symtab reader changes

Giuliano Procida (4):
  crc_changed: eliminate copying of shared_ptr values
  optional: minor improvements
  Linux symbol CRCs: support 0 and report presence changes
  add Linux kernel symbol namespace support

 include/abg-cxx-compat.h                      | 30 +++++++--
 include/abg-ir.h                              | 17 +++--
 src/abg-comp-filter.cc                        | 46 +++++++++++--
 src/abg-ir.cc                                 | 47 ++++++++++----
 src/abg-reader.cc                             | 15 +++--
 src/abg-reporter-priv.cc                      | 38 +++++++++--
 src/abg-symtab-reader.cc                      | 64 +++++++++++++++++++
 src/abg-writer.cc                             |  9 ++-
 tests/data/Makefile.am                        |  7 +-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         | 12 +++-
 tests/test-symtab.cc                          |  8 +--
 17 files changed, 325 insertions(+), 47 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
@ 2022-03-21 16:02       ` Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida
                         ` (7 subsequent siblings)
  8 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

As pointed out in a review of similar code, it is possible to avoid
copying a couple of shared pointers in this function, by taking
references instead.

This commit also splits declarations to one per line and removes the
unnecessary parentheses around the return expression.

	* src/abg-comp-filter.cc (crc_changed): Take references to
	avoid std::shared_ptr copying. Split declarations into one per
	line. Remove unnecessary return expression parentheses.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 src/abg-comp-filter.cc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 56251274..f90fdc78 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -230,11 +230,13 @@ static bool
 crc_changed(const function_or_var_decl_sptr& f,
 	    const function_or_var_decl_sptr& s)
 {
-  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc();
-  return (crc_f != 0 && crc_s != 0 && crc_f != crc_s);
+  const auto crc_f = symbol_f->get_crc();
+  const auto crc_s = symbol_s->get_crc();
+  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v4 2/4] optional: minor improvements
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-03-21 16:02       ` Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
                         ` (6 subsequent siblings)
  8 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

This change makes minor improvements to the optional class used with
pre-C++17 compilers.

- adds operator== and operator!=
- adds various missing noexcept (but not constexpr) decorations
- defines operator bool in terms of has_value

Note that some constexpr decorations would require C++17 anyway.

	* include/abg-cxx-compat.h (optional): Add operator== and
	operator!=. Add noexcept decorations. Tweak operator bool.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
index 443905c7..5c5943d0 100644
--- a/include/abg-cxx-compat.h
+++ b/include/abg-cxx-compat.h
@@ -45,7 +45,7 @@ public:
   optional(const T& value) : has_value_(true), value_(value) {}
 
   bool
-  has_value() const
+  has_value() const noexcept
   {
     return has_value_;
   }
@@ -67,19 +67,19 @@ public:
   }
 
   const T&
-  operator*() const
+  operator*() const& noexcept
   { return value_; }
 
   T&
-  operator*()
+  operator*() & noexcept
   { return value_; }
 
   const T*
-  operator->() const
+  operator->() const noexcept
   { return &value_; }
 
   T*
-  operator->()
+  operator->() noexcept
   { return &value_; }
 
   optional&
@@ -90,9 +90,27 @@ public:
     return *this;
   }
 
-  explicit operator bool() const { return has_value_; }
+  explicit operator bool() const noexcept { return has_value(); }
 };
 
+template <typename T, typename U>
+bool
+operator==(const optional<T>& lhs, const optional<U>& rhs)
+{
+  if (!lhs.has_value() && !rhs.has_value())
+    return true;
+  if (!lhs.has_value() || !rhs.has_value())
+    return false;
+  return lhs.value() == rhs.value();
+}
+
+template <typename T, typename U>
+bool
+operator!=(const optional<T>& lhs, const optional<U>& rhs)
+{
+  return !(lhs == rhs);
+}
+
 #endif
 }
 
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida
@ 2022-03-21 16:02       ` Giuliano Procida
  2022-03-21 16:02       ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida
                         ` (5 subsequent siblings)
  8 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

The CRC with value zero was used to mean "absent". This can be better
modelled using optional.

This commit makes this change and also tweaks reporting so that
disappearing / appearing CRCs are noted. This should be essentially
impossible unless CRCs are enabled / disabled altogether but would be
very noteworthy otherwise.

	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
	now an optional defaulted to absent.
	(elf_symbol::create): Likewise.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_src): Now takes an optional uint64_t.
	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
	optional uint64_t.
	(elf_symbol::priv::priv): Argument crc is now an optional
	uint64_t.
	(elf_symbol::elf_symbol): Likewise.
	(elf_symbol::create): Argument crc is now an optional uint64_t
	and defaults to absent.
	(textually_equals): Simplify comparison.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_crc): Now takes an optional uint64_t.
	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
	Treat CRC 0 the same as other CRC values and also report
	changes to CRC presence.
	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
	showing additional of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
	showing removal of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
	from tests/data/test-abidiff/test-crc-report.txt.
	* tests/test-abidiff.cc: Update test cases that no longer
	generate empty reports.
	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  9 +++++----
 src/abg-comp-filter.cc                        |  4 +---
 src/abg-ir.cc                                 | 19 +++++++++---------
 src/abg-reader.cc                             |  8 ++------
 src/abg-reporter-priv.cc                      | 20 ++++++++++++++-----
 src/abg-writer.cc                             |  6 +++---
 tests/data/Makefile.am                        |  4 +++-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/test-abidiff.cc                         |  6 +++---
 tests/test-symtab.cc                          |  8 ++++----
 12 files changed, 77 insertions(+), 39 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)

diff --git a/include/abg-ir.h b/include/abg-ir.h
index a2f4e1a7..b05a8c6f 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -21,6 +21,7 @@
 #include <functional>
 #include <set>
 #include <unordered_map>
+#include "abg-cxx-compat.h"
 #include "abg-fwd.h"
 #include "abg-hash.h"
 #include "abg-traverse.h"
@@ -920,7 +921,7 @@ private:
 	     const version&	ve,
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
-	     uint64_t		crc = 0,
+	     const abg_compat::optional<uint64_t>&	crc = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -945,7 +946,7 @@ public:
 	 const version&	    ve,
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
-	 uint64_t	    crc = 0,
+	 const abg_compat::optional<uint64_t>&		crc = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1017,11 +1018,11 @@ public:
   void
   set_is_in_ksymtab(bool is_in_ksymtab);
 
-  uint64_t
+  const abg_compat::optional<uint64_t>&
   get_crc() const;
 
   void
-  set_crc(uint64_t crc);
+  set_crc(const abg_compat::optional<uint64_t>& crc);
 
   bool
   is_suppressed() const;
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index f90fdc78..31590284 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f,
   const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc();
-  const auto crc_s = symbol_s->get_crc();
-  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
+  return symbol_f->get_crc() != symbol_s->get_crc();
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 0ef5e8b2..853b01ee 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1727,7 +1727,7 @@ struct elf_symbol::priv
   //     STT_COMMON definition of that name that has the largest size.
   bool			is_common_;
   bool			is_in_ksymtab_;
-  uint64_t		crc_;
+  abg_compat::optional<uint64_t>	crc_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1744,7 +1744,7 @@ struct elf_symbol::priv
       is_defined_(false),
       is_common_(false),
       is_in_ksymtab_(false),
-      crc_(0),
+      crc_(),
       is_suppressed_(false)
   {}
 
@@ -1759,7 +1759,7 @@ struct elf_symbol::priv
        const elf_symbol::version& ve,
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
-       uint64_t			  crc,
+       const abg_compat::optional<uint64_t>&	crc,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       const version&	  ve,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
-		       uint64_t		  crc,
+		       const abg_compat::optional<uint64_t>&	crc,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e,
 		   const version&     ve,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
-		   uint64_t	      crc,
+		   const abg_compat::optional<uint64_t>&	crc,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
@@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && (l.get_crc() == 0 || r.get_crc() == 0
-		     || l.get_crc() == r.get_crc()));
+		 && l.get_crc() == r.get_crc());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab)
 
 /// Getter of the 'crc' property.
 ///
-/// @return the CRC (modversions) value for Linux Kernel symbols (if present)
-uint64_t
+/// @return the CRC (modversions) value for Linux Kernel symbols, if any
+const abg_compat::optional<uint64_t>&
 elf_symbol::get_crc() const
 {return priv_->crc_;}
 
@@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const
 ///
 /// @param crc the new CRC (modversions) value for Linux Kernel symbols
 void
-elf_symbol::set_crc(uint64_t crc)
+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
 /// Getter for the 'is-suppressed' property.
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 31885692..7a9ad1f9 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 	is_default_version = true;
     }
 
-  uint64_t crc = 0;
-  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
-    crc = strtoull(CHAR_STR(s), NULL, 0);
-
   elf_symbol::type type = elf_symbol::NOTYPE_TYPE;
   read_elf_symbol_type(node, type);
 
@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 
   e->set_is_suppressed(is_suppressed);
 
-  if (crc != 0)
-    e->set_crc(crc);
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
+    e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
   return e;
 }
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 7012f5dc..f9dd928b 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << "\n";
     }
 
-  if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0
-      && symbol1->get_crc() != symbol2->get_crc())
+  const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc();
+  const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc();
+  if (crc1 != crc2)
     {
+      const std::string none = "(none)";
       out << indent << "CRC (modversions) changed from "
-	  << std::showbase << std::hex
-	  << symbol1->get_crc() << " to " << symbol2->get_crc()
-	  << std::noshowbase << std::dec
+	  << std::showbase << std::hex;
+      if (crc1.has_value())
+	out << crc1.value();
+      else
+	out << none;
+      out << " to ";
+      if (crc2.has_value())
+	out << crc2.value();
+      else
+	out << none;
+      out << std::noshowbase << std::dec
 	  << "\n";
     }
 }
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 7802128d..0a1e7b66 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
   if (sym->is_common_symbol())
     o << " is-common='yes'";
 
-  if (sym->get_crc() != 0)
+  if (sym->get_crc().has_value())
     o << " crc='"
-      << std::hex << std::showbase << sym->get_crc() << "'"
-      << std::dec << std::noshowbase;
+      << std::hex << std::showbase << sym->get_crc().value()
+      << std::dec << std::noshowbase << "'";
 
   o << "/>\n";
 
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index a7eb7ff0..90bd1934 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml		\
 test-abidiff/test-crc-0.xml \
 test-abidiff/test-crc-1.xml \
 test-abidiff/test-crc-2.xml \
-test-abidiff/test-crc-report.txt \
+test-abidiff/test-crc-report-0-1.txt \
+test-abidiff/test-crc-report-1-0.txt \
+test-abidiff/test-crc-report-1-2.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt
new file mode 100644
index 00000000..0db42f68
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from (none) to 0xe52d5bcf
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from (none) to 0xee94d699
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from (none) to 0x41336c46
+
diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt
new file mode 100644
index 00000000..e11f29c1
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from 0xe52d5bcf to (none)
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from 0xee94d699 to (none)
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from 0x41336c46 to (none)
+
diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt
similarity index 100%
rename from tests/data/test-abidiff/test-crc-report.txt
rename to tests/data/test-abidiff/test-crc-report-1-2.txt
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index 32858ece..a02bbe3d 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -113,19 +113,19 @@ static InOutSpec specs[] =
   {
     "data/test-abidiff/test-crc-0.xml",
     "data/test-abidiff/test-crc-1.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-0-1.txt",
     "output/test-abidiff/test-crc-report-0-1.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-0.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-1-0.txt",
     "output/test-abidiff/test-crc-report-1-0.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-2.xml",
-    "data/test-abidiff/test-crc-report.txt",
+    "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
   {
diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc
index 7d7a2df0..85303563 100644
--- a/tests/test-symtab.cc
+++ b/tests/test-symtab.cc
@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]")
   {
     const std::string  binary = base_path + "one_of_each.ko";
     const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2);
-    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0);
-    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0);
+    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc());
+    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc());
   }
 }
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v4 4/4] add Linux kernel symbol namespace support
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (2 preceding siblings ...)
  2022-03-21 16:02       ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-03-21 16:02       ` Giuliano Procida
  2022-06-13 14:25       ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (4 subsequent siblings)
  8 siblings, 0 replies; 37+ messages in thread
From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Bug 28954 - add Linux Kernel symbol namespace support

Each Linux kernel symbol can be exported to a specified named
namespace or left in the global (nameless) namespace.

One complexity is that the symbol values which identify a string in
the __ksymtab_strings section must be interpretated differently for
vmlinux and .ko loadable modules as the former has a fixed load
address but the latter are relocatable. For vmlinux, the section base
address needs to be subtracted to obtain a section-relative offset.

The global namespace is explicitly represented as the empty string, at
least when it comes to the value of __kstrtabns_FOO symbols, but the
common interpretation is that such symbols lack an export namespace.

I would rather not have to make use of "empty implies missing" in many
places, so the code here represents namespace as optional<string> and
only the symtab reader cares about empty strings in __ksymtab_strings.

	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
	(elf_symbol::create): Add ns argument.
	(elf_symbol::get_namespace): Declare new function.
	(elf_symbol::set_namespace): Declare new function.
	and set_namespace.
	* src/abg-comp-filter.cc (namespace_changed): Define new
	helper functions.
	(categorize_harmful_diff_node): Also call namespace_changed().
	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
	(elf_symbol::priv::priv): Add namespace_ to initialisers.
	(elf_symbol::elf_symbol): Take new ns argument and pass it to
	priv constructor.
	(elf_symbol::create): Take new ns argument and pass it to
	elf_symbol constructor.
	(elf_symbol::get_namespace): Define new function.
	(elf_symbol::set_namespace): Define new function.
	* src/abg-reader.cc (build_elf_symbol): If namespace
	attribute is present, set symbol namespace.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
	symbol namespaces differ, report this.
	* src/abg-symtab-reader.cc (symtab::load): Get ELF header to
	distinguish vmlinux from .ko. Try to get __ksymtab_strings
	metadata and data. Use these to look up __kstrtabns_FOO
	namespace entries. Set symbol namespace where found.
	* src/abg-writer.cc (write_elf_symbol): Emit namespace
	attribute, if symbol has a namespace.
	* tests/data/Makefile.am: Add new test files.
	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
	* tests/test-abidiff.cc: Add new test case.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  8 +++
 src/abg-comp-filter.cc                        | 40 +++++++++++-
 src/abg-ir.cc                                 | 30 ++++++++-
 src/abg-reader.cc                             |  7 ++
 src/abg-reporter-priv.cc                      | 18 ++++++
 src/abg-symtab-reader.cc                      | 64 +++++++++++++++++++
 src/abg-writer.cc                             |  3 +
 tests/data/Makefile.am                        |  3 +
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         |  6 ++
 12 files changed, 223 insertions(+), 3 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

diff --git a/include/abg-ir.h b/include/abg-ir.h
index b05a8c6f..7c558828 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -922,6 +922,7 @@ private:
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
 	     const abg_compat::optional<uint64_t>&	crc = {},
+	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -947,6 +948,7 @@ public:
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
 	 const abg_compat::optional<uint64_t>&		crc = {},
+	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1024,6 +1026,12 @@ public:
   void
   set_crc(const abg_compat::optional<uint64_t>& crc);
 
+  const abg_compat::optional<std::string>&
+  get_namespace() const;
+
+  void
+  set_namespace(const abg_compat::optional<std::string>& ns);
+
   bool
   is_suppressed() const;
 
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 31590284..22da5244 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -254,6 +254,43 @@ crc_changed(const diff* diff)
   return false;
 }
 
+/// Test if there was a function or variable namespace change.
+///
+/// @param f the first function or variable to consider.
+///
+/// @param s the second function or variable to consider.
+///
+/// @return true if the test is positive, false otherwise.
+template <typename function_or_var_decl_sptr>
+static bool
+namespace_changed(const function_or_var_decl_sptr& f,
+		  const function_or_var_decl_sptr& s)
+{
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
+  if (!symbol_f || !symbol_s)
+    return false;
+  return symbol_f->get_namespace() != symbol_s->get_namespace();
+}
+
+/// Test if the current diff tree node carries a namespace change in
+/// either a function or a variable.
+///
+/// @param diff the diff tree node to consider.
+///
+/// @return true if the test is positive, false otherwise.
+static bool
+namespace_changed(const diff* diff)
+{
+  if (const function_decl_diff* d =
+	dynamic_cast<const function_decl_diff*>(diff))
+    return namespace_changed(d->first_function_decl(),
+			     d->second_function_decl());
+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
+    return namespace_changed(d->first_var(), d->second_var());
+  return false;
+}
+
 /// Test if there was a function name change, but there there was no
 /// change in name of the underlying symbol.  IOW, if the name of a
 /// function changed, but the symbol of the new function is equal to
@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
 	      || non_static_data_member_added_or_removed(d)
 	      || base_classes_added_or_removed(d)
 	      || has_harmful_enum_change(d)
-	      || crc_changed(d)))
+	      || crc_changed(d)
+	      || namespace_changed(d)))
 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
 
       if (has_virtual_mem_fn_change(d))
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 853b01ee..f90be2e4 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
   bool			is_common_;
   bool			is_in_ksymtab_;
   abg_compat::optional<uint64_t>	crc_;
+  abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
       is_common_(false),
       is_in_ksymtab_(false),
       crc_(),
+      namespace_(),
       is_suppressed_(false)
   {}
 
@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
        const abg_compat::optional<uint64_t>&	crc,
+       const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
       is_common_(c),
       is_in_ksymtab_(is_in_ksymtab),
       crc_(crc),
+      namespace_(ns),
       is_suppressed_(is_suppressed)
   {
     if (!is_common_)
@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
 /// @param vi the visibility of the symbol.
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
+///
+/// @param ns the namespace of Linux Kernel symbols, if any
 elf_symbol::elf_symbol(const environment* e,
 		       size_t		  i,
 		       size_t		  s,
@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
 		       const abg_compat::optional<uint64_t>&	crc,
+		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
 		   vi,
 		   is_in_ksymtab,
 		   crc,
+		   ns,
 		   is_suppressed))
 {}
 
@@ -1886,6 +1894,8 @@ elf_symbol::create()
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
 ///
+/// @param ns the namespace of Linux Kernel symbols, if any
+///
 /// @return a (smart) pointer to a newly created instance of @ref
 /// elf_symbol.
 elf_symbol_sptr
@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
 		   const abg_compat::optional<uint64_t>&	crc,
+		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
-				     is_in_ksymtab, crc, is_suppressed));
+				     is_in_ksymtab, crc, ns, is_suppressed));
   sym->priv_->main_symbol_ = sym;
   return sym;
 }
@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && l.get_crc() == r.get_crc());
+		 && l.get_crc() == r.get_crc()
+		 && l.get_namespace() == r.get_namespace());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2146,6 +2158,20 @@ void
 elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
+/// Getter of the 'namespace' property.
+///
+/// @return the namespace for Linux Kernel symbols, if any
+const abg_compat::optional<std::string>&
+elf_symbol::get_namespace() const
+{return priv_->namespace_;}
+
+/// Setter of the 'namespace' property.
+///
+/// @param ns the new namespace for Linux Kernel symbols, if any
+void
+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
+{priv_->namespace_ = ns;}
+
 /// Getter for the 'is-suppressed' property.
 ///
 /// @return true iff the current symbol has been suppressed by a
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 7a9ad1f9..f9e420f1 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
     e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
+    {
+      std::string ns;
+      xml::xml_char_sptr_to_string(s, ns);
+      e->set_namespace(ns);
+    }
+
   return e;
 }
 
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index f9dd928b..9ad733f4 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
       out << std::noshowbase << std::dec
 	  << "\n";
     }
+
+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
+  if (ns1 != ns2)
+    {
+      const std::string none = "(none)";
+      out << indent << "namespace changed from ";
+      if (ns1.has_value())
+	out << "'" << ns1.value() << "'";
+      else
+	out << none;
+      out << " to ";
+      if (ns2.has_value())
+	out << "'" << ns2.value() << "'";
+      else
+	out << none;
+      out << "\n";
+    }
 }
 
 /// For a given symbol, emit a string made of its name and version.
diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
index b42ce87d..92509054 100644
--- a/src/abg-symtab-reader.cc
+++ b/src/abg-symtab-reader.cc
@@ -204,6 +204,13 @@ symtab::load_(Elf*	       elf_handle,
 	      ir::environment* env,
 	      symbol_predicate is_suppressed)
 {
+  GElf_Ehdr ehdr_mem;
+  GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem);
+  if (!header)
+    {
+      std::cerr << "Could not load get ELF header: Skipping symtab load.\n";
+      return false;
+    }
 
   Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle);
   if (!symtab_section)
@@ -232,9 +239,34 @@ symtab::load_(Elf*	       elf_handle,
       return false;
     }
 
+  // The __kstrtab_strings sections is basically an ELF strtab but does not
+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
+  // washed section data.
+  //
+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
+  // within the __kstrtab_strings section. To look up the string value, we need
+  // to translate from vmlinux load address to section offset by subtracting the
+  // base address of the section. This adjustment is not needed for loadable
+  // modules which are relocatable and so identifiable by ELF type ET_REL.
+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
+  size_t strings_offset = 0;
+  const char* strings_data = nullptr;
+  size_t strings_size = 0;
+  if (strings_section)
+    {
+      GElf_Shdr strings_sheader;
+      gelf_getshdr(strings_section, &strings_sheader);
+      strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr;
+      Elf_Data* data = elf_getdata(strings_section, nullptr);
+      ABG_ASSERT(data->d_off == 0);
+      strings_data = reinterpret_cast<const char *>(data->d_buf);
+      strings_size = data->d_size;
+    }
+
   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
   std::unordered_set<std::string> exported_kernel_symbols;
   std::unordered_map<std::string, uint64_t> crc_values;
+  std::unordered_map<std::string, std::string> namespaces;
 
   const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle);
   const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle);
@@ -288,6 +320,27 @@ symtab::load_(Elf*	       elf_handle,
 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
 	  continue;
 	}
+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
+	{
+	  // This symbol lives in the __ksymtab_strings section but st_value may
+	  // be a vmlinux load address so we need to subtract the offset before
+	  // looking it up in that section.
+	  const size_t value = sym->st_value;
+	  const size_t offset = value - strings_offset;
+	  // check offset
+	  ABG_ASSERT(offset < strings_size);
+	  // find the terminating NULL
+	  const char* first = strings_data + offset;
+	  const char* last = strings_data + strings_size;
+	  const char* limit = std::find(first, last, 0);
+	  // check NULL found
+	  ABG_ASSERT(limit < last);
+	  // interpret the empty namespace name as no namespace name
+	  if (first < limit)
+	    ABG_ASSERT(namespaces.emplace(
+		name.substr(12), std::string(first, limit - first)).second);
+	  continue;
+	}
 
       // filter out uninteresting entries and only keep functions/variables for
       // now. The rest might be interesting in the future though.
@@ -402,6 +455,17 @@ symtab::load_(Elf*	       elf_handle,
 	symbol->set_crc(crc_entry.second);
     }
 
+  // Now add the namespaces
+  for (const auto& namespace_entry : namespaces)
+    {
+      const auto r = name_symbol_map_.find(namespace_entry.first);
+      if (r == name_symbol_map_.end())
+	continue;
+
+      for (const auto& symbol : r->second)
+	symbol->set_namespace(namespace_entry.second);
+    }
+
   // sort the symbols for deterministic output
   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 0a1e7b66..03fb658b 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
       << std::hex << std::showbase << sym->get_crc().value()
       << std::dec << std::noshowbase << "'";
 
+  if (sym->get_namespace().has_value())
+    o << " namespace='" << sym->get_namespace().value() << "'";
+
   o << "/>\n";
 
   return true;
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index 90bd1934..7388697b 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \
 test-abidiff/test-crc-report-0-1.txt \
 test-abidiff/test-crc-report-1-0.txt \
 test-abidiff/test-crc-report-1-2.txt \
+test-abidiff/test-namespace-0.xml \
+test-abidiff/test-namespace-1.xml \
+test-abidiff/test-namespace-report.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
new file mode 100644
index 00000000..5a9f5cd2
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-0.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
new file mode 100644
index 00000000..9814844b
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-1.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
new file mode 100644
index 00000000..d2c421ed
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-report.txt
@@ -0,0 +1,17 @@
+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
+
+4 Changed variables:
+
+  [C] 'int v1' was changed:
+    namespace changed from (none) to 'this'
+
+  [C] 'int v2' was changed:
+    namespace changed from 'this' to 'that'
+
+  [C] 'int v3' was changed:
+    namespace changed from 'that' to ''
+
+  [C] 'int v4' was changed:
+    namespace changed from '' to (none)
+
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index a02bbe3d..93e44d6a 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -128,6 +128,12 @@ static InOutSpec specs[] =
     "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
+  {
+    "data/test-abidiff/test-namespace-0.xml",
+    "data/test-abidiff/test-namespace-1.xml",
+    "data/test-abidiff/test-namespace-report.txt",
+    "output/test-abidiff/test-namespace-report-0-1.txt"
+  },
   {
     "data/test-abidiff/test-PR27616-v0.xml",
     "data/test-abidiff/test-PR27616-v1.xml",
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v5 0/4] add symbol namespace support, update symbol CRC support
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (3 preceding siblings ...)
  2022-03-21 16:02       ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-06-13 14:25       ` Giuliano Procida
  2022-06-30 16:39         ` Dodji Seketeli
  2022-06-13 14:25       ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
                         ` (3 subsequent siblings)
  8 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Hi Dodji.

I'd be grateful if could take a look at this patch series. Patch 4/4
in particular addresses Bug 28954.

Thank you!

Regards,
Giuliano.

New in v5:

* improve error message when ELF header not found

Giuliano Procida (4):
  crc_changed: eliminate copying of shared_ptr values
  optional: minor improvements
  Linux symbol CRCs: support 0 and report presence changes
  add Linux kernel symbol namespace support

 include/abg-cxx-compat.h                      | 30 +++++++--
 include/abg-ir.h                              | 17 +++--
 src/abg-comp-filter.cc                        | 46 +++++++++++--
 src/abg-ir.cc                                 | 47 ++++++++++----
 src/abg-reader.cc                             | 15 +++--
 src/abg-reporter-priv.cc                      | 38 +++++++++--
 src/abg-symtab-reader.cc                      | 64 +++++++++++++++++++
 src/abg-writer.cc                             |  9 ++-
 tests/data/Makefile.am                        |  7 +-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         | 12 +++-
 tests/test-symtab.cc                          |  8 +--
 17 files changed, 325 insertions(+), 47 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

-- 
2.36.1.476.g0c4daa206d-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (4 preceding siblings ...)
  2022-06-13 14:25       ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
@ 2022-06-13 14:25       ` Giuliano Procida
  2022-06-30 16:40         ` Dodji Seketeli
  2022-06-13 14:25       ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida
                         ` (2 subsequent siblings)
  8 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

As pointed out in a review of similar code, it is possible to avoid
copying a couple of shared pointers in this function, by taking
references instead.

This commit also splits declarations to one per line and removes the
unnecessary parentheses around the return expression.

	* src/abg-comp-filter.cc (crc_changed): Take references to
	avoid std::shared_ptr copying. Split declarations into one per
	line. Remove unnecessary return expression parentheses.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 src/abg-comp-filter.cc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 56251274..f90fdc78 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -230,11 +230,13 @@ static bool
 crc_changed(const function_or_var_decl_sptr& f,
 	    const function_or_var_decl_sptr& s)
 {
-  const auto symbol_f  = f->get_symbol(), symbol_s = s->get_symbol();
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc();
-  return (crc_f != 0 && crc_s != 0 && crc_f != crc_s);
+  const auto crc_f = symbol_f->get_crc();
+  const auto crc_s = symbol_s->get_crc();
+  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
-- 
2.36.1.476.g0c4daa206d-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v5 2/4] optional: minor improvements
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (5 preceding siblings ...)
  2022-06-13 14:25       ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-06-13 14:25       ` Giuliano Procida
  2022-06-30 16:40         ` Dodji Seketeli
  2022-06-13 14:25       ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
  2022-06-13 14:25       ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida
  8 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

This change makes minor improvements to the optional class used with
pre-C++17 compilers.

- adds operator== and operator!=
- adds various missing noexcept (but not constexpr) decorations
- defines operator bool in terms of has_value

Note that some constexpr decorations would require C++17 anyway.

	* include/abg-cxx-compat.h (optional): Add operator== and
	operator!=. Add noexcept decorations. Tweak operator bool.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h
index 443905c7..5c5943d0 100644
--- a/include/abg-cxx-compat.h
+++ b/include/abg-cxx-compat.h
@@ -45,7 +45,7 @@ public:
   optional(const T& value) : has_value_(true), value_(value) {}
 
   bool
-  has_value() const
+  has_value() const noexcept
   {
     return has_value_;
   }
@@ -67,19 +67,19 @@ public:
   }
 
   const T&
-  operator*() const
+  operator*() const& noexcept
   { return value_; }
 
   T&
-  operator*()
+  operator*() & noexcept
   { return value_; }
 
   const T*
-  operator->() const
+  operator->() const noexcept
   { return &value_; }
 
   T*
-  operator->()
+  operator->() noexcept
   { return &value_; }
 
   optional&
@@ -90,9 +90,27 @@ public:
     return *this;
   }
 
-  explicit operator bool() const { return has_value_; }
+  explicit operator bool() const noexcept { return has_value(); }
 };
 
+template <typename T, typename U>
+bool
+operator==(const optional<T>& lhs, const optional<U>& rhs)
+{
+  if (!lhs.has_value() && !rhs.has_value())
+    return true;
+  if (!lhs.has_value() || !rhs.has_value())
+    return false;
+  return lhs.value() == rhs.value();
+}
+
+template <typename T, typename U>
+bool
+operator!=(const optional<T>& lhs, const optional<U>& rhs)
+{
+  return !(lhs == rhs);
+}
+
 #endif
 }
 
-- 
2.36.1.476.g0c4daa206d-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (6 preceding siblings ...)
  2022-06-13 14:25       ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida
@ 2022-06-13 14:25       ` Giuliano Procida
  2022-06-30 16:41         ` Dodji Seketeli
  2022-06-13 14:25       ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida
  8 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

The CRC with value zero was used to mean "absent". This can be better
modelled using optional.

This commit makes this change and also tweaks reporting so that
disappearing / appearing CRCs are noted. This should be essentially
impossible unless CRCs are enabled / disabled altogether but would be
very noteworthy otherwise.

	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
	now an optional defaulted to absent.
	(elf_symbol::create): Likewise.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_src): Now takes an optional uint64_t.
	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
	optional uint64_t.
	(elf_symbol::priv::priv): Argument crc is now an optional
	uint64_t.
	(elf_symbol::elf_symbol): Likewise.
	(elf_symbol::create): Argument crc is now an optional uint64_t
	and defaults to absent.
	(textually_equals): Simplify comparison.
	(elf_symbol::get_crc): Now returns an optional uint64_t.
	(elf_symbol::set_crc): Now takes an optional uint64_t.
	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
	Treat CRC 0 the same as other CRC values and also report
	changes to CRC presence.
	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
	as other CRC values.
	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
	showing additional of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
	showing removal of CRCs.
	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
	from tests/data/test-abidiff/test-crc-report.txt.
	* tests/test-abidiff.cc: Update test cases that no longer
	generate empty reports.
	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  9 +++++----
 src/abg-comp-filter.cc                        |  4 +---
 src/abg-ir.cc                                 | 19 +++++++++---------
 src/abg-reader.cc                             |  8 ++------
 src/abg-reporter-priv.cc                      | 20 ++++++++++++++-----
 src/abg-writer.cc                             |  6 +++---
 tests/data/Makefile.am                        |  4 +++-
 .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++
 .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++
 ...crc-report.txt => test-crc-report-1-2.txt} |  0
 tests/test-abidiff.cc                         |  6 +++---
 tests/test-symtab.cc                          |  8 ++++----
 12 files changed, 77 insertions(+), 39 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt
 create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt
 rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%)

diff --git a/include/abg-ir.h b/include/abg-ir.h
index a2f4e1a7..b05a8c6f 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -21,6 +21,7 @@
 #include <functional>
 #include <set>
 #include <unordered_map>
+#include "abg-cxx-compat.h"
 #include "abg-fwd.h"
 #include "abg-hash.h"
 #include "abg-traverse.h"
@@ -920,7 +921,7 @@ private:
 	     const version&	ve,
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
-	     uint64_t		crc = 0,
+	     const abg_compat::optional<uint64_t>&	crc = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -945,7 +946,7 @@ public:
 	 const version&	    ve,
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
-	 uint64_t	    crc = 0,
+	 const abg_compat::optional<uint64_t>&		crc = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1017,11 +1018,11 @@ public:
   void
   set_is_in_ksymtab(bool is_in_ksymtab);
 
-  uint64_t
+  const abg_compat::optional<uint64_t>&
   get_crc() const;
 
   void
-  set_crc(uint64_t crc);
+  set_crc(const abg_compat::optional<uint64_t>& crc);
 
   bool
   is_suppressed() const;
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index f90fdc78..31590284 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f,
   const auto& symbol_s = s->get_symbol();
   if (!symbol_f || !symbol_s)
     return false;
-  const auto crc_f = symbol_f->get_crc();
-  const auto crc_s = symbol_s->get_crc();
-  return crc_f != 0 && crc_s != 0 && crc_f != crc_s;
+  return symbol_f->get_crc() != symbol_s->get_crc();
 }
 
 /// Test if the current diff tree node carries a CRC change in either a
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 4e907620..34ae486f 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1727,7 +1727,7 @@ struct elf_symbol::priv
   //     STT_COMMON definition of that name that has the largest size.
   bool			is_common_;
   bool			is_in_ksymtab_;
-  uint64_t		crc_;
+  abg_compat::optional<uint64_t>	crc_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1744,7 +1744,7 @@ struct elf_symbol::priv
       is_defined_(false),
       is_common_(false),
       is_in_ksymtab_(false),
-      crc_(0),
+      crc_(),
       is_suppressed_(false)
   {}
 
@@ -1759,7 +1759,7 @@ struct elf_symbol::priv
        const elf_symbol::version& ve,
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
-       uint64_t			  crc,
+       const abg_compat::optional<uint64_t>&	crc,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       const version&	  ve,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
-		       uint64_t		  crc,
+		       const abg_compat::optional<uint64_t>&	crc,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e,
 		   const version&     ve,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
-		   uint64_t	      crc,
+		   const abg_compat::optional<uint64_t>&	crc,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
@@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && (l.get_crc() == 0 || r.get_crc() == 0
-		     || l.get_crc() == r.get_crc()));
+		 && l.get_crc() == r.get_crc());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab)
 
 /// Getter of the 'crc' property.
 ///
-/// @return the CRC (modversions) value for Linux Kernel symbols (if present)
-uint64_t
+/// @return the CRC (modversions) value for Linux Kernel symbols, if any
+const abg_compat::optional<uint64_t>&
 elf_symbol::get_crc() const
 {return priv_->crc_;}
 
@@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const
 ///
 /// @param crc the new CRC (modversions) value for Linux Kernel symbols
 void
-elf_symbol::set_crc(uint64_t crc)
+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
 /// Getter for the 'is-suppressed' property.
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 31885692..7a9ad1f9 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 	is_default_version = true;
     }
 
-  uint64_t crc = 0;
-  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
-    crc = strtoull(CHAR_STR(s), NULL, 0);
-
   elf_symbol::type type = elf_symbol::NOTYPE_TYPE;
   read_elf_symbol_type(node, type);
 
@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
 
   e->set_is_suppressed(is_suppressed);
 
-  if (crc != 0)
-    e->set_crc(crc);
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
+    e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
   return e;
 }
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index 06145a1b..d000620b 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
 	  << "\n";
     }
 
-  if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0
-      && symbol1->get_crc() != symbol2->get_crc())
+  const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc();
+  const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc();
+  if (crc1 != crc2)
     {
+      const std::string none = "(none)";
       out << indent << "CRC (modversions) changed from "
-	  << std::showbase << std::hex
-	  << symbol1->get_crc() << " to " << symbol2->get_crc()
-	  << std::noshowbase << std::dec
+	  << std::showbase << std::hex;
+      if (crc1.has_value())
+	out << crc1.value();
+      else
+	out << none;
+      out << " to ";
+      if (crc2.has_value())
+	out << crc2.value();
+      else
+	out << none;
+      out << std::noshowbase << std::dec
 	  << "\n";
     }
 }
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 3f50a900..23598ef0 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3070,10 +3070,10 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
   if (sym->is_common_symbol())
     o << " is-common='yes'";
 
-  if (sym->get_crc() != 0)
+  if (sym->get_crc().has_value())
     o << " crc='"
-      << std::hex << std::showbase << sym->get_crc() << "'"
-      << std::dec << std::noshowbase;
+      << std::hex << std::showbase << sym->get_crc().value()
+      << std::dec << std::noshowbase << "'";
 
   o << "/>\n";
 
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index a0c086d9..be342684 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml		\
 test-abidiff/test-crc-0.xml \
 test-abidiff/test-crc-1.xml \
 test-abidiff/test-crc-2.xml \
-test-abidiff/test-crc-report.txt \
+test-abidiff/test-crc-report-0-1.txt \
+test-abidiff/test-crc-report-1-0.txt \
+test-abidiff/test-crc-report-1-2.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt
new file mode 100644
index 00000000..0db42f68
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from (none) to 0xe52d5bcf
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from (none) to 0xee94d699
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from (none) to 0x41336c46
+
diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt
new file mode 100644
index 00000000..e11f29c1
--- /dev/null
+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt
@@ -0,0 +1,16 @@
+Functions changes summary: 0 Removed, 1 Changed, 0 Added function
+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables
+
+1 function with some indirect sub-type change:
+
+  [C] 'function void exported_function()' has some indirect sub-type changes:
+    CRC (modversions) changed from 0xe52d5bcf to (none)
+
+2 Changed variables:
+
+  [C] 'int exported_variable' was changed:
+    CRC (modversions) changed from 0xee94d699 to (none)
+
+  [C] 'int exported_variable_gpl' was changed:
+    CRC (modversions) changed from 0x41336c46 to (none)
+
diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt
similarity index 100%
rename from tests/data/test-abidiff/test-crc-report.txt
rename to tests/data/test-abidiff/test-crc-report-1-2.txt
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index 32858ece..a02bbe3d 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -113,19 +113,19 @@ static InOutSpec specs[] =
   {
     "data/test-abidiff/test-crc-0.xml",
     "data/test-abidiff/test-crc-1.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-0-1.txt",
     "output/test-abidiff/test-crc-report-0-1.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-0.xml",
-    "data/test-abidiff/empty-report.txt",
+    "data/test-abidiff/test-crc-report-1-0.txt",
     "output/test-abidiff/test-crc-report-1-0.txt"
   },
   {
     "data/test-abidiff/test-crc-1.xml",
     "data/test-abidiff/test-crc-2.xml",
-    "data/test-abidiff/test-crc-report.txt",
+    "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
   {
diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc
index 7d7a2df0..85303563 100644
--- a/tests/test-symtab.cc
+++ b/tests/test-symtab.cc
@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]")
   {
     const std::string  binary = base_path + "one_of_each.ko";
     const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2);
-    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0);
-    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0);
-    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0);
+    CHECK(corpus->lookup_function_symbol("exported_function")->get_crc());
+    CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc());
+    CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc());
   }
 }
-- 
2.36.1.476.g0c4daa206d-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* [PATCH v5 4/4] add Linux kernel symbol namespace support
  2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
                         ` (7 preceding siblings ...)
  2022-06-13 14:25       ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-06-13 14:25       ` Giuliano Procida
  2022-07-01 12:54         ` Dodji Seketeli
  8 siblings, 1 reply; 37+ messages in thread
From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw)
  To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich

Bug 28954 - add Linux Kernel symbol namespace support

Each Linux kernel symbol can be exported to a specified named
namespace or left in the global (nameless) namespace.

One complexity is that the symbol values which identify a string in
the __ksymtab_strings section must be interpretated differently for
vmlinux and .ko loadable modules as the former has a fixed load
address but the latter are relocatable. For vmlinux, the section base
address needs to be subtracted to obtain a section-relative offset.

The global namespace is explicitly represented as the empty string, at
least when it comes to the value of __kstrtabns_FOO symbols, but the
common interpretation is that such symbols lack an export namespace.

I would rather not have to make use of "empty implies missing" in many
places, so the code here represents namespace as optional<string> and
only the symtab reader cares about empty strings in __ksymtab_strings.

	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
	(elf_symbol::create): Add ns argument.
	(elf_symbol::get_namespace): Declare new function.
	(elf_symbol::set_namespace): Declare new function.
	and set_namespace.
	* src/abg-comp-filter.cc (namespace_changed): Define new
	helper functions.
	(categorize_harmful_diff_node): Also call namespace_changed().
	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
	(elf_symbol::priv::priv): Add namespace_ to initialisers.
	(elf_symbol::elf_symbol): Take new ns argument and pass it to
	priv constructor.
	(elf_symbol::create): Take new ns argument and pass it to
	elf_symbol constructor.
	(elf_symbol::get_namespace): Define new function.
	(elf_symbol::set_namespace): Define new function.
	* src/abg-reader.cc (build_elf_symbol): If namespace
	attribute is present, set symbol namespace.
	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
	symbol namespaces differ, report this.
	* src/abg-symtab-reader.cc (symtab::load): Get ELF header to
	distinguish vmlinux from .ko. Try to get __ksymtab_strings
	metadata and data. Use these to look up __kstrtabns_FOO
	namespace entries. Set symbol namespace where found.
	* src/abg-writer.cc (write_elf_symbol): Emit namespace
	attribute, if symbol has a namespace.
	* tests/data/Makefile.am: Add new test files.
	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
	* tests/test-abidiff.cc: Add new test case.

Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Giuliano Procida <gprocida@google.com>
---
 include/abg-ir.h                              |  8 +++
 src/abg-comp-filter.cc                        | 40 +++++++++++-
 src/abg-ir.cc                                 | 30 ++++++++-
 src/abg-reader.cc                             |  7 ++
 src/abg-reporter-priv.cc                      | 18 ++++++
 src/abg-symtab-reader.cc                      | 64 +++++++++++++++++++
 src/abg-writer.cc                             |  3 +
 tests/data/Makefile.am                        |  3 +
 tests/data/test-abidiff/test-namespace-0.xml  | 15 +++++
 tests/data/test-abidiff/test-namespace-1.xml  | 15 +++++
 .../test-abidiff/test-namespace-report.txt    | 17 +++++
 tests/test-abidiff.cc                         |  6 ++
 12 files changed, 223 insertions(+), 3 deletions(-)
 create mode 100644 tests/data/test-abidiff/test-namespace-0.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-1.xml
 create mode 100644 tests/data/test-abidiff/test-namespace-report.txt

diff --git a/include/abg-ir.h b/include/abg-ir.h
index b05a8c6f..7c558828 100644
--- a/include/abg-ir.h
+++ b/include/abg-ir.h
@@ -922,6 +922,7 @@ private:
 	     visibility		vi,
 	     bool		is_in_ksymtab = false,
 	     const abg_compat::optional<uint64_t>&	crc = {},
+	     const abg_compat::optional<std::string>&	ns = {},
 	     bool		is_suppressed = false);
 
   elf_symbol(const elf_symbol&);
@@ -947,6 +948,7 @@ public:
 	 visibility	    vi,
 	 bool		    is_in_ksymtab = false,
 	 const abg_compat::optional<uint64_t>&		crc = {},
+	 const abg_compat::optional<std::string>&	ns = {},
 	 bool		    is_suppressed = false);
 
   const environment*
@@ -1024,6 +1026,12 @@ public:
   void
   set_crc(const abg_compat::optional<uint64_t>& crc);
 
+  const abg_compat::optional<std::string>&
+  get_namespace() const;
+
+  void
+  set_namespace(const abg_compat::optional<std::string>& ns);
+
   bool
   is_suppressed() const;
 
diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc
index 31590284..22da5244 100644
--- a/src/abg-comp-filter.cc
+++ b/src/abg-comp-filter.cc
@@ -254,6 +254,43 @@ crc_changed(const diff* diff)
   return false;
 }
 
+/// Test if there was a function or variable namespace change.
+///
+/// @param f the first function or variable to consider.
+///
+/// @param s the second function or variable to consider.
+///
+/// @return true if the test is positive, false otherwise.
+template <typename function_or_var_decl_sptr>
+static bool
+namespace_changed(const function_or_var_decl_sptr& f,
+		  const function_or_var_decl_sptr& s)
+{
+  const auto& symbol_f = f->get_symbol();
+  const auto& symbol_s = s->get_symbol();
+  if (!symbol_f || !symbol_s)
+    return false;
+  return symbol_f->get_namespace() != symbol_s->get_namespace();
+}
+
+/// Test if the current diff tree node carries a namespace change in
+/// either a function or a variable.
+///
+/// @param diff the diff tree node to consider.
+///
+/// @return true if the test is positive, false otherwise.
+static bool
+namespace_changed(const diff* diff)
+{
+  if (const function_decl_diff* d =
+	dynamic_cast<const function_decl_diff*>(diff))
+    return namespace_changed(d->first_function_decl(),
+			     d->second_function_decl());
+  if (const var_diff* d = dynamic_cast<const var_diff*>(diff))
+    return namespace_changed(d->first_var(), d->second_var());
+  return false;
+}
+
 /// Test if there was a function name change, but there there was no
 /// change in name of the underlying symbol.  IOW, if the name of a
 /// function changed, but the symbol of the new function is equal to
@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre)
 	      || non_static_data_member_added_or_removed(d)
 	      || base_classes_added_or_removed(d)
 	      || has_harmful_enum_change(d)
-	      || crc_changed(d)))
+	      || crc_changed(d)
+	      || namespace_changed(d)))
 	category |= SIZE_OR_OFFSET_CHANGE_CATEGORY;
 
       if (has_virtual_mem_fn_change(d))
diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index 34ae486f..a271bfde 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -1728,6 +1728,7 @@ struct elf_symbol::priv
   bool			is_common_;
   bool			is_in_ksymtab_;
   abg_compat::optional<uint64_t>	crc_;
+  abg_compat::optional<std::string>	namespace_;
   bool			is_suppressed_;
   elf_symbol_wptr	main_symbol_;
   elf_symbol_wptr	next_alias_;
@@ -1745,6 +1746,7 @@ struct elf_symbol::priv
       is_common_(false),
       is_in_ksymtab_(false),
       crc_(),
+      namespace_(),
       is_suppressed_(false)
   {}
 
@@ -1760,6 +1762,7 @@ struct elf_symbol::priv
        elf_symbol::visibility	  vi,
        bool			  is_in_ksymtab,
        const abg_compat::optional<uint64_t>&	crc,
+       const abg_compat::optional<std::string>&	ns,
        bool			  is_suppressed)
     : env_(e),
       index_(i),
@@ -1773,6 +1776,7 @@ struct elf_symbol::priv
       is_common_(c),
       is_in_ksymtab_(is_in_ksymtab),
       crc_(crc),
+      namespace_(ns),
       is_suppressed_(is_suppressed)
   {
     if (!is_common_)
@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol()
 /// @param vi the visibility of the symbol.
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
+///
+/// @param ns the namespace of Linux Kernel symbols, if any
 elf_symbol::elf_symbol(const environment* e,
 		       size_t		  i,
 		       size_t		  s,
@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e,
 		       visibility	  vi,
 		       bool		  is_in_ksymtab,
 		       const abg_compat::optional<uint64_t>&	crc,
+		       const abg_compat::optional<std::string>&	ns,
 		       bool		  is_suppressed)
   : priv_(new priv(e,
 		   i,
@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e,
 		   vi,
 		   is_in_ksymtab,
 		   crc,
+		   ns,
 		   is_suppressed))
 {}
 
@@ -1886,6 +1894,8 @@ elf_symbol::create()
 ///
 /// @param crc the CRC (modversions) value of Linux Kernel symbols
 ///
+/// @param ns the namespace of Linux Kernel symbols, if any
+///
 /// @return a (smart) pointer to a newly created instance of @ref
 /// elf_symbol.
 elf_symbol_sptr
@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e,
 		   visibility	      vi,
 		   bool		      is_in_ksymtab,
 		   const abg_compat::optional<uint64_t>&	crc,
+		   const abg_compat::optional<std::string>&	ns,
 		   bool		      is_suppressed)
 {
   elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi,
-				     is_in_ksymtab, crc, is_suppressed));
+				     is_in_ksymtab, crc, ns, is_suppressed));
   sym->priv_->main_symbol_ = sym;
   return sym;
 }
@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l,
 		 && l.is_defined() == r.is_defined()
 		 && l.is_common_symbol() == r.is_common_symbol()
 		 && l.get_version() == r.get_version()
-		 && l.get_crc() == r.get_crc());
+		 && l.get_crc() == r.get_crc()
+		 && l.get_namespace() == r.get_namespace());
 
   if (equals && l.is_variable())
     // These are variable symbols.  Let's compare their symbol size.
@@ -2146,6 +2158,20 @@ void
 elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc)
 {priv_->crc_ = crc;}
 
+/// Getter of the 'namespace' property.
+///
+/// @return the namespace for Linux Kernel symbols, if any
+const abg_compat::optional<std::string>&
+elf_symbol::get_namespace() const
+{return priv_->namespace_;}
+
+/// Setter of the 'namespace' property.
+///
+/// @param ns the new namespace for Linux Kernel symbols, if any
+void
+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns)
+{priv_->namespace_ = ns;}
+
 /// Getter for the 'is-suppressed' property.
 ///
 /// @return true iff the current symbol has been suppressed by a
diff --git a/src/abg-reader.cc b/src/abg-reader.cc
index 7a9ad1f9..f9e420f1 100644
--- a/src/abg-reader.cc
+++ b/src/abg-reader.cc
@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node,
   if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc"))
     e->set_crc(strtoull(CHAR_STR(s), NULL, 0));
 
+  if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace"))
+    {
+      std::string ns;
+      xml::xml_char_sptr_to_string(s, ns);
+      e->set_namespace(ns);
+    }
+
   return e;
 }
 
diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc
index d000620b..fcfc9dce 100644
--- a/src/abg-reporter-priv.cc
+++ b/src/abg-reporter-priv.cc
@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr&	symbol1,
       out << std::noshowbase << std::dec
 	  << "\n";
     }
+
+  const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace();
+  const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace();
+  if (ns1 != ns2)
+    {
+      const std::string none = "(none)";
+      out << indent << "namespace changed from ";
+      if (ns1.has_value())
+	out << "'" << ns1.value() << "'";
+      else
+	out << none;
+      out << " to ";
+      if (ns2.has_value())
+	out << "'" << ns2.value() << "'";
+      else
+	out << none;
+      out << "\n";
+    }
 }
 
 /// For a given symbol, emit a string made of its name and version.
diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc
index 0c2f8ccc..e20ee2a7 100644
--- a/src/abg-symtab-reader.cc
+++ b/src/abg-symtab-reader.cc
@@ -204,6 +204,13 @@ symtab::load_(Elf*	       elf_handle,
 	      ir::environment* env,
 	      symbol_predicate is_suppressed)
 {
+  GElf_Ehdr ehdr_mem;
+  GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem);
+  if (!header)
+    {
+      std::cerr << "Could not get ELF header: Skipping symtab load.\n";
+      return false;
+    }
 
   Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle);
   if (!symtab_section)
@@ -232,9 +239,34 @@ symtab::load_(Elf*	       elf_handle,
       return false;
     }
 
+  // The __kstrtab_strings sections is basically an ELF strtab but does not
+  // support elf_strptr lookups. A single call to elf_getdata gives a handle to
+  // washed section data.
+  //
+  // The value of a __kstrtabns_FOO (or other similar) symbol is an address
+  // within the __kstrtab_strings section. To look up the string value, we need
+  // to translate from vmlinux load address to section offset by subtracting the
+  // base address of the section. This adjustment is not needed for loadable
+  // modules which are relocatable and so identifiable by ELF type ET_REL.
+  Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle);
+  size_t strings_offset = 0;
+  const char* strings_data = nullptr;
+  size_t strings_size = 0;
+  if (strings_section)
+    {
+      GElf_Shdr strings_sheader;
+      gelf_getshdr(strings_section, &strings_sheader);
+      strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr;
+      Elf_Data* data = elf_getdata(strings_section, nullptr);
+      ABG_ASSERT(data->d_off == 0);
+      strings_data = reinterpret_cast<const char *>(data->d_buf);
+      strings_size = data->d_size;
+    }
+
   const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle);
   std::unordered_set<std::string> exported_kernel_symbols;
   std::unordered_map<std::string, uint64_t> crc_values;
+  std::unordered_map<std::string, std::string> namespaces;
 
   for (size_t i = 0; i < number_syms; ++i)
     {
@@ -285,6 +317,27 @@ symtab::load_(Elf*	       elf_handle,
 	  ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second);
 	  continue;
 	}
+      if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0)
+	{
+	  // This symbol lives in the __ksymtab_strings section but st_value may
+	  // be a vmlinux load address so we need to subtract the offset before
+	  // looking it up in that section.
+	  const size_t value = sym->st_value;
+	  const size_t offset = value - strings_offset;
+	  // check offset
+	  ABG_ASSERT(offset < strings_size);
+	  // find the terminating NULL
+	  const char* first = strings_data + offset;
+	  const char* last = strings_data + strings_size;
+	  const char* limit = std::find(first, last, 0);
+	  // check NULL found
+	  ABG_ASSERT(limit < last);
+	  // interpret the empty namespace name as no namespace name
+	  if (first < limit)
+	    ABG_ASSERT(namespaces.emplace(
+		name.substr(12), std::string(first, limit - first)).second);
+	  continue;
+	}
 
       // filter out uninteresting entries and only keep functions/variables for
       // now. The rest might be interesting in the future though.
@@ -374,6 +427,17 @@ symtab::load_(Elf*	       elf_handle,
 	symbol->set_crc(crc_entry.second);
     }
 
+  // Now add the namespaces
+  for (const auto& namespace_entry : namespaces)
+    {
+      const auto r = name_symbol_map_.find(namespace_entry.first);
+      if (r == name_symbol_map_.end())
+	continue;
+
+      for (const auto& symbol : r->second)
+	symbol->set_namespace(namespace_entry.second);
+    }
+
   // sort the symbols for deterministic output
   std::sort(symbols_.begin(), symbols_.end(), symbol_sort);
 
diff --git a/src/abg-writer.cc b/src/abg-writer.cc
index 23598ef0..90db34be 100644
--- a/src/abg-writer.cc
+++ b/src/abg-writer.cc
@@ -3075,6 +3075,9 @@ write_elf_symbol(const elf_symbol_sptr&	sym,
       << std::hex << std::showbase << sym->get_crc().value()
       << std::dec << std::noshowbase << "'";
 
+  if (sym->get_namespace().has_value())
+    o << " namespace='" << sym->get_namespace().value() << "'";
+
   o << "/>\n";
 
   return true;
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
index be342684..c3112075 100644
--- a/tests/data/Makefile.am
+++ b/tests/data/Makefile.am
@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \
 test-abidiff/test-crc-report-0-1.txt \
 test-abidiff/test-crc-report-1-0.txt \
 test-abidiff/test-crc-report-1-2.txt \
+test-abidiff/test-namespace-0.xml \
+test-abidiff/test-namespace-1.xml \
+test-abidiff/test-namespace-report.txt \
 test-abidiff/test-PR27985-report.txt	 \
 test-abidiff/test-PR27985-v0.c		 \
 test-abidiff/test-PR27985-v0.o		 \
diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml
new file mode 100644
index 00000000..5a9f5cd2
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-0.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml
new file mode 100644
index 00000000..9814844b
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-1.xml
@@ -0,0 +1,15 @@
+<abi-corpus path='test.o' architecture='elf-amd-x86_64'>
+  <elf-variable-symbols>
+    <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/>
+    <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/>
+    <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/>
+    <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/>
+  </elf-variable-symbols>
+  <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'>
+    <type-decl name='int' size-in-bits='32' id='type-id-1'/>
+    <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/>
+    <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/>
+    <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/>
+    <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/>
+  </abi-instr>
+</abi-corpus>
diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt
new file mode 100644
index 00000000..d2c421ed
--- /dev/null
+++ b/tests/data/test-abidiff/test-namespace-report.txt
@@ -0,0 +1,17 @@
+Functions changes summary: 0 Removed, 0 Changed, 0 Added function
+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables
+
+4 Changed variables:
+
+  [C] 'int v1' was changed:
+    namespace changed from (none) to 'this'
+
+  [C] 'int v2' was changed:
+    namespace changed from 'this' to 'that'
+
+  [C] 'int v3' was changed:
+    namespace changed from 'that' to ''
+
+  [C] 'int v4' was changed:
+    namespace changed from '' to (none)
+
diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc
index a02bbe3d..93e44d6a 100644
--- a/tests/test-abidiff.cc
+++ b/tests/test-abidiff.cc
@@ -128,6 +128,12 @@ static InOutSpec specs[] =
     "data/test-abidiff/test-crc-report-1-2.txt",
     "output/test-abidiff/test-crc-report-1-2.txt"
   },
+  {
+    "data/test-abidiff/test-namespace-0.xml",
+    "data/test-abidiff/test-namespace-1.xml",
+    "data/test-abidiff/test-namespace-report.txt",
+    "output/test-abidiff/test-namespace-report-0-1.txt"
+  },
   {
     "data/test-abidiff/test-PR27616-v0.xml",
     "data/test-abidiff/test-PR27616-v1.xml",
-- 
2.36.1.476.g0c4daa206d-goog


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v5 0/4] add symbol namespace support, update symbol CRC support
  2022-06-13 14:25       ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
@ 2022-06-30 16:39         ` Dodji Seketeli
  0 siblings, 0 replies; 37+ messages in thread
From: Dodji Seketeli @ 2022-06-30 16:39 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich

Giuliano Procida <gprocida@google.com> a écrit:

> Hi Dodji.

Hey Giuliano,

Sorry for the late reply.

> I'd be grateful if could take a look at this patch series. Patch 4/4
> in particular addresses Bug 28954.

Yeah, I am looking into it at the moment.

Thanks for the great work!

[...]

Cheers,

-- 
		Dodji

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values
  2022-06-13 14:25       ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
@ 2022-06-30 16:40         ` Dodji Seketeli
  0 siblings, 0 replies; 37+ messages in thread
From: Dodji Seketeli @ 2022-06-30 16:40 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich

Hello,

Giuliano Procida <gprocida@google.com> a écrit:

> As pointed out in a review of similar code, it is possible to avoid
> copying a couple of shared pointers in this function, by taking
> references instead.
>
> This commit also splits declarations to one per line and removes the
> unnecessary parentheses around the return expression.
>
> 	* src/abg-comp-filter.cc (crc_changed): Take references to
> 	avoid std::shared_ptr copying. Split declarations into one per
> 	line. Remove unnecessary return expression parentheses.
>
> Reviewed-by: Matthias Maennich <maennich@google.com>
> Signed-off-by: Giuliano Procida <gprocida@google.com>

Applied to master, thanks!

[...]

Cheers,

-- 
		Dodji

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v5 2/4] optional: minor improvements
  2022-06-13 14:25       ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida
@ 2022-06-30 16:40         ` Dodji Seketeli
  0 siblings, 0 replies; 37+ messages in thread
From: Dodji Seketeli @ 2022-06-30 16:40 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich

Hello,

Giuliano Procida <gprocida@google.com> a écrit:

> This change makes minor improvements to the optional class used with
> pre-C++17 compilers.
>
> - adds operator== and operator!=
> - adds various missing noexcept (but not constexpr) decorations
> - defines operator bool in terms of has_value
>
> Note that some constexpr decorations would require C++17 anyway.
>
> 	* include/abg-cxx-compat.h (optional): Add operator== and
> 	operator!=. Add noexcept decorations. Tweak operator bool.
>
> Reviewed-by: Matthias Maennich <maennich@google.com>
> Signed-off-by: Giuliano Procida <gprocida@google.com>

Applied to master, thanks!

[...]

Cheers,

-- 
		Dodji

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes
  2022-06-13 14:25       ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
@ 2022-06-30 16:41         ` Dodji Seketeli
  0 siblings, 0 replies; 37+ messages in thread
From: Dodji Seketeli @ 2022-06-30 16:41 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich

Hello,

Giuliano Procida <gprocida@google.com> a écrit:

> The CRC with value zero was used to mean "absent". This can be better
> modelled using optional.
>
> This commit makes this change and also tweaks reporting so that
> disappearing / appearing CRCs are noted. This should be essentially
> impossible unless CRCs are enabled / disabled altogether but would be
> very noteworthy otherwise.
>
> 	* include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is
> 	now an optional defaulted to absent.
> 	(elf_symbol::create): Likewise.
> 	(elf_symbol::get_crc): Now returns an optional uint64_t.
> 	(elf_symbol::set_src): Now takes an optional uint64_t.
> 	* src/abg-comp-filter.cc (crc_changed): Simplify comparison.
> 	* src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an
> 	optional uint64_t.
> 	(elf_symbol::priv::priv): Argument crc is now an optional
> 	uint64_t.
> 	(elf_symbol::elf_symbol): Likewise.
> 	(elf_symbol::create): Argument crc is now an optional uint64_t
> 	and defaults to absent.
> 	(textually_equals): Simplify comparison.
> 	(elf_symbol::get_crc): Now returns an optional uint64_t.
> 	(elf_symbol::set_crc): Now takes an optional uint64_t.
> 	* src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same
> 	as other CRC values.
> 	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol):
> 	Treat CRC 0 the same as other CRC values and also report
> 	changes to CRC presence.
> 	* src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same
> 	as other CRC values.
> 	* tests/data/Makefile: Remove test-abidiff/test-crc-report.txt
> 	and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt.
> 	* tests/data/test-abidiff/test-crc-report-0-1.txt: Report
> 	showing additional of CRCs.
> 	* tests/data/test-abidiff/test-crc-report-1-0.txt: Report
> 	showing removal of CRCs.
> 	* tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed
> 	from tests/data/test-abidiff/test-crc-report.txt.
> 	* tests/test-abidiff.cc: Update test cases that no longer
> 	generate empty reports.
> 	* tests/test-symtab.cc: Update KernelSymtabsWithCRC test.
>
> Reviewed-by: Matthias Maennich <maennich@google.com>
> Signed-off-by: Giuliano Procida <gprocida@google.com>

Applied to master, thanks !

[...]

Cheers,

-- 
		Dodji

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [PATCH v5 4/4] add Linux kernel symbol namespace support
  2022-06-13 14:25       ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida
@ 2022-07-01 12:54         ` Dodji Seketeli
  0 siblings, 0 replies; 37+ messages in thread
From: Dodji Seketeli @ 2022-07-01 12:54 UTC (permalink / raw)
  To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich

Hello,

Giuliano Procida <gprocida@google.com> a écrit:

> Bug 28954 - add Linux Kernel symbol namespace support
>
> Each Linux kernel symbol can be exported to a specified named
> namespace or left in the global (nameless) namespace.
>
> One complexity is that the symbol values which identify a string in
> the __ksymtab_strings section must be interpretated differently for
> vmlinux and .ko loadable modules as the former has a fixed load
> address but the latter are relocatable. For vmlinux, the section base
> address needs to be subtracted to obtain a section-relative offset.
>
> The global namespace is explicitly represented as the empty string, at
> least when it comes to the value of __kstrtabns_FOO symbols, but the
> common interpretation is that such symbols lack an export namespace.
>
> I would rather not have to make use of "empty implies missing" in many
> places, so the code here represents namespace as optional<string> and
> only the symtab reader cares about empty strings in __ksymtab_strings.
>
> 	* include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument.
> 	(elf_symbol::create): Add ns argument.
> 	(elf_symbol::get_namespace): Declare new function.
> 	(elf_symbol::set_namespace): Declare new function.
> 	and set_namespace.
> 	* src/abg-comp-filter.cc (namespace_changed): Define new
> 	helper functions.
> 	(categorize_harmful_diff_node): Also call namespace_changed().
> 	* src/abg-ir.cc (elf_symbol::priv): Add namespace_ member.
> 	(elf_symbol::priv::priv): Add namespace_ to initialisers.
> 	(elf_symbol::elf_symbol): Take new ns argument and pass it to
> 	priv constructor.
> 	(elf_symbol::create): Take new ns argument and pass it to
> 	elf_symbol constructor.
> 	(elf_symbol::get_namespace): Define new function.
> 	(elf_symbol::set_namespace): Define new function.
> 	* src/abg-reader.cc (build_elf_symbol): If namespace
> 	attribute is present, set symbol namespace.
> 	* src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If
> 	symbol namespaces differ, report this.
> 	* src/abg-symtab-reader.cc (symtab::load): Get ELF header to
> 	distinguish vmlinux from .ko. Try to get __ksymtab_strings
> 	metadata and data. Use these to look up __kstrtabns_FOO
> 	namespace entries. Set symbol namespace where found.
> 	* src/abg-writer.cc (write_elf_symbol): Emit namespace
> 	attribute, if symbol has a namespace.
> 	* tests/data/Makefile.am: Add new test files.
> 	* tests/data/test-abidiff/test-namespace-0.xml: New test file.
> 	* tests/data/test-abidiff/test-namespace-1.xml: Likewise
> 	* tests/data/test-abidiff/test-namespace-report.txt: Likewise.
> 	* tests/test-abidiff.cc: Add new test case.
>
> Reviewed-by: Matthias Maennich <maennich@google.com>
> Signed-off-by: Giuliano Procida <gprocida@google.com>

Applied to master, thanks!

[...]

Cheers,

-- 
		Dodji

^ permalink raw reply	[flat|nested] 37+ messages in thread

end of thread, other threads:[~2022-07-01 12:54 UTC | newest]

Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida
2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida
2022-03-15 11:02   ` Matthias Maennich
2022-03-16  9:31     ` Giuliano Procida
2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida
2022-03-15 11:25   ` Matthias Maennich
2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
2022-03-16 16:30   ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida
2022-03-17 10:56     ` Matthias Maennich
2022-03-16 16:30   ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
2022-03-17 11:01     ` Matthias Maennich
2022-03-16 16:30   ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida
2022-03-17 11:57     ` Matthias Maennich
2022-03-16 16:30   ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
2022-03-17 12:01     ` Matthias Maennich
2022-03-17 16:38   ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
2022-03-17 16:38     ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
2022-03-17 16:38     ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida
2022-03-17 16:38     ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
2022-03-17 16:38     ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida
2022-03-21 12:53       ` Matthias Maennich
2022-03-21 15:52         ` Giuliano Procida
2022-03-21 16:02     ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
2022-03-21 16:02       ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
2022-03-21 16:02       ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida
2022-03-21 16:02       ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
2022-03-21 16:02       ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida
2022-06-13 14:25       ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida
2022-06-30 16:39         ` Dodji Seketeli
2022-06-13 14:25       ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida
2022-06-30 16:40         ` Dodji Seketeli
2022-06-13 14:25       ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida
2022-06-30 16:40         ` Dodji Seketeli
2022-06-13 14:25       ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida
2022-06-30 16:41         ` Dodji Seketeli
2022-06-13 14:25       ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida
2022-07-01 12:54         ` Dodji Seketeli

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).