* [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support @ 2022-03-14 18:13 Giuliano Procida 2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida ` (2 more replies) 0 siblings, 3 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Hi Dodji. These two patches aim to add kernel symbol namespace support. There are two bits of this that could be debated (and easily changed). A. choice of XML attribute name I've chosen "ns", mostly because "namespace" implies C++ namespace everywhere else. B. choice of representation There are several possible levels of distinction of presence: 1. Non-kernel symbols cannot be exported to a namespace. 2. Kernel symbols can be exported globally and not to namespace. 3. Kernel symbols can be exported to a named namespace. And there is the option of treating the empty namespace name string specially. I've chosen to represent namespace internally as optional<string> and not distinguish between cases 1 and 2. Regards. Giuliano Procida (2): optional: add operator== and operator!= add Linux kernel symbol namespace support include/abg-cxx-compat.h | 16 +++++++++++++ include/abg-ir.h | 9 ++++++++ src/abg-comp-filter.cc | 39 +++++++++++++++++++++++++++++++- src/abg-ir.cc | 27 +++++++++++++++++++++- src/abg-reader.cc | 7 ++++++ src/abg-reporter-priv.cc | 12 ++++++++++ src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++ src/abg-writer.cc | 4 ++++ 8 files changed, 161 insertions(+), 2 deletions(-) -- 2.35.1.723.g4982287a31-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH 1/2] optional: add operator== and operator!= 2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida @ 2022-03-14 18:13 ` Giuliano Procida 2022-03-15 11:02 ` Matthias Maennich 2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich * include/abg-cxx-compat.h (optional): Add operator== and operator!=. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-cxx-compat.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h index 443905c7..a2cf9095 100644 --- a/include/abg-cxx-compat.h +++ b/include/abg-cxx-compat.h @@ -91,6 +91,22 @@ public: } explicit operator bool() const { return has_value_; } + + bool + operator==(const optional<T>& other) const + { + if (!has_value_ && !other.has_value_) + return true; + if (!has_value_ || !other.has_value_) + return false; + return value_ == other.value_; + } + + bool + operator!=(const optional<T>& other) const + { + return !operator==(other); + } }; #endif -- 2.35.1.723.g4982287a31-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 1/2] optional: add operator== and operator!= 2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida @ 2022-03-15 11:02 ` Matthias Maennich 2022-03-16 9:31 ` Giuliano Procida 0 siblings, 1 reply; 37+ messages in thread From: Matthias Maennich @ 2022-03-15 11:02 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Mon, Mar 14, 2022 at 06:13:11PM +0000, Giuliano Procida wrote: > * include/abg-cxx-compat.h (optional): Add operator== and > operator!=. > >Signed-off-by: Giuliano Procida <gprocida@google.com> >--- > include/abg-cxx-compat.h | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > >diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h >index 443905c7..a2cf9095 100644 >--- a/include/abg-cxx-compat.h >+++ b/include/abg-cxx-compat.h >@@ -91,6 +91,22 @@ public: > } > > explicit operator bool() const { return has_value_; } std signature is (at least make it noexcept): > constexpr explicit operator bool() const noexcept; >+ >+ bool >+ operator==(const optional<T>& other) const >+ { >+ if (!has_value_ && !other.has_value_) >+ return true; >+ if (!has_value_ || !other.has_value_) >+ return false; >+ return value_ == other.value_; >+ } >+ >+ bool >+ operator!=(const optional<T>& other) const >+ { >+ return !operator==(other); >+ } I believe C++17 defines those as non-member functions. Should we follow that for compatibility. Eventually, when switching to C++17 or later we would want to just replace abg_compat:: with std:: and things should "just work". Cheers, Matthias > }; > > #endif >-- >2.35.1.723.g4982287a31-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 1/2] optional: add operator== and operator!= 2022-03-15 11:02 ` Matthias Maennich @ 2022-03-16 9:31 ` Giuliano Procida 0 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 9:31 UTC (permalink / raw) To: Matthias Maennich; +Cc: libabigail, dodji, kernel-team Hi. On Tue, 15 Mar 2022 at 11:02, Matthias Maennich <maennich@google.com> wrote: > > On Mon, Mar 14, 2022 at 06:13:11PM +0000, Giuliano Procida wrote: > > * include/abg-cxx-compat.h (optional): Add operator== and > > operator!=. > > > >Signed-off-by: Giuliano Procida <gprocida@google.com> > >--- > > include/abg-cxx-compat.h | 16 ++++++++++++++++ > > 1 file changed, 16 insertions(+) > > > >diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h > >index 443905c7..a2cf9095 100644 > >--- a/include/abg-cxx-compat.h > >+++ b/include/abg-cxx-compat.h > >@@ -91,6 +91,22 @@ public: > > } > > > > explicit operator bool() const { return has_value_; } > > std signature is (at least make it noexcept): > > constexpr explicit operator bool() const noexcept; > Will do. I could add some, but not all, the constexpr as C++11 complains about ambiguous overloads. > >+ > >+ bool > >+ operator==(const optional<T>& other) const > >+ { > >+ if (!has_value_ && !other.has_value_) > >+ return true; > >+ if (!has_value_ || !other.has_value_) > >+ return false; > >+ return value_ == other.value_; > >+ } > >+ > >+ bool > >+ operator!=(const optional<T>& other) const > >+ { > >+ return !operator==(other); > >+ } > > I believe C++17 defines those as non-member functions. Should we follow > that for compatibility. Eventually, when switching to C++17 or later we > would want to just replace abg_compat:: with std:: and things should > "just work". > Sure. > Cheers, > Matthias > v2 will follow. Giuliano. > > }; > > > > #endif > >-- > >2.35.1.723.g4982287a31-goog > > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH 2/2] add Linux kernel symbol namespace support 2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida 2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida @ 2022-03-14 18:13 ` Giuliano Procida 2022-03-15 11:25 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-14 18:13 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Bug 28954 - add Linux Kernel symbol namespace support This commit adds Linux Kernel symbol namespace support roughly as discussed. Each symbol can be exported to a specified named namespace or left in the global (nameless) namespace. The latter is explicitly represented as the empty string, at least when it comes to the value of __kstrtabns_FOO symbols, but the common interpretation is that such symbols are not exported to a namespace. I would rather not make this assumption in more than one place in the code and would also prefer to protect against __kstrtabns_FOO for globally export FOO disappearing with a future change to the export macros. So the code here represents namespace as optional<string> and treats empty strings found in __ksymtab_strings as missing. * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. (elf_symbol::create): Add ns argument. (elf_symbol::get_namespace): Declare new function. (elf_symbol::set_namespace): Declare new function. and set_namespace. * src/abg-comp-filter.cc (namespace_changed): Define new helper functions. (categorize_harmful_diff_node): Also call namespace_changed(). * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. (elf_symbol::priv::priv): Add namespace_ to initialisers. (elf_symbol::elf_symbol): Take new ns argument and pass it to priv constructor. (elf_symbol::create): Take new ns argument and pass it to elf_symbol constructor. (elf_symbol::get_namespace): Define new function. (elf_symbol::set_namespace): Define new function. * src/abg-reader.cc (build_elf_symbol): If ns attribute is present, set symbol namespace. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If symbol namespaces differ, report this. * src/abg-symtab-reader.cc (symtab::load): Try to get __ksymtab_strings metadata and data. Use this to look up __kstrtabns_FOO namespace entries. Set symbol namespaces where found. * src/abg-writer.cc (write_elf_symbol): Emit ns attribute, if symbol has a namespace. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 9 ++++++++ src/abg-comp-filter.cc | 39 +++++++++++++++++++++++++++++++- src/abg-ir.cc | 27 +++++++++++++++++++++- src/abg-reader.cc | 7 ++++++ src/abg-reporter-priv.cc | 12 ++++++++++ src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++ src/abg-writer.cc | 4 ++++ 7 files changed, 145 insertions(+), 2 deletions(-) diff --git a/include/abg-ir.h b/include/abg-ir.h index a2f4e1a7..7c42bea7 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -21,6 +21,7 @@ #include <functional> #include <set> #include <unordered_map> +#include "abg-cxx-compat.h" #include "abg-fwd.h" #include "abg-hash.h" #include "abg-traverse.h" @@ -921,6 +922,7 @@ private: visibility vi, bool is_in_ksymtab = false, uint64_t crc = 0, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -946,6 +948,7 @@ public: visibility vi, bool is_in_ksymtab = false, uint64_t crc = 0, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); const environment* @@ -1023,6 +1026,12 @@ public: void set_crc(uint64_t crc); + const abg_compat::optional<std::string>& + get_namespace() const; + + void + set_namespace(const abg_compat::optional<std::string>& ns); + bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 56251274..6ee31e35 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -254,6 +254,42 @@ crc_changed(const diff* diff) return false; } +/// Test if there was a function or variable namespace change. +/// +/// @param f the first function or variable to consider. +/// +/// @param s the second function or variable to consider. +/// +/// @return true if the test is positive, false otherwise. +template <typename function_or_var_decl_sptr> +static bool +namespace_changed(const function_or_var_decl_sptr& f, + const function_or_var_decl_sptr& s) +{ + const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); + if (!symbol_f || !symbol_s) + return false; + return symbol_f->get_namespace() != symbol_s->get_namespace(); +} + +/// test if the current diff tree node carries a namespace change in +/// either a function or a variable. +/// +/// @param diff the diff tree node to consider. +/// +/// @return true if the test is positive, false otherwise. +static bool +namespace_changed(const diff* diff) +{ + if (const function_decl_diff* d = + dynamic_cast<const function_decl_diff*>(diff)) + return namespace_changed(d->first_function_decl(), + d->second_function_decl()); + if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) + return namespace_changed(d->first_var(), d->second_var()); + return false; +} + /// Test if there was a function name change, but there there was no /// change in name of the underlying symbol. IOW, if the name of a /// function changed, but the symbol of the new function is equal to @@ -1781,7 +1817,8 @@ categorize_harmful_diff_node(diff *d, bool pre) || non_static_data_member_added_or_removed(d) || base_classes_added_or_removed(d) || has_harmful_enum_change(d) - || crc_changed(d))) + || crc_changed(d) + || namespace_changed(d))) category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; if (has_virtual_mem_fn_change(d)) diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 0ef5e8b2..c510bc55 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1728,6 +1728,7 @@ struct elf_symbol::priv bool is_common_; bool is_in_ksymtab_; uint64_t crc_; + abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1745,6 +1746,7 @@ struct elf_symbol::priv is_common_(false), is_in_ksymtab_(false), crc_(0), + namespace_(), is_suppressed_(false) {} @@ -1760,6 +1762,7 @@ struct elf_symbol::priv elf_symbol::visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), index_(i), @@ -1773,6 +1776,7 @@ struct elf_symbol::priv is_common_(c), is_in_ksymtab_(is_in_ksymtab), crc_(crc), + namespace_(ns), is_suppressed_(is_suppressed) { if (!is_common_) @@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() /// @param vi the visibility of the symbol. /// /// @param crc the CRC (modversions) value of Linux Kernel symbols +/// +/// @param ns the namespace of Linux Kernel symbols, if any elf_symbol::elf_symbol(const environment* e, size_t i, size_t s, @@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, i, @@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, vi, is_in_ksymtab, crc, + ns, is_suppressed)) {} @@ -1886,6 +1894,8 @@ elf_symbol::create() /// /// @param crc the CRC (modversions) value of Linux Kernel symbols /// +/// @param ns the namespace of Linux Kernel symbols, if any +/// /// @return a (smart) pointer to a newly created instance of @ref /// elf_symbol. elf_symbol_sptr @@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, - is_in_ksymtab, crc, is_suppressed)); + is_in_ksymtab, crc, ns, is_suppressed)); sym->priv_->main_symbol_ = sym; return sym; } @@ -2147,6 +2158,20 @@ void elf_symbol::set_crc(uint64_t crc) {priv_->crc_ = crc;} +/// Getter of the 'namespace' property. +/// +/// @return the namespace for Linux Kernel symbols, if any +const abg_compat::optional<std::string>& +elf_symbol::get_namespace() const +{return priv_->namespace_;} + +/// Setter of the 'namespace' property. +/// +/// @param ns the new namespace for Linux Kernel symbols, if any +void +elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) +{priv_->namespace_ = ns;} + /// Getter for the 'is-suppressed' property. /// /// @return true iff the current symbol has been suppressed by a diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 31885692..55b7348d 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, if (crc != 0) e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "ns")) + { + std::string ns; + xml::xml_char_sptr_to_string(s, ns); + e->set_namespace({ns}); + } + return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 7012f5dc..dd40690c 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1158,6 +1158,18 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << std::noshowbase << std::dec << "\n"; } + + const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); + const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); + if (ns1 != ns2) + { + const std::string none = "(none)"; + out << indent << "namespace changed from " + << (ns1.has_value() ? ns1.value() : none) + << " to " + << (ns2.has_value() ? ns2.value() : none) + << "\n"; + } } /// For a given symbol, emit a string made of its name and version. diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc index b42ce87d..589f3703 100644 --- a/src/abg-symtab-reader.cc +++ b/src/abg-symtab-reader.cc @@ -232,9 +232,33 @@ symtab::load_(Elf* elf_handle, return false; } + // The __kstrtab_strings sections is basically an ELF strtab but does not + // support elf_strptr lookups. A single call to elf_getdata gives a handle to + // section data. + // + // The value of a __kstrtabns_FOO (or other similar) symbol is an address + // within the __kstrtab_strings section. To look up the string value, we need + // to translate from load address to section offset by subtracting the base + // address of the section. + Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); + size_t strings_offset = 0; + const char* strings_data = nullptr; + size_t strings_size = 0; + if (strings_section) + { + GElf_Shdr strings_sheader; + gelf_getshdr(strings_section, &strings_sheader); + strings_offset = strings_sheader.sh_addr; + Elf_Data* data = elf_getdata(strings_section, nullptr); + ABG_ASSERT(data->d_off == 0); + strings_data = reinterpret_cast<const char *>(data->d_buf); + strings_size = data->d_size; + } + const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); std::unordered_set<std::string> exported_kernel_symbols; std::unordered_map<std::string, uint64_t> crc_values; + std::unordered_map<std::string, std::string> namespaces; const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); @@ -288,6 +312,20 @@ symtab::load_(Elf* elf_handle, ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); continue; } + if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) + { + // This symbol lives in the __ksymtab_strings section but st_value is + // a load address so we need to subtract an offset before looking it + // up in that section. + const size_t offset = sym->st_value - strings_offset; + if (offset < strings_size) + { + const char* ns = strings_data + offset; + if (*ns) + ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); + } + continue; + } // filter out uninteresting entries and only keep functions/variables for // now. The rest might be interesting in the future though. @@ -402,6 +440,17 @@ symtab::load_(Elf* elf_handle, symbol->set_crc(crc_entry.second); } + // Now add the namespaces + for (const auto& namespace_entry : namespaces) + { + const auto r = name_symbol_map_.find(namespace_entry.first); + if (r == name_symbol_map_.end()) + continue; + + for (const auto& symbol : r->second) + symbol->set_namespace({namespace_entry.second}); + } + // sort the symbols for deterministic output std::sort(symbols_.begin(), symbols_.end(), symbol_sort); diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 7802128d..4dfa72a6 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3136,6 +3136,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, << std::hex << std::showbase << sym->get_crc() << "'" << std::dec << std::noshowbase; + if (sym->get_namespace().has_value()) + o << " ns='" + << sym->get_namespace().value() << "'"; + o << "/>\n"; return true; -- 2.35.1.723.g4982287a31-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH 2/2] add Linux kernel symbol namespace support 2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-15 11:25 ` Matthias Maennich 0 siblings, 0 replies; 37+ messages in thread From: Matthias Maennich @ 2022-03-15 11:25 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Mon, Mar 14, 2022 at 06:13:12PM +0000, Giuliano Procida wrote: >Bug 28954 - add Linux Kernel symbol namespace support > >This commit adds Linux Kernel symbol namespace support roughly as >discussed. > >Each symbol can be exported to a specified named namespace or left in >the global (nameless) namespace. The latter is explicitly represented >as the empty string, at least when it comes to the value of >__kstrtabns_FOO symbols, but the common interpretation is that such >symbols are not exported to a namespace. > >I would rather not make this assumption in more than one place in the >code and would also prefer to protect against __kstrtabns_FOO for >globally export FOO disappearing with a future change to the export >macros. > >So the code here represents namespace as optional<string> and treats >empty strings found in __ksymtab_strings as missing. > > * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. > (elf_symbol::create): Add ns argument. > (elf_symbol::get_namespace): Declare new function. > (elf_symbol::set_namespace): Declare new function. > and set_namespace. > * src/abg-comp-filter.cc (namespace_changed): Define new > helper functions. > (categorize_harmful_diff_node): Also call namespace_changed(). > * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. > (elf_symbol::priv::priv): Add namespace_ to initialisers. > (elf_symbol::elf_symbol): Take new ns argument and pass it to > priv constructor. > (elf_symbol::create): Take new ns argument and pass it to > elf_symbol constructor. > (elf_symbol::get_namespace): Define new function. > (elf_symbol::set_namespace): Define new function. > * src/abg-reader.cc (build_elf_symbol): If ns attribute is > present, set symbol namespace. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If > symbol namespaces differ, report this. > * src/abg-symtab-reader.cc (symtab::load): Try to get > __ksymtab_strings metadata and data. Use this to look up > __kstrtabns_FOO namespace entries. Set symbol namespaces where > found. > * src/abg-writer.cc (write_elf_symbol): Emit ns attribute, if > symbol has a namespace. > >Signed-off-by: Giuliano Procida <gprocida@google.com> Thanks for working on that! I think this is conceptually the right thing to do to allow compatibility with different kernel versions and configurations. I just had some minor comments. With those addressed, feel free to add: Reviewed-by: Matthias Maennich <maennich@google.com> >--- > include/abg-ir.h | 9 ++++++++ > src/abg-comp-filter.cc | 39 +++++++++++++++++++++++++++++++- > src/abg-ir.cc | 27 +++++++++++++++++++++- > src/abg-reader.cc | 7 ++++++ > src/abg-reporter-priv.cc | 12 ++++++++++ > src/abg-symtab-reader.cc | 49 ++++++++++++++++++++++++++++++++++++++++ > src/abg-writer.cc | 4 ++++ > 7 files changed, 145 insertions(+), 2 deletions(-) > >diff --git a/include/abg-ir.h b/include/abg-ir.h >index a2f4e1a7..7c42bea7 100644 >--- a/include/abg-ir.h >+++ b/include/abg-ir.h >@@ -21,6 +21,7 @@ > #include <functional> > #include <set> > #include <unordered_map> >+#include "abg-cxx-compat.h" > #include "abg-fwd.h" > #include "abg-hash.h" > #include "abg-traverse.h" >@@ -921,6 +922,7 @@ private: > visibility vi, > bool is_in_ksymtab = false, > uint64_t crc = 0, >+ const abg_compat::optional<std::string>& ns = {}, When implementing this in the kernel itself, I also got trapped in naming inconsistencies, ns ... NS ... namespace. I believe I went with consistently calling it 'namespace'. I suggest to not abbreviate with 'ns'. > bool is_suppressed = false); > > elf_symbol(const elf_symbol&); >@@ -946,6 +948,7 @@ public: > visibility vi, > bool is_in_ksymtab = false, > uint64_t crc = 0, >+ const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > > const environment* >@@ -1023,6 +1026,12 @@ public: > void > set_crc(uint64_t crc); > >+ const abg_compat::optional<std::string>& >+ get_namespace() const; >+ >+ void >+ set_namespace(const abg_compat::optional<std::string>& ns); >+ > bool > is_suppressed() const; > >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc >index 56251274..6ee31e35 100644 >--- a/src/abg-comp-filter.cc >+++ b/src/abg-comp-filter.cc >@@ -254,6 +254,42 @@ crc_changed(const diff* diff) > return false; > } > >+/// Test if there was a function or variable namespace change. >+/// >+/// @param f the first function or variable to consider. >+/// >+/// @param s the second function or variable to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+template <typename function_or_var_decl_sptr> >+static bool >+namespace_changed(const function_or_var_decl_sptr& f, >+ const function_or_var_decl_sptr& s) >+{ >+ const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); const auto& to avoid some refcounts on the symbol_sptr. >+ if (!symbol_f || !symbol_s) >+ return false; >+ return symbol_f->get_namespace() != symbol_s->get_namespace(); >+} >+ >+/// test if the current diff tree node carries a namespace change in ^ Test >+/// either a function or a variable. >+/// >+/// @param diff the diff tree node to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+static bool >+namespace_changed(const diff* diff) >+{ >+ if (const function_decl_diff* d = >+ dynamic_cast<const function_decl_diff*>(diff)) >+ return namespace_changed(d->first_function_decl(), >+ d->second_function_decl()); >+ if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) >+ return namespace_changed(d->first_var(), d->second_var()); >+ return false; >+} >+ > /// Test if there was a function name change, but there there was no > /// change in name of the underlying symbol. IOW, if the name of a > /// function changed, but the symbol of the new function is equal to >@@ -1781,7 +1817,8 @@ categorize_harmful_diff_node(diff *d, bool pre) > || non_static_data_member_added_or_removed(d) > || base_classes_added_or_removed(d) > || has_harmful_enum_change(d) >- || crc_changed(d))) >+ || crc_changed(d) >+ || namespace_changed(d))) > category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; > > if (has_virtual_mem_fn_change(d)) >diff --git a/src/abg-ir.cc b/src/abg-ir.cc >index 0ef5e8b2..c510bc55 100644 >--- a/src/abg-ir.cc >+++ b/src/abg-ir.cc >@@ -1728,6 +1728,7 @@ struct elf_symbol::priv > bool is_common_; > bool is_in_ksymtab_; > uint64_t crc_; >+ abg_compat::optional<std::string> namespace_; > bool is_suppressed_; > elf_symbol_wptr main_symbol_; > elf_symbol_wptr next_alias_; >@@ -1745,6 +1746,7 @@ struct elf_symbol::priv > is_common_(false), > is_in_ksymtab_(false), > crc_(0), >+ namespace_(), > is_suppressed_(false) > {} > >@@ -1760,6 +1762,7 @@ struct elf_symbol::priv > elf_symbol::visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : env_(e), > index_(i), >@@ -1773,6 +1776,7 @@ struct elf_symbol::priv > is_common_(c), > is_in_ksymtab_(is_in_ksymtab), > crc_(crc), >+ namespace_(ns), > is_suppressed_(is_suppressed) > { > if (!is_common_) >@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() > /// @param vi the visibility of the symbol. > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols >+/// >+/// @param ns the namespace of Linux Kernel symbols, if any > elf_symbol::elf_symbol(const environment* e, > size_t i, > size_t s, >@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, > visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : priv_(new priv(e, > i, >@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, > vi, > is_in_ksymtab, > crc, >+ ns, > is_suppressed)) > {} > >@@ -1886,6 +1894,8 @@ elf_symbol::create() > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols > /// >+/// @param ns the namespace of Linux Kernel symbols, if any >+/// > /// @return a (smart) pointer to a newly created instance of @ref > /// elf_symbol. > elf_symbol_sptr >@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, > visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > { > elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, >- is_in_ksymtab, crc, is_suppressed)); >+ is_in_ksymtab, crc, ns, is_suppressed)); > sym->priv_->main_symbol_ = sym; > return sym; > } >@@ -2147,6 +2158,20 @@ void > elf_symbol::set_crc(uint64_t crc) > {priv_->crc_ = crc;} > >+/// Getter of the 'namespace' property. >+/// >+/// @return the namespace for Linux Kernel symbols, if any >+const abg_compat::optional<std::string>& >+elf_symbol::get_namespace() const >+{return priv_->namespace_;} >+ >+/// Setter of the 'namespace' property. >+/// >+/// @param ns the new namespace for Linux Kernel symbols, if any >+void >+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) While I like the std::optional in the data structure, I am not sure I like it exposed to the interface. Is there ever a case where set_namespace is called with an unset optional? And if so, would that not be a better interface to offer ::unset_namespace ? >+{priv_->namespace_ = ns;} >+ > /// Getter for the 'is-suppressed' property. > /// > /// @return true iff the current symbol has been suppressed by a >diff --git a/src/abg-reader.cc b/src/abg-reader.cc >index 31885692..55b7348d 100644 >--- a/src/abg-reader.cc >+++ b/src/abg-reader.cc >@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > if (crc != 0) > e->set_crc(crc); > >+ if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "ns")) In particular, I would call the attribute 'namespace' in the XML unless we want to save bytes here. >+ { >+ std::string ns; >+ xml::xml_char_sptr_to_string(s, ns); >+ e->set_namespace({ns}); >+ } >+ > return e; > } > >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc >index 7012f5dc..dd40690c 100644 >--- a/src/abg-reporter-priv.cc >+++ b/src/abg-reporter-priv.cc >@@ -1158,6 +1158,18 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, > << std::noshowbase << std::dec > << "\n"; > } >+ >+ const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); >+ const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); >+ if (ns1 != ns2) >+ { >+ const std::string none = "(none)"; >+ out << indent << "namespace changed from " >+ << (ns1.has_value() ? ns1.value() : none) >+ << " to " >+ << (ns2.has_value() ? ns2.value() : none) >+ << "\n"; >+ } > } > > /// For a given symbol, emit a string made of its name and version. >diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc >index b42ce87d..589f3703 100644 >--- a/src/abg-symtab-reader.cc >+++ b/src/abg-symtab-reader.cc >@@ -232,9 +232,33 @@ symtab::load_(Elf* elf_handle, > return false; > } > >+ // The __kstrtab_strings sections is basically an ELF strtab but does not >+ // support elf_strptr lookups. A single call to elf_getdata gives a handle to >+ // section data. >+ // >+ // The value of a __kstrtabns_FOO (or other similar) symbol is an address >+ // within the __kstrtab_strings section. To look up the string value, we need >+ // to translate from load address to section offset by subtracting the base >+ // address of the section. >+ Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); >+ size_t strings_offset = 0; >+ const char* strings_data = nullptr; >+ size_t strings_size = 0; >+ if (strings_section) >+ { >+ GElf_Shdr strings_sheader; >+ gelf_getshdr(strings_section, &strings_sheader); >+ strings_offset = strings_sheader.sh_addr; >+ Elf_Data* data = elf_getdata(strings_section, nullptr); >+ ABG_ASSERT(data->d_off == 0); >+ strings_data = reinterpret_cast<const char *>(data->d_buf); >+ strings_size = data->d_size; >+ } >+ > const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); > std::unordered_set<std::string> exported_kernel_symbols; > std::unordered_map<std::string, uint64_t> crc_values; >+ std::unordered_map<std::string, std::string> namespaces; > > const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); > const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); >@@ -288,6 +312,20 @@ symtab::load_(Elf* elf_handle, > ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); > continue; > } >+ if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) >+ { >+ // This symbol lives in the __ksymtab_strings section but st_value is >+ // a load address so we need to subtract an offset before looking it >+ // up in that section. >+ const size_t offset = sym->st_value - strings_offset; >+ if (offset < strings_size) >+ { >+ const char* ns = strings_data + offset; >+ if (*ns) >+ ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); constexpr auto name_offset = sizeof("__kstrtabns_"); ABG_ASSERT(namespaces.emplace(name.substr(name_offset), ns).second); Also, consider limiting the string length to the section end to not accidentally read beyond. Cheers, Matthias >+ } >+ continue; >+ } > > // filter out uninteresting entries and only keep functions/variables for > // now. The rest might be interesting in the future though. >@@ -402,6 +440,17 @@ symtab::load_(Elf* elf_handle, > symbol->set_crc(crc_entry.second); > } > >+ // Now add the namespaces >+ for (const auto& namespace_entry : namespaces) >+ { >+ const auto r = name_symbol_map_.find(namespace_entry.first); >+ if (r == name_symbol_map_.end()) >+ continue; >+ >+ for (const auto& symbol : r->second) >+ symbol->set_namespace({namespace_entry.second}); >+ } >+ > // sort the symbols for deterministic output > std::sort(symbols_.begin(), symbols_.end(), symbol_sort); > >diff --git a/src/abg-writer.cc b/src/abg-writer.cc >index 7802128d..4dfa72a6 100644 >--- a/src/abg-writer.cc >+++ b/src/abg-writer.cc >@@ -3136,6 +3136,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, > << std::hex << std::showbase << sym->get_crc() << "'" > << std::dec << std::noshowbase; > >+ if (sym->get_namespace().has_value()) >+ o << " ns='" >+ << sym->get_namespace().value() << "'"; >+ > o << "/>\n"; > > return true; >-- >2.35.1.723.g4982287a31-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 0/4] add symbol namespace support, update symbol CRC support 2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida 2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida 2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-16 16:30 ` Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida ` (4 more replies) 2 siblings, 5 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich New in v2: * updates following review (includng tweaking a function I copied from) * better offset checking and working extraction of namespace from .ko * rework of CRCs to be similar to namespaces (use of optional etc.) * test case additions / updates Giuliano Procida (4): optional: minor improvements crc_changed: eliminate copying of shared_ptr values add Linux kernel symbol namespace support Linux symbol CRCs: support 0 and report presence changes include/abg-cxx-compat.h | 30 +++++++-- include/abg-ir.h | 17 ++++-- src/abg-comp-filter.cc | 46 ++++++++++++-- src/abg-ir.cc | 47 ++++++++++---- src/abg-reader.cc | 15 +++-- src/abg-reporter-priv.cc | 38 ++++++++++-- src/abg-symtab-reader.cc | 61 +++++++++++++++++++ src/abg-writer.cc | 9 ++- tests/data/Makefile.am | 7 ++- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 ++++++ tests/test-abidiff.cc | 12 +++- tests/test-symtab.cc | 8 +-- 17 files changed, 322 insertions(+), 47 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 1/4] optional: minor improvements 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida @ 2022-03-16 16:30 ` Giuliano Procida 2022-03-17 10:56 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida ` (3 subsequent siblings) 4 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich This change makes minor improvements to the optional class used with pre-C++17 compilers. - adds operator== and operator!= - adds various missing noexcept (but not constexpr) decorations - defines operator bool in terms of has_value Note that some constexpr decorations would require C++17 anyway. * include/abg-cxx-compat.h (optional): Add operator== and operator!=. Add noexcept decorations. Tweak operator bool. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h index 443905c7..5c5943d0 100644 --- a/include/abg-cxx-compat.h +++ b/include/abg-cxx-compat.h @@ -45,7 +45,7 @@ public: optional(const T& value) : has_value_(true), value_(value) {} bool - has_value() const + has_value() const noexcept { return has_value_; } @@ -67,19 +67,19 @@ public: } const T& - operator*() const + operator*() const& noexcept { return value_; } T& - operator*() + operator*() & noexcept { return value_; } const T* - operator->() const + operator->() const noexcept { return &value_; } T* - operator->() + operator->() noexcept { return &value_; } optional& @@ -90,9 +90,27 @@ public: return *this; } - explicit operator bool() const { return has_value_; } + explicit operator bool() const noexcept { return has_value(); } }; +template <typename T, typename U> +bool +operator==(const optional<T>& lhs, const optional<U>& rhs) +{ + if (!lhs.has_value() && !rhs.has_value()) + return true; + if (!lhs.has_value() || !rhs.has_value()) + return false; + return lhs.value() == rhs.value(); +} + +template <typename T, typename U> +bool +operator!=(const optional<T>& lhs, const optional<U>& rhs) +{ + return !(lhs == rhs); +} + #endif } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 1/4] optional: minor improvements 2022-03-16 16:30 ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida @ 2022-03-17 10:56 ` Matthias Maennich 0 siblings, 0 replies; 37+ messages in thread From: Matthias Maennich @ 2022-03-17 10:56 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Wed, Mar 16, 2022 at 04:30:52PM +0000, Giuliano Procida wrote: >This change makes minor improvements to the optional class used with >pre-C++17 compilers. > >- adds operator== and operator!= >- adds various missing noexcept (but not constexpr) decorations >- defines operator bool in terms of has_value > >Note that some constexpr decorations would require C++17 anyway. > > * include/abg-cxx-compat.h (optional): Add operator== and > operator!=. Add noexcept decorations. Tweak operator bool. > >Signed-off-by: Giuliano Procida <gprocida@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Cheers, Matthias >--- > include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------ > 1 file changed, 24 insertions(+), 6 deletions(-) > >diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h >index 443905c7..5c5943d0 100644 >--- a/include/abg-cxx-compat.h >+++ b/include/abg-cxx-compat.h >@@ -45,7 +45,7 @@ public: > optional(const T& value) : has_value_(true), value_(value) {} > > bool >- has_value() const >+ has_value() const noexcept > { > return has_value_; > } >@@ -67,19 +67,19 @@ public: > } > > const T& >- operator*() const >+ operator*() const& noexcept > { return value_; } > > T& >- operator*() >+ operator*() & noexcept > { return value_; } > > const T* >- operator->() const >+ operator->() const noexcept > { return &value_; } > > T* >- operator->() >+ operator->() noexcept > { return &value_; } > > optional& >@@ -90,9 +90,27 @@ public: > return *this; > } > >- explicit operator bool() const { return has_value_; } >+ explicit operator bool() const noexcept { return has_value(); } > }; > >+template <typename T, typename U> >+bool >+operator==(const optional<T>& lhs, const optional<U>& rhs) >+{ >+ if (!lhs.has_value() && !rhs.has_value()) >+ return true; >+ if (!lhs.has_value() || !rhs.has_value()) >+ return false; >+ return lhs.value() == rhs.value(); >+} >+ >+template <typename T, typename U> >+bool >+operator!=(const optional<T>& lhs, const optional<U>& rhs) >+{ >+ return !(lhs == rhs); >+} >+ > #endif > } > >-- >2.35.1.894.gb6a874cedc-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida @ 2022-03-16 16:30 ` Giuliano Procida 2022-03-17 11:01 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida ` (2 subsequent siblings) 4 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich As pointed out in a review of similar code, it is possible to avoid copying a couple of shared pointers in this function, by taking references instead. This commit also splits declarations to one per line and removes the unnecessary parentheses around the return expression. * src/abg-comp-filter.cc (crc_changed): Take references to avoid std::shared_ptr copying. Split declarations into one per line. Remove unnecessary return expression parentheses. Signed-off-by: Giuliano Procida <gprocida@google.com> --- src/abg-comp-filter.cc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 56251274..f90fdc78 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -230,11 +230,13 @@ static bool crc_changed(const function_or_var_decl_sptr& f, const function_or_var_decl_sptr& s) { - const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc(); - return (crc_f != 0 && crc_s != 0 && crc_f != crc_s); + const auto crc_f = symbol_f->get_crc(); + const auto crc_s = symbol_s->get_crc(); + return crc_f != 0 && crc_s != 0 && crc_f != crc_s; } /// Test if the current diff tree node carries a CRC change in either a -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values 2022-03-16 16:30 ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-03-17 11:01 ` Matthias Maennich 0 siblings, 0 replies; 37+ messages in thread From: Matthias Maennich @ 2022-03-17 11:01 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team On Wed, Mar 16, 2022 at 04:30:53PM +0000, Giuliano Procida via Libabigail wrote: >As pointed out in a review of similar code, it is possible to avoid >copying a couple of shared pointers in this function, by taking >references instead. > >This commit also splits declarations to one per line and removes the >unnecessary parentheses around the return expression. > > * src/abg-comp-filter.cc (crc_changed): Take references to > avoid std::shared_ptr copying. Split declarations into one per > line. Remove unnecessary return expression parentheses. > >Signed-off-by: Giuliano Procida <gprocida@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Cheers, Matthias >--- > src/abg-comp-filter.cc | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc >index 56251274..f90fdc78 100644 >--- a/src/abg-comp-filter.cc >+++ b/src/abg-comp-filter.cc >@@ -230,11 +230,13 @@ static bool > crc_changed(const function_or_var_decl_sptr& f, > const function_or_var_decl_sptr& s) > { >- const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); >+ const auto& symbol_f = f->get_symbol(); >+ const auto& symbol_s = s->get_symbol(); > if (!symbol_f || !symbol_s) > return false; >- const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc(); >- return (crc_f != 0 && crc_s != 0 && crc_f != crc_s); >+ const auto crc_f = symbol_f->get_crc(); >+ const auto crc_s = symbol_s->get_crc(); >+ return crc_f != 0 && crc_s != 0 && crc_f != crc_s; > } > > /// Test if the current diff tree node carries a CRC change in either a >-- >2.35.1.894.gb6a874cedc-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 3/4] add Linux kernel symbol namespace support 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-03-16 16:30 ` Giuliano Procida 2022-03-17 11:57 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 4 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Bug 28954 - add Linux Kernel symbol namespace support Each Linux kernel symbol can be exported to a specified named namespace or left in the global (nameless) namespace. One complexity is that the offset symbol values require slightly different interpretation for vmlinux and .ko loadable modules as the former has a fixed load address but the latter are relocatable. The global namespace is explicitly represented as the empty string, at least when it comes to the value of __kstrtabns_FOO symbols, but the common interpretation is that such symbols lack an export namespace. I would rather not have to make use of "empty implies missing" in many places, so the code here represents namespace as optional<string> and only the symtab reader cares about empty strings in __ksymtab_strings. * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. (elf_symbol::create): Add ns argument. (elf_symbol::get_namespace): Declare new function. (elf_symbol::set_namespace): Declare new function. and set_namespace. * src/abg-comp-filter.cc (namespace_changed): Define new helper functions. (categorize_harmful_diff_node): Also call namespace_changed(). * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. (elf_symbol::priv::priv): Add namespace_ to initialisers. (elf_symbol::elf_symbol): Take new ns argument and pass it to priv constructor. (elf_symbol::create): Take new ns argument and pass it to elf_symbol constructor. (elf_symbol::get_namespace): Define new function. (elf_symbol::set_namespace): Define new function. * src/abg-reader.cc (build_elf_symbol): If namespace attribute is present, set symbol namespace. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If symbol namespaces differ, report this. * src/abg-symtab-reader.cc (symtab::load): Try to get __ksymtab_strings metadata and data. Use these to look up __kstrtabns_FOO namespace entries. Set symbol namespace where found. * src/abg-writer.cc (write_elf_symbol): Emit namespace attribute, if symbol has a namespace. * tests/data/Makefile.am: Add new test files. * tests/data/test-abidiff/test-namespace-0.xml: New test file. * tests/data/test-abidiff/test-namespace-1.xml: Likewise * tests/data/test-abidiff/test-namespace-report.txt: Likewise. * tests/test-abidiff.cc: Add new test case. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 9 +++ src/abg-comp-filter.cc | 40 +++++++++++- src/abg-ir.cc | 30 ++++++++- src/abg-reader.cc | 7 +++ src/abg-reporter-priv.cc | 18 ++++++ src/abg-symtab-reader.cc | 61 +++++++++++++++++++ src/abg-writer.cc | 3 + tests/data/Makefile.am | 3 + tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 ++++++ tests/test-abidiff.cc | 6 ++ 12 files changed, 221 insertions(+), 3 deletions(-) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt diff --git a/include/abg-ir.h b/include/abg-ir.h index a2f4e1a7..7c42bea7 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -21,6 +21,7 @@ #include <functional> #include <set> #include <unordered_map> +#include "abg-cxx-compat.h" #include "abg-fwd.h" #include "abg-hash.h" #include "abg-traverse.h" @@ -921,6 +922,7 @@ private: visibility vi, bool is_in_ksymtab = false, uint64_t crc = 0, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -946,6 +948,7 @@ public: visibility vi, bool is_in_ksymtab = false, uint64_t crc = 0, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); const environment* @@ -1023,6 +1026,12 @@ public: void set_crc(uint64_t crc); + const abg_compat::optional<std::string>& + get_namespace() const; + + void + set_namespace(const abg_compat::optional<std::string>& ns); + bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index f90fdc78..c179b6bd 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -256,6 +256,43 @@ crc_changed(const diff* diff) return false; } +/// Test if there was a function or variable namespace change. +/// +/// @param f the first function or variable to consider. +/// +/// @param s the second function or variable to consider. +/// +/// @return true if the test is positive, false otherwise. +template <typename function_or_var_decl_sptr> +static bool +namespace_changed(const function_or_var_decl_sptr& f, + const function_or_var_decl_sptr& s) +{ + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); + if (!symbol_f || !symbol_s) + return false; + return symbol_f->get_namespace() != symbol_s->get_namespace(); +} + +/// Test if the current diff tree node carries a namespace change in +/// either a function or a variable. +/// +/// @param diff the diff tree node to consider. +/// +/// @return true if the test is positive, false otherwise. +static bool +namespace_changed(const diff* diff) +{ + if (const function_decl_diff* d = + dynamic_cast<const function_decl_diff*>(diff)) + return namespace_changed(d->first_function_decl(), + d->second_function_decl()); + if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) + return namespace_changed(d->first_var(), d->second_var()); + return false; +} + /// Test if there was a function name change, but there there was no /// change in name of the underlying symbol. IOW, if the name of a /// function changed, but the symbol of the new function is equal to @@ -1783,7 +1820,8 @@ categorize_harmful_diff_node(diff *d, bool pre) || non_static_data_member_added_or_removed(d) || base_classes_added_or_removed(d) || has_harmful_enum_change(d) - || crc_changed(d))) + || crc_changed(d) + || namespace_changed(d))) category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; if (has_virtual_mem_fn_change(d)) diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 0ef5e8b2..78154622 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1728,6 +1728,7 @@ struct elf_symbol::priv bool is_common_; bool is_in_ksymtab_; uint64_t crc_; + abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1745,6 +1746,7 @@ struct elf_symbol::priv is_common_(false), is_in_ksymtab_(false), crc_(0), + namespace_(), is_suppressed_(false) {} @@ -1760,6 +1762,7 @@ struct elf_symbol::priv elf_symbol::visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), index_(i), @@ -1773,6 +1776,7 @@ struct elf_symbol::priv is_common_(c), is_in_ksymtab_(is_in_ksymtab), crc_(crc), + namespace_(ns), is_suppressed_(is_suppressed) { if (!is_common_) @@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() /// @param vi the visibility of the symbol. /// /// @param crc the CRC (modversions) value of Linux Kernel symbols +/// +/// @param ns the namespace of Linux Kernel symbols, if any elf_symbol::elf_symbol(const environment* e, size_t i, size_t s, @@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, i, @@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, vi, is_in_ksymtab, crc, + ns, is_suppressed)) {} @@ -1886,6 +1894,8 @@ elf_symbol::create() /// /// @param crc the CRC (modversions) value of Linux Kernel symbols /// +/// @param ns the namespace of Linux Kernel symbols, if any +/// /// @return a (smart) pointer to a newly created instance of @ref /// elf_symbol. elf_symbol_sptr @@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, visibility vi, bool is_in_ksymtab, uint64_t crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, - is_in_ksymtab, crc, is_suppressed)); + is_in_ksymtab, crc, ns, is_suppressed)); sym->priv_->main_symbol_ = sym; return sym; } @@ -1927,7 +1938,8 @@ textually_equals(const elf_symbol&l, && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() && (l.get_crc() == 0 || r.get_crc() == 0 - || l.get_crc() == r.get_crc())); + || l.get_crc() == r.get_crc()) + && l.get_namespace() == r.get_namespace()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2147,6 +2159,20 @@ void elf_symbol::set_crc(uint64_t crc) {priv_->crc_ = crc;} +/// Getter of the 'namespace' property. +/// +/// @return the namespace for Linux Kernel symbols, if any +const abg_compat::optional<std::string>& +elf_symbol::get_namespace() const +{return priv_->namespace_;} + +/// Setter of the 'namespace' property. +/// +/// @param ns the new namespace for Linux Kernel symbols, if any +void +elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) +{priv_->namespace_ = ns;} + /// Getter for the 'is-suppressed' property. /// /// @return true iff the current symbol has been suppressed by a diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 31885692..0a8ecd70 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, if (crc != 0) e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) + { + std::string ns; + xml::xml_char_sptr_to_string(s, ns); + e->set_namespace(ns); + } + return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 7012f5dc..9ebcdf00 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1158,6 +1158,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << std::noshowbase << std::dec << "\n"; } + + const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); + const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); + if (ns1 != ns2) + { + const std::string none = "(none)"; + out << indent << "namespace changed from "; + if (ns1.has_value()) + out << "'" << ns1.value() << "'"; + else + out << none; + out << " to "; + if (ns2.has_value()) + out << "'" << ns2.value() << "'"; + else + out << none; + out << "\n"; + } } /// For a given symbol, emit a string made of its name and version. diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc index b42ce87d..e59bd62c 100644 --- a/src/abg-symtab-reader.cc +++ b/src/abg-symtab-reader.cc @@ -232,9 +232,33 @@ symtab::load_(Elf* elf_handle, return false; } + // The __kstrtab_strings sections is basically an ELF strtab but does not + // support elf_strptr lookups. A single call to elf_getdata gives a handle to + // section data. + // + // The value of a __kstrtabns_FOO (or other similar) symbol is an address + // within the __kstrtab_strings section. To look up the string value, we need + // to translate from load address to section offset by subtracting the base + // address of the section. + Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); + size_t strings_offset = 0; + const char* strings_data = nullptr; + size_t strings_size = 0; + if (strings_section) + { + GElf_Shdr strings_sheader; + gelf_getshdr(strings_section, &strings_sheader); + strings_offset = strings_sheader.sh_addr; + Elf_Data* data = elf_getdata(strings_section, nullptr); + ABG_ASSERT(data->d_off == 0); + strings_data = reinterpret_cast<const char *>(data->d_buf); + strings_size = data->d_size; + } + const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); std::unordered_set<std::string> exported_kernel_symbols; std::unordered_map<std::string, uint64_t> crc_values; + std::unordered_map<std::string, std::string> namespaces; const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); @@ -288,6 +312,32 @@ symtab::load_(Elf* elf_handle, ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); continue; } + if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) + { + // This symbol lives in the __ksymtab_strings section but st_value is + // a load address so, for vmlinux only, we need to subtract an offset + // before looking it up in that section. Kernel module offsets must + // not be adjusted. The simple way to distinguish the cases is to see + // if the adjustment would overflow. + const size_t value = sym->st_value; + const size_t offset = + value < strings_offset ? value : value - strings_offset; + // check offset + ABG_ASSERT(offset < strings_size); + // find the terminating NULL + const char* first = strings_data + offset; + const char* last = strings_data + strings_size; + const char* limit = std::find(first, last, 0); + // check NULL found + ABG_ASSERT(limit < last); + // interpret the empty namespace name as no namespace name + if (first < limit) + { + const std::string ns(first, limit - first); + ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); + } + continue; + } // filter out uninteresting entries and only keep functions/variables for // now. The rest might be interesting in the future though. @@ -402,6 +452,17 @@ symtab::load_(Elf* elf_handle, symbol->set_crc(crc_entry.second); } + // Now add the namespaces + for (const auto& namespace_entry : namespaces) + { + const auto r = name_symbol_map_.find(namespace_entry.first); + if (r == name_symbol_map_.end()) + continue; + + for (const auto& symbol : r->second) + symbol->set_namespace(namespace_entry.second); + } + // sort the symbols for deterministic output std::sort(symbols_.begin(), symbols_.end(), symbol_sort); diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 7802128d..692abc75 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, << std::hex << std::showbase << sym->get_crc() << "'" << std::dec << std::noshowbase; + if (sym->get_namespace().has_value()) + o << " namespace='" << sym->get_namespace().value() << "'"; + o << "/>\n"; return true; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index a7eb7ff0..7ea24d58 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -91,6 +91,9 @@ test-abidiff/test-crc-0.xml \ test-abidiff/test-crc-1.xml \ test-abidiff/test-crc-2.xml \ test-abidiff/test-crc-report.txt \ +test-abidiff/test-namespace-0.xml \ +test-abidiff/test-namespace-1.xml \ +test-abidiff/test-namespace-report.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml new file mode 100644 index 00000000..5a9f5cd2 --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-0.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml new file mode 100644 index 00000000..9814844b --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-1.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt new file mode 100644 index 00000000..d2c421ed --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-report.txt @@ -0,0 +1,17 @@ +Functions changes summary: 0 Removed, 0 Changed, 0 Added function +Variables changes summary: 0 Removed, 4 Changed, 0 Added variables + +4 Changed variables: + + [C] 'int v1' was changed: + namespace changed from (none) to 'this' + + [C] 'int v2' was changed: + namespace changed from 'this' to 'that' + + [C] 'int v3' was changed: + namespace changed from 'that' to '' + + [C] 'int v4' was changed: + namespace changed from '' to (none) + diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index 32858ece..009f7de5 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -128,6 +128,12 @@ static InOutSpec specs[] = "data/test-abidiff/test-crc-report.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, + { + "data/test-abidiff/test-namespace-0.xml", + "data/test-abidiff/test-namespace-1.xml", + "data/test-abidiff/test-namespace-report.txt", + "output/test-abidiff/test-namespace-report-0-1.txt" + }, { "data/test-abidiff/test-PR27616-v0.xml", "data/test-abidiff/test-PR27616-v1.xml", -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 3/4] add Linux kernel symbol namespace support 2022-03-16 16:30 ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-17 11:57 ` Matthias Maennich 0 siblings, 0 replies; 37+ messages in thread From: Matthias Maennich @ 2022-03-17 11:57 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Wed, Mar 16, 2022 at 04:30:54PM +0000, Giuliano Procida wrote: >Bug 28954 - add Linux Kernel symbol namespace support > >Each Linux kernel symbol can be exported to a specified named >namespace or left in the global (nameless) namespace. > >One complexity is that the offset symbol values require slightly >different interpretation for vmlinux and .ko loadable modules as the >former has a fixed load address but the latter are relocatable. > >The global namespace is explicitly represented as the empty string, at >least when it comes to the value of __kstrtabns_FOO symbols, but the >common interpretation is that such symbols lack an export namespace. > >I would rather not have to make use of "empty implies missing" in many >places, so the code here represents namespace as optional<string> and >only the symtab reader cares about empty strings in __ksymtab_strings. > > * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. > (elf_symbol::create): Add ns argument. > (elf_symbol::get_namespace): Declare new function. > (elf_symbol::set_namespace): Declare new function. > and set_namespace. > * src/abg-comp-filter.cc (namespace_changed): Define new > helper functions. > (categorize_harmful_diff_node): Also call namespace_changed(). > * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. > (elf_symbol::priv::priv): Add namespace_ to initialisers. > (elf_symbol::elf_symbol): Take new ns argument and pass it to > priv constructor. > (elf_symbol::create): Take new ns argument and pass it to > elf_symbol constructor. > (elf_symbol::get_namespace): Define new function. > (elf_symbol::set_namespace): Define new function. > * src/abg-reader.cc (build_elf_symbol): If namespace > attribute is present, set symbol namespace. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If > symbol namespaces differ, report this. > * src/abg-symtab-reader.cc (symtab::load): Try to get > __ksymtab_strings metadata and data. Use these to look up > __kstrtabns_FOO namespace entries. Set symbol namespace where > found. > * src/abg-writer.cc (write_elf_symbol): Emit namespace > attribute, if symbol has a namespace. > * tests/data/Makefile.am: Add new test files. > * tests/data/test-abidiff/test-namespace-0.xml: New test file. > * tests/data/test-abidiff/test-namespace-1.xml: Likewise > * tests/data/test-abidiff/test-namespace-report.txt: Likewise. > * tests/test-abidiff.cc: Add new test case. > Perhaps we can add an actual test case for a kernel module at least? There are already test cases in test-symtab that could be forked for that case. >Signed-off-by: Giuliano Procida <gprocida@google.com> >--- > include/abg-ir.h | 9 +++ > src/abg-comp-filter.cc | 40 +++++++++++- > src/abg-ir.cc | 30 ++++++++- > src/abg-reader.cc | 7 +++ > src/abg-reporter-priv.cc | 18 ++++++ > src/abg-symtab-reader.cc | 61 +++++++++++++++++++ > src/abg-writer.cc | 3 + > tests/data/Makefile.am | 3 + > tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ > tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ > .../test-abidiff/test-namespace-report.txt | 17 ++++++ > tests/test-abidiff.cc | 6 ++ > 12 files changed, 221 insertions(+), 3 deletions(-) > create mode 100644 tests/data/test-abidiff/test-namespace-0.xml > create mode 100644 tests/data/test-abidiff/test-namespace-1.xml > create mode 100644 tests/data/test-abidiff/test-namespace-report.txt > >diff --git a/include/abg-ir.h b/include/abg-ir.h >index a2f4e1a7..7c42bea7 100644 >--- a/include/abg-ir.h >+++ b/include/abg-ir.h >@@ -21,6 +21,7 @@ > #include <functional> > #include <set> > #include <unordered_map> >+#include "abg-cxx-compat.h" > #include "abg-fwd.h" > #include "abg-hash.h" > #include "abg-traverse.h" >@@ -921,6 +922,7 @@ private: > visibility vi, > bool is_in_ksymtab = false, > uint64_t crc = 0, >+ const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > > elf_symbol(const elf_symbol&); >@@ -946,6 +948,7 @@ public: > visibility vi, > bool is_in_ksymtab = false, > uint64_t crc = 0, >+ const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > > const environment* >@@ -1023,6 +1026,12 @@ public: > void > set_crc(uint64_t crc); > >+ const abg_compat::optional<std::string>& >+ get_namespace() const; >+ >+ void >+ set_namespace(const abg_compat::optional<std::string>& ns); >+ > bool > is_suppressed() const; > >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc >index f90fdc78..c179b6bd 100644 >--- a/src/abg-comp-filter.cc >+++ b/src/abg-comp-filter.cc >@@ -256,6 +256,43 @@ crc_changed(const diff* diff) > return false; > } > >+/// Test if there was a function or variable namespace change. >+/// >+/// @param f the first function or variable to consider. >+/// >+/// @param s the second function or variable to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+template <typename function_or_var_decl_sptr> >+static bool >+namespace_changed(const function_or_var_decl_sptr& f, >+ const function_or_var_decl_sptr& s) >+{ >+ const auto& symbol_f = f->get_symbol(); >+ const auto& symbol_s = s->get_symbol(); >+ if (!symbol_f || !symbol_s) >+ return false; >+ return symbol_f->get_namespace() != symbol_s->get_namespace(); >+} >+ >+/// Test if the current diff tree node carries a namespace change in >+/// either a function or a variable. >+/// >+/// @param diff the diff tree node to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+static bool >+namespace_changed(const diff* diff) >+{ >+ if (const function_decl_diff* d = >+ dynamic_cast<const function_decl_diff*>(diff)) >+ return namespace_changed(d->first_function_decl(), >+ d->second_function_decl()); >+ if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) >+ return namespace_changed(d->first_var(), d->second_var()); >+ return false; >+} >+ > /// Test if there was a function name change, but there there was no > /// change in name of the underlying symbol. IOW, if the name of a > /// function changed, but the symbol of the new function is equal to >@@ -1783,7 +1820,8 @@ categorize_harmful_diff_node(diff *d, bool pre) > || non_static_data_member_added_or_removed(d) > || base_classes_added_or_removed(d) > || has_harmful_enum_change(d) >- || crc_changed(d))) >+ || crc_changed(d) >+ || namespace_changed(d))) > category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; > > if (has_virtual_mem_fn_change(d)) >diff --git a/src/abg-ir.cc b/src/abg-ir.cc >index 0ef5e8b2..78154622 100644 >--- a/src/abg-ir.cc >+++ b/src/abg-ir.cc >@@ -1728,6 +1728,7 @@ struct elf_symbol::priv > bool is_common_; > bool is_in_ksymtab_; > uint64_t crc_; >+ abg_compat::optional<std::string> namespace_; > bool is_suppressed_; > elf_symbol_wptr main_symbol_; > elf_symbol_wptr next_alias_; >@@ -1745,6 +1746,7 @@ struct elf_symbol::priv > is_common_(false), > is_in_ksymtab_(false), > crc_(0), >+ namespace_(), > is_suppressed_(false) > {} > >@@ -1760,6 +1762,7 @@ struct elf_symbol::priv > elf_symbol::visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : env_(e), > index_(i), >@@ -1773,6 +1776,7 @@ struct elf_symbol::priv > is_common_(c), > is_in_ksymtab_(is_in_ksymtab), > crc_(crc), >+ namespace_(ns), > is_suppressed_(is_suppressed) > { > if (!is_common_) >@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() > /// @param vi the visibility of the symbol. > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols >+/// >+/// @param ns the namespace of Linux Kernel symbols, if any > elf_symbol::elf_symbol(const environment* e, > size_t i, > size_t s, >@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, > visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : priv_(new priv(e, > i, >@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, > vi, > is_in_ksymtab, > crc, >+ ns, > is_suppressed)) > {} > >@@ -1886,6 +1894,8 @@ elf_symbol::create() > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols > /// >+/// @param ns the namespace of Linux Kernel symbols, if any >+/// > /// @return a (smart) pointer to a newly created instance of @ref > /// elf_symbol. > elf_symbol_sptr >@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, > visibility vi, > bool is_in_ksymtab, > uint64_t crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > { > elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, >- is_in_ksymtab, crc, is_suppressed)); >+ is_in_ksymtab, crc, ns, is_suppressed)); > sym->priv_->main_symbol_ = sym; > return sym; > } >@@ -1927,7 +1938,8 @@ textually_equals(const elf_symbol&l, > && l.is_common_symbol() == r.is_common_symbol() > && l.get_version() == r.get_version() > && (l.get_crc() == 0 || r.get_crc() == 0 >- || l.get_crc() == r.get_crc())); >+ || l.get_crc() == r.get_crc()) >+ && l.get_namespace() == r.get_namespace()); > > if (equals && l.is_variable()) > // These are variable symbols. Let's compare their symbol size. >@@ -2147,6 +2159,20 @@ void > elf_symbol::set_crc(uint64_t crc) > {priv_->crc_ = crc;} > >+/// Getter of the 'namespace' property. >+/// >+/// @return the namespace for Linux Kernel symbols, if any >+const abg_compat::optional<std::string>& >+elf_symbol::get_namespace() const >+{return priv_->namespace_;} >+ >+/// Setter of the 'namespace' property. >+/// >+/// @param ns the new namespace for Linux Kernel symbols, if any >+void >+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) >+{priv_->namespace_ = ns;} >+ > /// Getter for the 'is-suppressed' property. > /// > /// @return true iff the current symbol has been suppressed by a >diff --git a/src/abg-reader.cc b/src/abg-reader.cc >index 31885692..0a8ecd70 100644 >--- a/src/abg-reader.cc >+++ b/src/abg-reader.cc >@@ -3269,6 +3269,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > if (crc != 0) > e->set_crc(crc); > >+ if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) >+ { >+ std::string ns; >+ xml::xml_char_sptr_to_string(s, ns); >+ e->set_namespace(ns); >+ } >+ > return e; > } > >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc >index 7012f5dc..9ebcdf00 100644 >--- a/src/abg-reporter-priv.cc >+++ b/src/abg-reporter-priv.cc >@@ -1158,6 +1158,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, > << std::noshowbase << std::dec > << "\n"; > } >+ >+ const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); >+ const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); >+ if (ns1 != ns2) >+ { >+ const std::string none = "(none)"; >+ out << indent << "namespace changed from "; >+ if (ns1.has_value()) >+ out << "'" << ns1.value() << "'"; >+ else >+ out << none; >+ out << " to "; >+ if (ns2.has_value()) >+ out << "'" << ns2.value() << "'"; >+ else >+ out << none; >+ out << "\n"; >+ } > } > > /// For a given symbol, emit a string made of its name and version. >diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc >index b42ce87d..e59bd62c 100644 >--- a/src/abg-symtab-reader.cc >+++ b/src/abg-symtab-reader.cc >@@ -232,9 +232,33 @@ symtab::load_(Elf* elf_handle, > return false; > } > >+ // The __kstrtab_strings sections is basically an ELF strtab but does not >+ // support elf_strptr lookups. A single call to elf_getdata gives a handle to >+ // section data. >+ // >+ // The value of a __kstrtabns_FOO (or other similar) symbol is an address >+ // within the __kstrtab_strings section. To look up the string value, we need >+ // to translate from load address to section offset by subtracting the base >+ // address of the section. >+ Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); >+ size_t strings_offset = 0; >+ const char* strings_data = nullptr; >+ size_t strings_size = 0; >+ if (strings_section) >+ { >+ GElf_Shdr strings_sheader; >+ gelf_getshdr(strings_section, &strings_sheader); >+ strings_offset = strings_sheader.sh_addr; >+ Elf_Data* data = elf_getdata(strings_section, nullptr); >+ ABG_ASSERT(data->d_off == 0); >+ strings_data = reinterpret_cast<const char *>(data->d_buf); >+ strings_size = data->d_size; >+ } >+ > const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); > std::unordered_set<std::string> exported_kernel_symbols; > std::unordered_map<std::string, uint64_t> crc_values; >+ std::unordered_map<std::string, std::string> namespaces; > > const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); > const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); >@@ -288,6 +312,32 @@ symtab::load_(Elf* elf_handle, > ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); > continue; > } >+ if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) >+ { >+ // This symbol lives in the __ksymtab_strings section but st_value is >+ // a load address so, for vmlinux only, we need to subtract an offset >+ // before looking it up in that section. Kernel module offsets must >+ // not be adjusted. The simple way to distinguish the cases is to see >+ // if the adjustment would overflow. Perhaps we can distinguish that from the ELF type? Kernel modules appear to be ET_REL. See elf_file_type() in abg-dwarf-reader.cc, a helper that could also live in abg-elf-helpers.cc actually. Cheers, Matthias >+ const size_t value = sym->st_value; >+ const size_t offset = >+ value < strings_offset ? value : value - strings_offset; >+ // check offset >+ ABG_ASSERT(offset < strings_size); >+ // find the terminating NULL >+ const char* first = strings_data + offset; >+ const char* last = strings_data + strings_size; >+ const char* limit = std::find(first, last, 0); >+ // check NULL found >+ ABG_ASSERT(limit < last); >+ // interpret the empty namespace name as no namespace name >+ if (first < limit) >+ { >+ const std::string ns(first, limit - first); >+ ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); >+ } >+ continue; >+ } > > // filter out uninteresting entries and only keep functions/variables for > // now. The rest might be interesting in the future though. >@@ -402,6 +452,17 @@ symtab::load_(Elf* elf_handle, > symbol->set_crc(crc_entry.second); > } > >+ // Now add the namespaces >+ for (const auto& namespace_entry : namespaces) >+ { >+ const auto r = name_symbol_map_.find(namespace_entry.first); >+ if (r == name_symbol_map_.end()) >+ continue; >+ >+ for (const auto& symbol : r->second) >+ symbol->set_namespace(namespace_entry.second); >+ } >+ > // sort the symbols for deterministic output > std::sort(symbols_.begin(), symbols_.end(), symbol_sort); > >diff --git a/src/abg-writer.cc b/src/abg-writer.cc >index 7802128d..692abc75 100644 >--- a/src/abg-writer.cc >+++ b/src/abg-writer.cc >@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, > << std::hex << std::showbase << sym->get_crc() << "'" > << std::dec << std::noshowbase; > >+ if (sym->get_namespace().has_value()) >+ o << " namespace='" << sym->get_namespace().value() << "'"; >+ > o << "/>\n"; > > return true; >diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am >index a7eb7ff0..7ea24d58 100644 >--- a/tests/data/Makefile.am >+++ b/tests/data/Makefile.am >@@ -91,6 +91,9 @@ test-abidiff/test-crc-0.xml \ > test-abidiff/test-crc-1.xml \ > test-abidiff/test-crc-2.xml \ > test-abidiff/test-crc-report.txt \ >+test-abidiff/test-namespace-0.xml \ >+test-abidiff/test-namespace-1.xml \ >+test-abidiff/test-namespace-report.txt \ > test-abidiff/test-PR27985-report.txt \ > test-abidiff/test-PR27985-v0.c \ > test-abidiff/test-PR27985-v0.o \ >diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml >new file mode 100644 >index 00000000..5a9f5cd2 >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-0.xml >@@ -0,0 +1,15 @@ >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> >+ <elf-variable-symbols> >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> >+ </elf-variable-symbols> >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> >+ </abi-instr> >+</abi-corpus> >diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml >new file mode 100644 >index 00000000..9814844b >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-1.xml >@@ -0,0 +1,15 @@ >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> >+ <elf-variable-symbols> >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> >+ </elf-variable-symbols> >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> >+ </abi-instr> >+</abi-corpus> >diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt >new file mode 100644 >index 00000000..d2c421ed >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-report.txt >@@ -0,0 +1,17 @@ >+Functions changes summary: 0 Removed, 0 Changed, 0 Added function >+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables >+ >+4 Changed variables: >+ >+ [C] 'int v1' was changed: >+ namespace changed from (none) to 'this' >+ >+ [C] 'int v2' was changed: >+ namespace changed from 'this' to 'that' >+ >+ [C] 'int v3' was changed: >+ namespace changed from 'that' to '' >+ >+ [C] 'int v4' was changed: >+ namespace changed from '' to (none) >+ >diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc >index 32858ece..009f7de5 100644 >--- a/tests/test-abidiff.cc >+++ b/tests/test-abidiff.cc >@@ -128,6 +128,12 @@ static InOutSpec specs[] = > "data/test-abidiff/test-crc-report.txt", > "output/test-abidiff/test-crc-report-1-2.txt" > }, >+ { >+ "data/test-abidiff/test-namespace-0.xml", >+ "data/test-abidiff/test-namespace-1.xml", >+ "data/test-abidiff/test-namespace-report.txt", >+ "output/test-abidiff/test-namespace-report-0-1.txt" >+ }, > { > "data/test-abidiff/test-PR27616-v0.xml", > "data/test-abidiff/test-PR27616-v1.xml", >-- >2.35.1.894.gb6a874cedc-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (2 preceding siblings ...) 2022-03-16 16:30 ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-16 16:30 ` Giuliano Procida 2022-03-17 12:01 ` Matthias Maennich 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 4 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-16 16:30 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich The CRC with value zero was used to mean "absent". This can be better modelled using optional. This commit makes this change and also tweaks reporting so that disappearing / appearing CRCs are noted. This should be essentially impossible unless CRCs are enabled / disabled altogether but would be very noteworthy otherwise. * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is now an optional defaulted to absent. (elf_symbol::create): Likewise. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_src): Now takes an optional uint64_t. * src/abg-comp-filter.cc (crc_changed): Simplify comparison. * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an optional uint64_t. (elf_symbol::priv::priv): Argument crc is now an optional uint64_t. (elf_symbol::elf_symbol): Likewise. (elf_symbol::create): Argument crc is now an optional uint64_t and defaults to absent. (textually_equals): Simplify comparison. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_crc): Now takes an optional uint64_t. * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same as other CRC values. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): Treat CRC 0 the same as other CRC values and also report changes to CRC presence. * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same as other CRC values. * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. * tests/data/test-abidiff/test-crc-report-0-1.txt: Report showing additional of CRCs. * tests/data/test-abidiff/test-crc-report-1-0.txt: Report showing removal of CRCs. * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed from tests/data/test-abidiff/test-crc-report.txt. * tests/test-abidiff.cc: Update test cases that no longer generate empty reports. * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 8 ++++---- src/abg-comp-filter.cc | 4 +--- src/abg-ir.cc | 19 +++++++++--------- src/abg-reader.cc | 8 ++------ src/abg-reporter-priv.cc | 20 ++++++++++++++----- src/abg-writer.cc | 6 +++--- tests/data/Makefile.am | 4 +++- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/test-abidiff.cc | 6 +++--- tests/test-symtab.cc | 8 ++++---- 12 files changed, 76 insertions(+), 39 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) diff --git a/include/abg-ir.h b/include/abg-ir.h index 7c42bea7..7c558828 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -921,7 +921,7 @@ private: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); @@ -947,7 +947,7 @@ public: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); @@ -1020,11 +1020,11 @@ public: void set_is_in_ksymtab(bool is_in_ksymtab); - uint64_t + const abg_compat::optional<uint64_t>& get_crc() const; void - set_crc(uint64_t crc); + set_crc(const abg_compat::optional<uint64_t>& crc); const abg_compat::optional<std::string>& get_namespace() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index c179b6bd..22da5244 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f, const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(); - const auto crc_s = symbol_s->get_crc(); - return crc_f != 0 && crc_s != 0 && crc_f != crc_s; + return symbol_f->get_crc() != symbol_s->get_crc(); } /// Test if the current diff tree node carries a CRC change in either a diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 78154622..f90be2e4 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1727,7 +1727,7 @@ struct elf_symbol::priv // STT_COMMON definition of that name that has the largest size. bool is_common_; bool is_in_ksymtab_; - uint64_t crc_; + abg_compat::optional<uint64_t> crc_; abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; @@ -1745,7 +1745,7 @@ struct elf_symbol::priv is_defined_(false), is_common_(false), is_in_ksymtab_(false), - crc_(0), + crc_(), namespace_(), is_suppressed_(false) {} @@ -1761,7 +1761,7 @@ struct elf_symbol::priv const elf_symbol::version& ve, elf_symbol::visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), @@ -1835,7 +1835,7 @@ elf_symbol::elf_symbol(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, @@ -1910,7 +1910,7 @@ elf_symbol::create(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, const abg_compat::optional<std::string>& ns, bool is_suppressed) { @@ -1937,8 +1937,7 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && (l.get_crc() == 0 || r.get_crc() == 0 - || l.get_crc() == r.get_crc()) + && l.get_crc() == r.get_crc() && l.get_namespace() == r.get_namespace()); if (equals && l.is_variable()) @@ -2147,8 +2146,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab) /// Getter of the 'crc' property. /// -/// @return the CRC (modversions) value for Linux Kernel symbols (if present) -uint64_t +/// @return the CRC (modversions) value for Linux Kernel symbols, if any +const abg_compat::optional<uint64_t>& elf_symbol::get_crc() const {return priv_->crc_;} @@ -2156,7 +2155,7 @@ elf_symbol::get_crc() const /// /// @param crc the new CRC (modversions) value for Linux Kernel symbols void -elf_symbol::set_crc(uint64_t crc) +elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} /// Getter of the 'namespace' property. diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 0a8ecd70..f9e420f1 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, is_default_version = true; } - uint64_t crc = 0; - if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) - crc = strtoull(CHAR_STR(s), NULL, 0); - elf_symbol::type type = elf_symbol::NOTYPE_TYPE; read_elf_symbol_type(node, type); @@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, e->set_is_suppressed(is_suppressed); - if (crc != 0) - e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) + e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) { diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 9ebcdf00..9ad733f4 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << "\n"; } - if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0 - && symbol1->get_crc() != symbol2->get_crc()) + const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc(); + const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc(); + if (crc1 != crc2) { + const std::string none = "(none)"; out << indent << "CRC (modversions) changed from " - << std::showbase << std::hex - << symbol1->get_crc() << " to " << symbol2->get_crc() - << std::noshowbase << std::dec + << std::showbase << std::hex; + if (crc1.has_value()) + out << crc1.value(); + else + out << none; + out << " to "; + if (crc2.has_value()) + out << crc2.value(); + else + out << none; + out << std::noshowbase << std::dec << "\n"; } diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 692abc75..03fb658b 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, if (sym->is_common_symbol()) o << " is-common='yes'"; - if (sym->get_crc() != 0) + if (sym->get_crc().has_value()) o << " crc='" - << std::hex << std::showbase << sym->get_crc() << "'" - << std::dec << std::noshowbase; + << std::hex << std::showbase << sym->get_crc().value() + << std::dec << std::noshowbase << "'"; if (sym->get_namespace().has_value()) o << " namespace='" << sym->get_namespace().value() << "'"; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index 7ea24d58..7388697b 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml \ test-abidiff/test-crc-0.xml \ test-abidiff/test-crc-1.xml \ test-abidiff/test-crc-2.xml \ -test-abidiff/test-crc-report.txt \ +test-abidiff/test-crc-report-0-1.txt \ +test-abidiff/test-crc-report-1-0.txt \ +test-abidiff/test-crc-report-1-2.txt \ test-abidiff/test-namespace-0.xml \ test-abidiff/test-namespace-1.xml \ test-abidiff/test-namespace-report.txt \ diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt new file mode 100644 index 00000000..0db42f68 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-0-1.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from (none) to 0xe52d5bcf + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from (none) to 0xee94d699 + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from (none) to 0x41336c46 + diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt new file mode 100644 index 00000000..e11f29c1 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-1-0.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from 0xe52d5bcf to (none) + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from 0xee94d699 to (none) + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from 0x41336c46 to (none) + diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt similarity index 100% rename from tests/data/test-abidiff/test-crc-report.txt rename to tests/data/test-abidiff/test-crc-report-1-2.txt diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index 009f7de5..93e44d6a 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -113,19 +113,19 @@ static InOutSpec specs[] = { "data/test-abidiff/test-crc-0.xml", "data/test-abidiff/test-crc-1.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-0-1.txt", "output/test-abidiff/test-crc-report-0-1.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-0.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-1-0.txt", "output/test-abidiff/test-crc-report-1-0.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-2.xml", - "data/test-abidiff/test-crc-report.txt", + "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, { diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc index 7d7a2df0..85303563 100644 --- a/tests/test-symtab.cc +++ b/tests/test-symtab.cc @@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]") { const std::string binary = base_path + "one_of_each.ko"; const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2); - CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0); - CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0); + CHECK(corpus->lookup_function_symbol("exported_function")->get_crc()); + CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc()); } } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes 2022-03-16 16:30 ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-03-17 12:01 ` Matthias Maennich 0 siblings, 0 replies; 37+ messages in thread From: Matthias Maennich @ 2022-03-17 12:01 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Wed, Mar 16, 2022 at 04:30:55PM +0000, Giuliano Procida wrote: >The CRC with value zero was used to mean "absent". This can be better >modelled using optional. > >This commit makes this change and also tweaks reporting so that >disappearing / appearing CRCs are noted. This should be essentially >impossible unless CRCs are enabled / disabled altogether but would be >very noteworthy otherwise. > > * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is > now an optional defaulted to absent. > (elf_symbol::create): Likewise. > (elf_symbol::get_crc): Now returns an optional uint64_t. > (elf_symbol::set_src): Now takes an optional uint64_t. > * src/abg-comp-filter.cc (crc_changed): Simplify comparison. > * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an > optional uint64_t. > (elf_symbol::priv::priv): Argument crc is now an optional > uint64_t. > (elf_symbol::elf_symbol): Likewise. > (elf_symbol::create): Argument crc is now an optional uint64_t > and defaults to absent. > (textually_equals): Simplify comparison. > (elf_symbol::get_crc): Now returns an optional uint64_t. > (elf_symbol::set_crc): Now takes an optional uint64_t. > * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same > as other CRC values. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): > Treat CRC 0 the same as other CRC values and also report > changes to CRC presence. > * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same > as other CRC values. > * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt > and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. > * tests/data/test-abidiff/test-crc-report-0-1.txt: Report > showing additional of CRCs. > * tests/data/test-abidiff/test-crc-report-1-0.txt: Report > showing removal of CRCs. > * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed > from tests/data/test-abidiff/test-crc-report.txt. > * tests/test-abidiff.cc: Update test cases that no longer > generate empty reports. > * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. > >Signed-off-by: Giuliano Procida <gprocida@google.com> Reviewed-by: Matthias Maennich <maennich@google.com> Cheers, Matthias >--- > include/abg-ir.h | 8 ++++---- > src/abg-comp-filter.cc | 4 +--- > src/abg-ir.cc | 19 +++++++++--------- > src/abg-reader.cc | 8 ++------ > src/abg-reporter-priv.cc | 20 ++++++++++++++----- > src/abg-writer.cc | 6 +++--- > tests/data/Makefile.am | 4 +++- > .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++ > .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++ > ...crc-report.txt => test-crc-report-1-2.txt} | 0 > tests/test-abidiff.cc | 6 +++--- > tests/test-symtab.cc | 8 ++++---- > 12 files changed, 76 insertions(+), 39 deletions(-) > create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt > create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt > rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) > >diff --git a/include/abg-ir.h b/include/abg-ir.h >index 7c42bea7..7c558828 100644 >--- a/include/abg-ir.h >+++ b/include/abg-ir.h >@@ -921,7 +921,7 @@ private: > const version& ve, > visibility vi, > bool is_in_ksymtab = false, >- uint64_t crc = 0, >+ const abg_compat::optional<uint64_t>& crc = {}, > const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > >@@ -947,7 +947,7 @@ public: > const version& ve, > visibility vi, > bool is_in_ksymtab = false, >- uint64_t crc = 0, >+ const abg_compat::optional<uint64_t>& crc = {}, > const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > >@@ -1020,11 +1020,11 @@ public: > void > set_is_in_ksymtab(bool is_in_ksymtab); > >- uint64_t >+ const abg_compat::optional<uint64_t>& > get_crc() const; > > void >- set_crc(uint64_t crc); >+ set_crc(const abg_compat::optional<uint64_t>& crc); > > const abg_compat::optional<std::string>& > get_namespace() const; >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc >index c179b6bd..22da5244 100644 >--- a/src/abg-comp-filter.cc >+++ b/src/abg-comp-filter.cc >@@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f, > const auto& symbol_s = s->get_symbol(); > if (!symbol_f || !symbol_s) > return false; >- const auto crc_f = symbol_f->get_crc(); >- const auto crc_s = symbol_s->get_crc(); >- return crc_f != 0 && crc_s != 0 && crc_f != crc_s; >+ return symbol_f->get_crc() != symbol_s->get_crc(); > } > > /// Test if the current diff tree node carries a CRC change in either a >diff --git a/src/abg-ir.cc b/src/abg-ir.cc >index 78154622..f90be2e4 100644 >--- a/src/abg-ir.cc >+++ b/src/abg-ir.cc >@@ -1727,7 +1727,7 @@ struct elf_symbol::priv > // STT_COMMON definition of that name that has the largest size. > bool is_common_; > bool is_in_ksymtab_; >- uint64_t crc_; >+ abg_compat::optional<uint64_t> crc_; > abg_compat::optional<std::string> namespace_; > bool is_suppressed_; > elf_symbol_wptr main_symbol_; >@@ -1745,7 +1745,7 @@ struct elf_symbol::priv > is_defined_(false), > is_common_(false), > is_in_ksymtab_(false), >- crc_(0), >+ crc_(), > namespace_(), > is_suppressed_(false) > {} >@@ -1761,7 +1761,7 @@ struct elf_symbol::priv > const elf_symbol::version& ve, > elf_symbol::visibility vi, > bool is_in_ksymtab, >- uint64_t crc, >+ const abg_compat::optional<uint64_t>& crc, > const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : env_(e), >@@ -1835,7 +1835,7 @@ elf_symbol::elf_symbol(const environment* e, > const version& ve, > visibility vi, > bool is_in_ksymtab, >- uint64_t crc, >+ const abg_compat::optional<uint64_t>& crc, > const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : priv_(new priv(e, >@@ -1910,7 +1910,7 @@ elf_symbol::create(const environment* e, > const version& ve, > visibility vi, > bool is_in_ksymtab, >- uint64_t crc, >+ const abg_compat::optional<uint64_t>& crc, > const abg_compat::optional<std::string>& ns, > bool is_suppressed) > { >@@ -1937,8 +1937,7 @@ textually_equals(const elf_symbol&l, > && l.is_defined() == r.is_defined() > && l.is_common_symbol() == r.is_common_symbol() > && l.get_version() == r.get_version() >- && (l.get_crc() == 0 || r.get_crc() == 0 >- || l.get_crc() == r.get_crc()) >+ && l.get_crc() == r.get_crc() > && l.get_namespace() == r.get_namespace()); > > if (equals && l.is_variable()) >@@ -2147,8 +2146,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab) > > /// Getter of the 'crc' property. > /// >-/// @return the CRC (modversions) value for Linux Kernel symbols (if present) >-uint64_t >+/// @return the CRC (modversions) value for Linux Kernel symbols, if any >+const abg_compat::optional<uint64_t>& > elf_symbol::get_crc() const > {return priv_->crc_;} > >@@ -2156,7 +2155,7 @@ elf_symbol::get_crc() const > /// > /// @param crc the new CRC (modversions) value for Linux Kernel symbols > void >-elf_symbol::set_crc(uint64_t crc) >+elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) > {priv_->crc_ = crc;} > > /// Getter of the 'namespace' property. >diff --git a/src/abg-reader.cc b/src/abg-reader.cc >index 0a8ecd70..f9e420f1 100644 >--- a/src/abg-reader.cc >+++ b/src/abg-reader.cc >@@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > is_default_version = true; > } > >- uint64_t crc = 0; >- if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) >- crc = strtoull(CHAR_STR(s), NULL, 0); >- > elf_symbol::type type = elf_symbol::NOTYPE_TYPE; > read_elf_symbol_type(node, type); > >@@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > > e->set_is_suppressed(is_suppressed); > >- if (crc != 0) >- e->set_crc(crc); >+ if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) >+ e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); > > if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) > { >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc >index 9ebcdf00..9ad733f4 100644 >--- a/src/abg-reporter-priv.cc >+++ b/src/abg-reporter-priv.cc >@@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, > << "\n"; > } > >- if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0 >- && symbol1->get_crc() != symbol2->get_crc()) >+ const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc(); >+ const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc(); >+ if (crc1 != crc2) > { >+ const std::string none = "(none)"; > out << indent << "CRC (modversions) changed from " >- << std::showbase << std::hex >- << symbol1->get_crc() << " to " << symbol2->get_crc() >- << std::noshowbase << std::dec >+ << std::showbase << std::hex; >+ if (crc1.has_value()) >+ out << crc1.value(); >+ else >+ out << none; >+ out << " to "; >+ if (crc2.has_value()) >+ out << crc2.value(); >+ else >+ out << none; >+ out << std::noshowbase << std::dec > << "\n"; > } > >diff --git a/src/abg-writer.cc b/src/abg-writer.cc >index 692abc75..03fb658b 100644 >--- a/src/abg-writer.cc >+++ b/src/abg-writer.cc >@@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, > if (sym->is_common_symbol()) > o << " is-common='yes'"; > >- if (sym->get_crc() != 0) >+ if (sym->get_crc().has_value()) > o << " crc='" >- << std::hex << std::showbase << sym->get_crc() << "'" >- << std::dec << std::noshowbase; >+ << std::hex << std::showbase << sym->get_crc().value() >+ << std::dec << std::noshowbase << "'"; > > if (sym->get_namespace().has_value()) > o << " namespace='" << sym->get_namespace().value() << "'"; >diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am >index 7ea24d58..7388697b 100644 >--- a/tests/data/Makefile.am >+++ b/tests/data/Makefile.am >@@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml \ > test-abidiff/test-crc-0.xml \ > test-abidiff/test-crc-1.xml \ > test-abidiff/test-crc-2.xml \ >-test-abidiff/test-crc-report.txt \ >+test-abidiff/test-crc-report-0-1.txt \ >+test-abidiff/test-crc-report-1-0.txt \ >+test-abidiff/test-crc-report-1-2.txt \ > test-abidiff/test-namespace-0.xml \ > test-abidiff/test-namespace-1.xml \ > test-abidiff/test-namespace-report.txt \ >diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt >new file mode 100644 >index 00000000..0db42f68 >--- /dev/null >+++ b/tests/data/test-abidiff/test-crc-report-0-1.txt >@@ -0,0 +1,16 @@ >+Functions changes summary: 0 Removed, 1 Changed, 0 Added function >+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables >+ >+1 function with some indirect sub-type change: >+ >+ [C] 'function void exported_function()' has some indirect sub-type changes: >+ CRC (modversions) changed from (none) to 0xe52d5bcf >+ >+2 Changed variables: >+ >+ [C] 'int exported_variable' was changed: >+ CRC (modversions) changed from (none) to 0xee94d699 >+ >+ [C] 'int exported_variable_gpl' was changed: >+ CRC (modversions) changed from (none) to 0x41336c46 >+ >diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt >new file mode 100644 >index 00000000..e11f29c1 >--- /dev/null >+++ b/tests/data/test-abidiff/test-crc-report-1-0.txt >@@ -0,0 +1,16 @@ >+Functions changes summary: 0 Removed, 1 Changed, 0 Added function >+Variables changes summary: 0 Removed, 2 Changed, 0 Added variables >+ >+1 function with some indirect sub-type change: >+ >+ [C] 'function void exported_function()' has some indirect sub-type changes: >+ CRC (modversions) changed from 0xe52d5bcf to (none) >+ >+2 Changed variables: >+ >+ [C] 'int exported_variable' was changed: >+ CRC (modversions) changed from 0xee94d699 to (none) >+ >+ [C] 'int exported_variable_gpl' was changed: >+ CRC (modversions) changed from 0x41336c46 to (none) >+ >diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt >similarity index 100% >rename from tests/data/test-abidiff/test-crc-report.txt >rename to tests/data/test-abidiff/test-crc-report-1-2.txt >diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc >index 009f7de5..93e44d6a 100644 >--- a/tests/test-abidiff.cc >+++ b/tests/test-abidiff.cc >@@ -113,19 +113,19 @@ static InOutSpec specs[] = > { > "data/test-abidiff/test-crc-0.xml", > "data/test-abidiff/test-crc-1.xml", >- "data/test-abidiff/empty-report.txt", >+ "data/test-abidiff/test-crc-report-0-1.txt", > "output/test-abidiff/test-crc-report-0-1.txt" > }, > { > "data/test-abidiff/test-crc-1.xml", > "data/test-abidiff/test-crc-0.xml", >- "data/test-abidiff/empty-report.txt", >+ "data/test-abidiff/test-crc-report-1-0.txt", > "output/test-abidiff/test-crc-report-1-0.txt" > }, > { > "data/test-abidiff/test-crc-1.xml", > "data/test-abidiff/test-crc-2.xml", >- "data/test-abidiff/test-crc-report.txt", >+ "data/test-abidiff/test-crc-report-1-2.txt", > "output/test-abidiff/test-crc-report-1-2.txt" > }, > { >diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc >index 7d7a2df0..85303563 100644 >--- a/tests/test-symtab.cc >+++ b/tests/test-symtab.cc >@@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]") > { > const std::string binary = base_path + "one_of_each.ko"; > const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2); >- CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0); >- CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0); >- CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0); >- CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0); >+ CHECK(corpus->lookup_function_symbol("exported_function")->get_crc()); >+ CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc()); >+ CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc()); >+ CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc()); > } > } >-- >2.35.1.894.gb6a874cedc-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v3 0/4] add symbol namespace support, update symbol CRC support 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (3 preceding siblings ...) 2022-03-16 16:30 ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-03-17 16:38 ` Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida ` (4 more replies) 4 siblings, 5 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich New in v3: * distinguish vmlinux vs .ko using ET_REL * more sensible patch order Giuliano Procida (4): crc_changed: eliminate copying of shared_ptr values optional: minor improvements Linux symbol CRCs: support 0 and report presence changes add Linux kernel symbol namespace support include/abg-cxx-compat.h | 30 +++++++-- include/abg-ir.h | 17 +++-- src/abg-comp-filter.cc | 46 +++++++++++-- src/abg-ir.cc | 47 +++++++++---- src/abg-reader.cc | 15 +++-- src/abg-reporter-priv.cc | 38 +++++++++-- src/abg-symtab-reader.cc | 66 +++++++++++++++++++ src/abg-writer.cc | 9 ++- tests/data/Makefile.am | 7 +- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 12 +++- tests/test-symtab.cc | 8 +-- 17 files changed, 327 insertions(+), 47 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida @ 2022-03-17 16:38 ` Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida ` (3 subsequent siblings) 4 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich As pointed out in a review of similar code, it is possible to avoid copying a couple of shared pointers in this function, by taking references instead. This commit also splits declarations to one per line and removes the unnecessary parentheses around the return expression. * src/abg-comp-filter.cc (crc_changed): Take references to avoid std::shared_ptr copying. Split declarations into one per line. Remove unnecessary return expression parentheses. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- src/abg-comp-filter.cc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 56251274..f90fdc78 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -230,11 +230,13 @@ static bool crc_changed(const function_or_var_decl_sptr& f, const function_or_var_decl_sptr& s) { - const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc(); - return (crc_f != 0 && crc_s != 0 && crc_f != crc_s); + const auto crc_f = symbol_f->get_crc(); + const auto crc_s = symbol_s->get_crc(); + return crc_f != 0 && crc_s != 0 && crc_f != crc_s; } /// Test if the current diff tree node carries a CRC change in either a -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v3 2/4] optional: minor improvements 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-03-17 16:38 ` Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida ` (2 subsequent siblings) 4 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich This change makes minor improvements to the optional class used with pre-C++17 compilers. - adds operator== and operator!= - adds various missing noexcept (but not constexpr) decorations - defines operator bool in terms of has_value Note that some constexpr decorations would require C++17 anyway. * include/abg-cxx-compat.h (optional): Add operator== and operator!=. Add noexcept decorations. Tweak operator bool. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h index 443905c7..5c5943d0 100644 --- a/include/abg-cxx-compat.h +++ b/include/abg-cxx-compat.h @@ -45,7 +45,7 @@ public: optional(const T& value) : has_value_(true), value_(value) {} bool - has_value() const + has_value() const noexcept { return has_value_; } @@ -67,19 +67,19 @@ public: } const T& - operator*() const + operator*() const& noexcept { return value_; } T& - operator*() + operator*() & noexcept { return value_; } const T* - operator->() const + operator->() const noexcept { return &value_; } T* - operator->() + operator->() noexcept { return &value_; } optional& @@ -90,9 +90,27 @@ public: return *this; } - explicit operator bool() const { return has_value_; } + explicit operator bool() const noexcept { return has_value(); } }; +template <typename T, typename U> +bool +operator==(const optional<T>& lhs, const optional<U>& rhs) +{ + if (!lhs.has_value() && !rhs.has_value()) + return true; + if (!lhs.has_value() || !rhs.has_value()) + return false; + return lhs.value() == rhs.value(); +} + +template <typename T, typename U> +bool +operator!=(const optional<T>& lhs, const optional<U>& rhs) +{ + return !(lhs == rhs); +} + #endif } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida @ 2022-03-17 16:38 ` Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 4 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich The CRC with value zero was used to mean "absent". This can be better modelled using optional. This commit makes this change and also tweaks reporting so that disappearing / appearing CRCs are noted. This should be essentially impossible unless CRCs are enabled / disabled altogether but would be very noteworthy otherwise. * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is now an optional defaulted to absent. (elf_symbol::create): Likewise. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_src): Now takes an optional uint64_t. * src/abg-comp-filter.cc (crc_changed): Simplify comparison. * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an optional uint64_t. (elf_symbol::priv::priv): Argument crc is now an optional uint64_t. (elf_symbol::elf_symbol): Likewise. (elf_symbol::create): Argument crc is now an optional uint64_t and defaults to absent. (textually_equals): Simplify comparison. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_crc): Now takes an optional uint64_t. * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same as other CRC values. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): Treat CRC 0 the same as other CRC values and also report changes to CRC presence. * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same as other CRC values. * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. * tests/data/test-abidiff/test-crc-report-0-1.txt: Report showing additional of CRCs. * tests/data/test-abidiff/test-crc-report-1-0.txt: Report showing removal of CRCs. * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed from tests/data/test-abidiff/test-crc-report.txt. * tests/test-abidiff.cc: Update test cases that no longer generate empty reports. * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 9 +++++---- src/abg-comp-filter.cc | 4 +--- src/abg-ir.cc | 19 +++++++++--------- src/abg-reader.cc | 8 ++------ src/abg-reporter-priv.cc | 20 ++++++++++++++----- src/abg-writer.cc | 6 +++--- tests/data/Makefile.am | 4 +++- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/test-abidiff.cc | 6 +++--- tests/test-symtab.cc | 8 ++++---- 12 files changed, 77 insertions(+), 39 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) diff --git a/include/abg-ir.h b/include/abg-ir.h index a2f4e1a7..b05a8c6f 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -21,6 +21,7 @@ #include <functional> #include <set> #include <unordered_map> +#include "abg-cxx-compat.h" #include "abg-fwd.h" #include "abg-hash.h" #include "abg-traverse.h" @@ -920,7 +921,7 @@ private: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -945,7 +946,7 @@ public: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); const environment* @@ -1017,11 +1018,11 @@ public: void set_is_in_ksymtab(bool is_in_ksymtab); - uint64_t + const abg_compat::optional<uint64_t>& get_crc() const; void - set_crc(uint64_t crc); + set_crc(const abg_compat::optional<uint64_t>& crc); bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index f90fdc78..31590284 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f, const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(); - const auto crc_s = symbol_s->get_crc(); - return crc_f != 0 && crc_s != 0 && crc_f != crc_s; + return symbol_f->get_crc() != symbol_s->get_crc(); } /// Test if the current diff tree node carries a CRC change in either a diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 0ef5e8b2..853b01ee 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1727,7 +1727,7 @@ struct elf_symbol::priv // STT_COMMON definition of that name that has the largest size. bool is_common_; bool is_in_ksymtab_; - uint64_t crc_; + abg_compat::optional<uint64_t> crc_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1744,7 +1744,7 @@ struct elf_symbol::priv is_defined_(false), is_common_(false), is_in_ksymtab_(false), - crc_(0), + crc_(), is_suppressed_(false) {} @@ -1759,7 +1759,7 @@ struct elf_symbol::priv const elf_symbol::version& ve, elf_symbol::visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : env_(e), index_(i), @@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : priv_(new priv(e, i, @@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, @@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && (l.get_crc() == 0 || r.get_crc() == 0 - || l.get_crc() == r.get_crc())); + && l.get_crc() == r.get_crc()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab) /// Getter of the 'crc' property. /// -/// @return the CRC (modversions) value for Linux Kernel symbols (if present) -uint64_t +/// @return the CRC (modversions) value for Linux Kernel symbols, if any +const abg_compat::optional<uint64_t>& elf_symbol::get_crc() const {return priv_->crc_;} @@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const /// /// @param crc the new CRC (modversions) value for Linux Kernel symbols void -elf_symbol::set_crc(uint64_t crc) +elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} /// Getter for the 'is-suppressed' property. diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 31885692..7a9ad1f9 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, is_default_version = true; } - uint64_t crc = 0; - if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) - crc = strtoull(CHAR_STR(s), NULL, 0); - elf_symbol::type type = elf_symbol::NOTYPE_TYPE; read_elf_symbol_type(node, type); @@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, e->set_is_suppressed(is_suppressed); - if (crc != 0) - e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) + e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 7012f5dc..f9dd928b 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << "\n"; } - if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0 - && symbol1->get_crc() != symbol2->get_crc()) + const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc(); + const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc(); + if (crc1 != crc2) { + const std::string none = "(none)"; out << indent << "CRC (modversions) changed from " - << std::showbase << std::hex - << symbol1->get_crc() << " to " << symbol2->get_crc() - << std::noshowbase << std::dec + << std::showbase << std::hex; + if (crc1.has_value()) + out << crc1.value(); + else + out << none; + out << " to "; + if (crc2.has_value()) + out << crc2.value(); + else + out << none; + out << std::noshowbase << std::dec << "\n"; } } diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 7802128d..0a1e7b66 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, if (sym->is_common_symbol()) o << " is-common='yes'"; - if (sym->get_crc() != 0) + if (sym->get_crc().has_value()) o << " crc='" - << std::hex << std::showbase << sym->get_crc() << "'" - << std::dec << std::noshowbase; + << std::hex << std::showbase << sym->get_crc().value() + << std::dec << std::noshowbase << "'"; o << "/>\n"; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index a7eb7ff0..90bd1934 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml \ test-abidiff/test-crc-0.xml \ test-abidiff/test-crc-1.xml \ test-abidiff/test-crc-2.xml \ -test-abidiff/test-crc-report.txt \ +test-abidiff/test-crc-report-0-1.txt \ +test-abidiff/test-crc-report-1-0.txt \ +test-abidiff/test-crc-report-1-2.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt new file mode 100644 index 00000000..0db42f68 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-0-1.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from (none) to 0xe52d5bcf + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from (none) to 0xee94d699 + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from (none) to 0x41336c46 + diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt new file mode 100644 index 00000000..e11f29c1 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-1-0.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from 0xe52d5bcf to (none) + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from 0xee94d699 to (none) + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from 0x41336c46 to (none) + diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt similarity index 100% rename from tests/data/test-abidiff/test-crc-report.txt rename to tests/data/test-abidiff/test-crc-report-1-2.txt diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index 32858ece..a02bbe3d 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -113,19 +113,19 @@ static InOutSpec specs[] = { "data/test-abidiff/test-crc-0.xml", "data/test-abidiff/test-crc-1.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-0-1.txt", "output/test-abidiff/test-crc-report-0-1.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-0.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-1-0.txt", "output/test-abidiff/test-crc-report-1-0.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-2.xml", - "data/test-abidiff/test-crc-report.txt", + "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, { diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc index 7d7a2df0..85303563 100644 --- a/tests/test-symtab.cc +++ b/tests/test-symtab.cc @@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]") { const std::string binary = base_path + "one_of_each.ko"; const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2); - CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0); - CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0); + CHECK(corpus->lookup_function_symbol("exported_function")->get_crc()); + CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc()); } } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v3 4/4] add Linux kernel symbol namespace support 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (2 preceding siblings ...) 2022-03-17 16:38 ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-03-17 16:38 ` Giuliano Procida 2022-03-21 12:53 ` Matthias Maennich 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 4 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-03-17 16:38 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Bug 28954 - add Linux Kernel symbol namespace support Each Linux kernel symbol can be exported to a specified named namespace or left in the global (nameless) namespace. One complexity is that the symbol values which identify a string in the __ksymtab_strings section must be interpretated differently for vmlinux and .ko loadable modules as the former has a fixed load address but the latter are relocatable. For vmlinux, the section base address needs to be subtracted to obtain a section-relative offset. The global namespace is explicitly represented as the empty string, at least when it comes to the value of __kstrtabns_FOO symbols, but the common interpretation is that such symbols lack an export namespace. I would rather not have to make use of "empty implies missing" in many places, so the code here represents namespace as optional<string> and only the symtab reader cares about empty strings in __ksymtab_strings. * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. (elf_symbol::create): Add ns argument. (elf_symbol::get_namespace): Declare new function. (elf_symbol::set_namespace): Declare new function. and set_namespace. * src/abg-comp-filter.cc (namespace_changed): Define new helper functions. (categorize_harmful_diff_node): Also call namespace_changed(). * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. (elf_symbol::priv::priv): Add namespace_ to initialisers. (elf_symbol::elf_symbol): Take new ns argument and pass it to priv constructor. (elf_symbol::create): Take new ns argument and pass it to elf_symbol constructor. (elf_symbol::get_namespace): Define new function. (elf_symbol::set_namespace): Define new function. * src/abg-reader.cc (build_elf_symbol): If namespace attribute is present, set symbol namespace. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If symbol namespaces differ, report this. * src/abg-symtab-reader.cc (symtab::load): Get ELF header to distinguish vmlinux from .ko. Try to get __ksymtab_strings metadata and data. Use these to look up __kstrtabns_FOO namespace entries. Set symbol namespace where found. * src/abg-writer.cc (write_elf_symbol): Emit namespace attribute, if symbol has a namespace. * tests/data/Makefile.am: Add new test files. * tests/data/test-abidiff/test-namespace-0.xml: New test file. * tests/data/test-abidiff/test-namespace-1.xml: Likewise * tests/data/test-abidiff/test-namespace-report.txt: Likewise. * tests/test-abidiff.cc: Add new test case. Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 8 +++ src/abg-comp-filter.cc | 40 ++++++++++- src/abg-ir.cc | 30 ++++++++- src/abg-reader.cc | 7 ++ src/abg-reporter-priv.cc | 18 +++++ src/abg-symtab-reader.cc | 66 +++++++++++++++++++ src/abg-writer.cc | 3 + tests/data/Makefile.am | 3 + tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 6 ++ 12 files changed, 225 insertions(+), 3 deletions(-) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt diff --git a/include/abg-ir.h b/include/abg-ir.h index b05a8c6f..7c558828 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -922,6 +922,7 @@ private: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -947,6 +948,7 @@ public: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); const environment* @@ -1024,6 +1026,12 @@ public: void set_crc(const abg_compat::optional<uint64_t>& crc); + const abg_compat::optional<std::string>& + get_namespace() const; + + void + set_namespace(const abg_compat::optional<std::string>& ns); + bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 31590284..22da5244 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -254,6 +254,43 @@ crc_changed(const diff* diff) return false; } +/// Test if there was a function or variable namespace change. +/// +/// @param f the first function or variable to consider. +/// +/// @param s the second function or variable to consider. +/// +/// @return true if the test is positive, false otherwise. +template <typename function_or_var_decl_sptr> +static bool +namespace_changed(const function_or_var_decl_sptr& f, + const function_or_var_decl_sptr& s) +{ + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); + if (!symbol_f || !symbol_s) + return false; + return symbol_f->get_namespace() != symbol_s->get_namespace(); +} + +/// Test if the current diff tree node carries a namespace change in +/// either a function or a variable. +/// +/// @param diff the diff tree node to consider. +/// +/// @return true if the test is positive, false otherwise. +static bool +namespace_changed(const diff* diff) +{ + if (const function_decl_diff* d = + dynamic_cast<const function_decl_diff*>(diff)) + return namespace_changed(d->first_function_decl(), + d->second_function_decl()); + if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) + return namespace_changed(d->first_var(), d->second_var()); + return false; +} + /// Test if there was a function name change, but there there was no /// change in name of the underlying symbol. IOW, if the name of a /// function changed, but the symbol of the new function is equal to @@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre) || non_static_data_member_added_or_removed(d) || base_classes_added_or_removed(d) || has_harmful_enum_change(d) - || crc_changed(d))) + || crc_changed(d) + || namespace_changed(d))) category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; if (has_virtual_mem_fn_change(d)) diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 853b01ee..f90be2e4 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1728,6 +1728,7 @@ struct elf_symbol::priv bool is_common_; bool is_in_ksymtab_; abg_compat::optional<uint64_t> crc_; + abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1745,6 +1746,7 @@ struct elf_symbol::priv is_common_(false), is_in_ksymtab_(false), crc_(), + namespace_(), is_suppressed_(false) {} @@ -1760,6 +1762,7 @@ struct elf_symbol::priv elf_symbol::visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), index_(i), @@ -1773,6 +1776,7 @@ struct elf_symbol::priv is_common_(c), is_in_ksymtab_(is_in_ksymtab), crc_(crc), + namespace_(ns), is_suppressed_(is_suppressed) { if (!is_common_) @@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() /// @param vi the visibility of the symbol. /// /// @param crc the CRC (modversions) value of Linux Kernel symbols +/// +/// @param ns the namespace of Linux Kernel symbols, if any elf_symbol::elf_symbol(const environment* e, size_t i, size_t s, @@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, i, @@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, vi, is_in_ksymtab, crc, + ns, is_suppressed)) {} @@ -1886,6 +1894,8 @@ elf_symbol::create() /// /// @param crc the CRC (modversions) value of Linux Kernel symbols /// +/// @param ns the namespace of Linux Kernel symbols, if any +/// /// @return a (smart) pointer to a newly created instance of @ref /// elf_symbol. elf_symbol_sptr @@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, - is_in_ksymtab, crc, is_suppressed)); + is_in_ksymtab, crc, ns, is_suppressed)); sym->priv_->main_symbol_ = sym; return sym; } @@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && l.get_crc() == r.get_crc()); + && l.get_crc() == r.get_crc() + && l.get_namespace() == r.get_namespace()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2146,6 +2158,20 @@ void elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} +/// Getter of the 'namespace' property. +/// +/// @return the namespace for Linux Kernel symbols, if any +const abg_compat::optional<std::string>& +elf_symbol::get_namespace() const +{return priv_->namespace_;} + +/// Setter of the 'namespace' property. +/// +/// @param ns the new namespace for Linux Kernel symbols, if any +void +elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) +{priv_->namespace_ = ns;} + /// Getter for the 'is-suppressed' property. /// /// @return true iff the current symbol has been suppressed by a diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 7a9ad1f9..f9e420f1 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) + { + std::string ns; + xml::xml_char_sptr_to_string(s, ns); + e->set_namespace(ns); + } + return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index f9dd928b..9ad733f4 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, out << std::noshowbase << std::dec << "\n"; } + + const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); + const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); + if (ns1 != ns2) + { + const std::string none = "(none)"; + out << indent << "namespace changed from "; + if (ns1.has_value()) + out << "'" << ns1.value() << "'"; + else + out << none; + out << " to "; + if (ns2.has_value()) + out << "'" << ns2.value() << "'"; + else + out << none; + out << "\n"; + } } /// For a given symbol, emit a string made of its name and version. diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc index b42ce87d..123ef84a 100644 --- a/src/abg-symtab-reader.cc +++ b/src/abg-symtab-reader.cc @@ -204,6 +204,13 @@ symtab::load_(Elf* elf_handle, ir::environment* env, symbol_predicate is_suppressed) { + GElf_Ehdr ehdr_mem; + GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem); + if (!header) + { + std::cerr << "Could not load get ELF header: Skipping symtab load.\n"; + return false; + } Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle); if (!symtab_section) @@ -232,9 +239,34 @@ symtab::load_(Elf* elf_handle, return false; } + // The __kstrtab_strings sections is basically an ELF strtab but does not + // support elf_strptr lookups. A single call to elf_getdata gives a handle to + // section data. + // + // The value of a __kstrtabns_FOO (or other similar) symbol is an address + // within the __kstrtab_strings section. To look up the string value, we need + // to translate from vmlinux load address to section offset by subtracting the + // base address of the section. This adjustment is not needed for loadable + // modules which are relocatable. + Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); + size_t strings_offset = 0; + const char* strings_data = nullptr; + size_t strings_size = 0; + if (strings_section) + { + GElf_Shdr strings_sheader; + gelf_getshdr(strings_section, &strings_sheader); + strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr; + Elf_Data* data = elf_getdata(strings_section, nullptr); + ABG_ASSERT(data->d_off == 0); + strings_data = reinterpret_cast<const char *>(data->d_buf); + strings_size = data->d_size; + } + const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); std::unordered_set<std::string> exported_kernel_symbols; std::unordered_map<std::string, uint64_t> crc_values; + std::unordered_map<std::string, std::string> namespaces; const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); @@ -288,6 +320,29 @@ symtab::load_(Elf* elf_handle, ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); continue; } + if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) + { + // This symbol lives in the __ksymtab_strings section but st_value may + // be a vmlinux load address so we need to subtract the offset before + // looking it up in that section. + const size_t value = sym->st_value; + const size_t offset = value - strings_offset; + // check offset + ABG_ASSERT(offset < strings_size); + // find the terminating NULL + const char* first = strings_data + offset; + const char* last = strings_data + strings_size; + const char* limit = std::find(first, last, 0); + // check NULL found + ABG_ASSERT(limit < last); + // interpret the empty namespace name as no namespace name + if (first < limit) + { + const std::string ns(first, limit - first); + ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); + } + continue; + } // filter out uninteresting entries and only keep functions/variables for // now. The rest might be interesting in the future though. @@ -402,6 +457,17 @@ symtab::load_(Elf* elf_handle, symbol->set_crc(crc_entry.second); } + // Now add the namespaces + for (const auto& namespace_entry : namespaces) + { + const auto r = name_symbol_map_.find(namespace_entry.first); + if (r == name_symbol_map_.end()) + continue; + + for (const auto& symbol : r->second) + symbol->set_namespace(namespace_entry.second); + } + // sort the symbols for deterministic output std::sort(symbols_.begin(), symbols_.end(), symbol_sort); diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 0a1e7b66..03fb658b 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, << std::hex << std::showbase << sym->get_crc().value() << std::dec << std::noshowbase << "'"; + if (sym->get_namespace().has_value()) + o << " namespace='" << sym->get_namespace().value() << "'"; + o << "/>\n"; return true; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index 90bd1934..7388697b 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \ test-abidiff/test-crc-report-0-1.txt \ test-abidiff/test-crc-report-1-0.txt \ test-abidiff/test-crc-report-1-2.txt \ +test-abidiff/test-namespace-0.xml \ +test-abidiff/test-namespace-1.xml \ +test-abidiff/test-namespace-report.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml new file mode 100644 index 00000000..5a9f5cd2 --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-0.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml new file mode 100644 index 00000000..9814844b --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-1.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt new file mode 100644 index 00000000..d2c421ed --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-report.txt @@ -0,0 +1,17 @@ +Functions changes summary: 0 Removed, 0 Changed, 0 Added function +Variables changes summary: 0 Removed, 4 Changed, 0 Added variables + +4 Changed variables: + + [C] 'int v1' was changed: + namespace changed from (none) to 'this' + + [C] 'int v2' was changed: + namespace changed from 'this' to 'that' + + [C] 'int v3' was changed: + namespace changed from 'that' to '' + + [C] 'int v4' was changed: + namespace changed from '' to (none) + diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index a02bbe3d..93e44d6a 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -128,6 +128,12 @@ static InOutSpec specs[] = "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, + { + "data/test-abidiff/test-namespace-0.xml", + "data/test-abidiff/test-namespace-1.xml", + "data/test-abidiff/test-namespace-report.txt", + "output/test-abidiff/test-namespace-report-0-1.txt" + }, { "data/test-abidiff/test-PR27616-v0.xml", "data/test-abidiff/test-PR27616-v1.xml", -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v3 4/4] add Linux kernel symbol namespace support 2022-03-17 16:38 ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-21 12:53 ` Matthias Maennich 2022-03-21 15:52 ` Giuliano Procida 0 siblings, 1 reply; 37+ messages in thread From: Matthias Maennich @ 2022-03-21 12:53 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, dodji, kernel-team On Thu, Mar 17, 2022 at 04:38:58PM +0000, Giuliano Procida wrote: >Bug 28954 - add Linux Kernel symbol namespace support > >Each Linux kernel symbol can be exported to a specified named >namespace or left in the global (nameless) namespace. > >One complexity is that the symbol values which identify a string in >the __ksymtab_strings section must be interpretated differently for >vmlinux and .ko loadable modules as the former has a fixed load >address but the latter are relocatable. For vmlinux, the section base >address needs to be subtracted to obtain a section-relative offset. > >The global namespace is explicitly represented as the empty string, at >least when it comes to the value of __kstrtabns_FOO symbols, but the >common interpretation is that such symbols lack an export namespace. > >I would rather not have to make use of "empty implies missing" in many >places, so the code here represents namespace as optional<string> and >only the symtab reader cares about empty strings in __ksymtab_strings. > > * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. > (elf_symbol::create): Add ns argument. > (elf_symbol::get_namespace): Declare new function. > (elf_symbol::set_namespace): Declare new function. > and set_namespace. > * src/abg-comp-filter.cc (namespace_changed): Define new > helper functions. > (categorize_harmful_diff_node): Also call namespace_changed(). > * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. > (elf_symbol::priv::priv): Add namespace_ to initialisers. > (elf_symbol::elf_symbol): Take new ns argument and pass it to > priv constructor. > (elf_symbol::create): Take new ns argument and pass it to > elf_symbol constructor. > (elf_symbol::get_namespace): Define new function. > (elf_symbol::set_namespace): Define new function. > * src/abg-reader.cc (build_elf_symbol): If namespace > attribute is present, set symbol namespace. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If > symbol namespaces differ, report this. > * src/abg-symtab-reader.cc (symtab::load): Get ELF header to > distinguish vmlinux from .ko. Try to get __ksymtab_strings > metadata and data. Use these to look up __kstrtabns_FOO > namespace entries. Set symbol namespace where found. > * src/abg-writer.cc (write_elf_symbol): Emit namespace > attribute, if symbol has a namespace. > * tests/data/Makefile.am: Add new test files. > * tests/data/test-abidiff/test-namespace-0.xml: New test file. > * tests/data/test-abidiff/test-namespace-1.xml: Likewise > * tests/data/test-abidiff/test-namespace-report.txt: Likewise. > * tests/test-abidiff.cc: Add new test case. > >Signed-off-by: Giuliano Procida <gprocida@google.com> >--- > include/abg-ir.h | 8 +++ > src/abg-comp-filter.cc | 40 ++++++++++- > src/abg-ir.cc | 30 ++++++++- > src/abg-reader.cc | 7 ++ > src/abg-reporter-priv.cc | 18 +++++ > src/abg-symtab-reader.cc | 66 +++++++++++++++++++ > src/abg-writer.cc | 3 + > tests/data/Makefile.am | 3 + > tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ > tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ > .../test-abidiff/test-namespace-report.txt | 17 +++++ > tests/test-abidiff.cc | 6 ++ > 12 files changed, 225 insertions(+), 3 deletions(-) > create mode 100644 tests/data/test-abidiff/test-namespace-0.xml > create mode 100644 tests/data/test-abidiff/test-namespace-1.xml > create mode 100644 tests/data/test-abidiff/test-namespace-report.txt > >diff --git a/include/abg-ir.h b/include/abg-ir.h >index b05a8c6f..7c558828 100644 >--- a/include/abg-ir.h >+++ b/include/abg-ir.h >@@ -922,6 +922,7 @@ private: > visibility vi, > bool is_in_ksymtab = false, > const abg_compat::optional<uint64_t>& crc = {}, >+ const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > > elf_symbol(const elf_symbol&); >@@ -947,6 +948,7 @@ public: > visibility vi, > bool is_in_ksymtab = false, > const abg_compat::optional<uint64_t>& crc = {}, >+ const abg_compat::optional<std::string>& ns = {}, > bool is_suppressed = false); > > const environment* >@@ -1024,6 +1026,12 @@ public: > void > set_crc(const abg_compat::optional<uint64_t>& crc); > >+ const abg_compat::optional<std::string>& >+ get_namespace() const; >+ >+ void >+ set_namespace(const abg_compat::optional<std::string>& ns); >+ > bool > is_suppressed() const; > >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc >index 31590284..22da5244 100644 >--- a/src/abg-comp-filter.cc >+++ b/src/abg-comp-filter.cc >@@ -254,6 +254,43 @@ crc_changed(const diff* diff) > return false; > } > >+/// Test if there was a function or variable namespace change. >+/// >+/// @param f the first function or variable to consider. >+/// >+/// @param s the second function or variable to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+template <typename function_or_var_decl_sptr> >+static bool >+namespace_changed(const function_or_var_decl_sptr& f, >+ const function_or_var_decl_sptr& s) >+{ >+ const auto& symbol_f = f->get_symbol(); >+ const auto& symbol_s = s->get_symbol(); >+ if (!symbol_f || !symbol_s) >+ return false; >+ return symbol_f->get_namespace() != symbol_s->get_namespace(); >+} >+ >+/// Test if the current diff tree node carries a namespace change in >+/// either a function or a variable. >+/// >+/// @param diff the diff tree node to consider. >+/// >+/// @return true if the test is positive, false otherwise. >+static bool >+namespace_changed(const diff* diff) >+{ >+ if (const function_decl_diff* d = >+ dynamic_cast<const function_decl_diff*>(diff)) >+ return namespace_changed(d->first_function_decl(), >+ d->second_function_decl()); >+ if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) >+ return namespace_changed(d->first_var(), d->second_var()); >+ return false; >+} >+ > /// Test if there was a function name change, but there there was no > /// change in name of the underlying symbol. IOW, if the name of a > /// function changed, but the symbol of the new function is equal to >@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre) > || non_static_data_member_added_or_removed(d) > || base_classes_added_or_removed(d) > || has_harmful_enum_change(d) >- || crc_changed(d))) >+ || crc_changed(d) >+ || namespace_changed(d))) > category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; > > if (has_virtual_mem_fn_change(d)) >diff --git a/src/abg-ir.cc b/src/abg-ir.cc >index 853b01ee..f90be2e4 100644 >--- a/src/abg-ir.cc >+++ b/src/abg-ir.cc >@@ -1728,6 +1728,7 @@ struct elf_symbol::priv > bool is_common_; > bool is_in_ksymtab_; > abg_compat::optional<uint64_t> crc_; >+ abg_compat::optional<std::string> namespace_; > bool is_suppressed_; > elf_symbol_wptr main_symbol_; > elf_symbol_wptr next_alias_; >@@ -1745,6 +1746,7 @@ struct elf_symbol::priv > is_common_(false), > is_in_ksymtab_(false), > crc_(), >+ namespace_(), > is_suppressed_(false) > {} > >@@ -1760,6 +1762,7 @@ struct elf_symbol::priv > elf_symbol::visibility vi, > bool is_in_ksymtab, > const abg_compat::optional<uint64_t>& crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : env_(e), > index_(i), >@@ -1773,6 +1776,7 @@ struct elf_symbol::priv > is_common_(c), > is_in_ksymtab_(is_in_ksymtab), > crc_(crc), >+ namespace_(ns), > is_suppressed_(is_suppressed) > { > if (!is_common_) >@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() > /// @param vi the visibility of the symbol. > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols >+/// >+/// @param ns the namespace of Linux Kernel symbols, if any > elf_symbol::elf_symbol(const environment* e, > size_t i, > size_t s, >@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, > visibility vi, > bool is_in_ksymtab, > const abg_compat::optional<uint64_t>& crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > : priv_(new priv(e, > i, >@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, > vi, > is_in_ksymtab, > crc, >+ ns, > is_suppressed)) > {} > >@@ -1886,6 +1894,8 @@ elf_symbol::create() > /// > /// @param crc the CRC (modversions) value of Linux Kernel symbols > /// >+/// @param ns the namespace of Linux Kernel symbols, if any >+/// > /// @return a (smart) pointer to a newly created instance of @ref > /// elf_symbol. > elf_symbol_sptr >@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, > visibility vi, > bool is_in_ksymtab, > const abg_compat::optional<uint64_t>& crc, >+ const abg_compat::optional<std::string>& ns, > bool is_suppressed) > { > elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, >- is_in_ksymtab, crc, is_suppressed)); >+ is_in_ksymtab, crc, ns, is_suppressed)); > sym->priv_->main_symbol_ = sym; > return sym; > } >@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l, > && l.is_defined() == r.is_defined() > && l.is_common_symbol() == r.is_common_symbol() > && l.get_version() == r.get_version() >- && l.get_crc() == r.get_crc()); >+ && l.get_crc() == r.get_crc() >+ && l.get_namespace() == r.get_namespace()); > > if (equals && l.is_variable()) > // These are variable symbols. Let's compare their symbol size. >@@ -2146,6 +2158,20 @@ void > elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) > {priv_->crc_ = crc;} > >+/// Getter of the 'namespace' property. >+/// >+/// @return the namespace for Linux Kernel symbols, if any >+const abg_compat::optional<std::string>& >+elf_symbol::get_namespace() const >+{return priv_->namespace_;} >+ >+/// Setter of the 'namespace' property. >+/// >+/// @param ns the new namespace for Linux Kernel symbols, if any >+void >+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) >+{priv_->namespace_ = ns;} >+ > /// Getter for the 'is-suppressed' property. > /// > /// @return true iff the current symbol has been suppressed by a >diff --git a/src/abg-reader.cc b/src/abg-reader.cc >index 7a9ad1f9..f9e420f1 100644 >--- a/src/abg-reader.cc >+++ b/src/abg-reader.cc >@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) > e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); > >+ if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) >+ { >+ std::string ns; >+ xml::xml_char_sptr_to_string(s, ns); >+ e->set_namespace(ns); >+ } >+ > return e; > } > >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc >index f9dd928b..9ad733f4 100644 >--- a/src/abg-reporter-priv.cc >+++ b/src/abg-reporter-priv.cc >@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, > out << std::noshowbase << std::dec > << "\n"; > } >+ >+ const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); >+ const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); >+ if (ns1 != ns2) >+ { >+ const std::string none = "(none)"; >+ out << indent << "namespace changed from "; >+ if (ns1.has_value()) >+ out << "'" << ns1.value() << "'"; >+ else >+ out << none; >+ out << " to "; >+ if (ns2.has_value()) >+ out << "'" << ns2.value() << "'"; >+ else >+ out << none; >+ out << "\n"; >+ } > } > > /// For a given symbol, emit a string made of its name and version. >diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc >index b42ce87d..123ef84a 100644 >--- a/src/abg-symtab-reader.cc >+++ b/src/abg-symtab-reader.cc >@@ -204,6 +204,13 @@ symtab::load_(Elf* elf_handle, > ir::environment* env, > symbol_predicate is_suppressed) > { >+ GElf_Ehdr ehdr_mem; >+ GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem); >+ if (!header) >+ { >+ std::cerr << "Could not load get ELF header: Skipping symtab load.\n"; nit: "Could not load ELF header: ..." >+ return false; >+ } > > Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle); > if (!symtab_section) >@@ -232,9 +239,34 @@ symtab::load_(Elf* elf_handle, > return false; > } > >+ // The __kstrtab_strings sections is basically an ELF strtab but does not >+ // support elf_strptr lookups. A single call to elf_getdata gives a handle to >+ // section data. raw section data >+ // >+ // The value of a __kstrtabns_FOO (or other similar) symbol is an address >+ // within the __kstrtab_strings section. To look up the string value, we need >+ // to translate from vmlinux load address to section offset by subtracting the >+ // base address of the section. This adjustment is not needed for loadable >+ // modules which are relocatable. ... identifiable by ELF type ET_REL. >+ Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); >+ size_t strings_offset = 0; >+ const char* strings_data = nullptr; >+ size_t strings_size = 0; >+ if (strings_section) >+ { >+ GElf_Shdr strings_sheader; >+ gelf_getshdr(strings_section, &strings_sheader); >+ strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr; >+ Elf_Data* data = elf_getdata(strings_section, nullptr); >+ ABG_ASSERT(data->d_off == 0); >+ strings_data = reinterpret_cast<const char *>(data->d_buf); >+ strings_size = data->d_size; Since you later assume strings_data and strings_size being valid based on the existence of strings_section, perhaps assert that the extraction worked? ABG_ASSERT(strings_data); ABG_ASSERT(strings_size > 0); // otherwise, why would there be a section? >+ } >+ > const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); > std::unordered_set<std::string> exported_kernel_symbols; > std::unordered_map<std::string, uint64_t> crc_values; >+ std::unordered_map<std::string, std::string> namespaces; > > const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); > const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); >@@ -288,6 +320,29 @@ symtab::load_(Elf* elf_handle, > ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); > continue; > } >+ if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) >+ { >+ // This symbol lives in the __ksymtab_strings section but st_value may >+ // be a vmlinux load address so we need to subtract the offset before >+ // looking it up in that section. >+ const size_t value = sym->st_value; odd indentation >+ const size_t offset = value - strings_offset; >+ // check offset >+ ABG_ASSERT(offset < strings_size); >+ // find the terminating NULL >+ const char* first = strings_data + offset; >+ const char* last = strings_data + strings_size; >+ const char* limit = std::find(first, last, 0); >+ // check NULL found >+ ABG_ASSERT(limit < last); >+ // interpret the empty namespace name as no namespace name >+ if (first < limit) >+ { >+ const std::string ns(first, limit - first); >+ ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); ABG_ASSERT(namespaces.emplace(name.substr(12), std::move(ns)).second); With the few nits addressed, feel free to add Reviewed-by: Matthias Maennich <maennich@google.com> Cheers, Matthias >+ } >+ continue; >+ } > > // filter out uninteresting entries and only keep functions/variables for > // now. The rest might be interesting in the future though. >@@ -402,6 +457,17 @@ symtab::load_(Elf* elf_handle, > symbol->set_crc(crc_entry.second); > } > >+ // Now add the namespaces >+ for (const auto& namespace_entry : namespaces) >+ { >+ const auto r = name_symbol_map_.find(namespace_entry.first); >+ if (r == name_symbol_map_.end()) >+ continue; >+ >+ for (const auto& symbol : r->second) >+ symbol->set_namespace(namespace_entry.second); >+ } >+ > // sort the symbols for deterministic output > std::sort(symbols_.begin(), symbols_.end(), symbol_sort); > >diff --git a/src/abg-writer.cc b/src/abg-writer.cc >index 0a1e7b66..03fb658b 100644 >--- a/src/abg-writer.cc >+++ b/src/abg-writer.cc >@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, > << std::hex << std::showbase << sym->get_crc().value() > << std::dec << std::noshowbase << "'"; > >+ if (sym->get_namespace().has_value()) >+ o << " namespace='" << sym->get_namespace().value() << "'"; >+ > o << "/>\n"; > > return true; >diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am >index 90bd1934..7388697b 100644 >--- a/tests/data/Makefile.am >+++ b/tests/data/Makefile.am >@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \ > test-abidiff/test-crc-report-0-1.txt \ > test-abidiff/test-crc-report-1-0.txt \ > test-abidiff/test-crc-report-1-2.txt \ >+test-abidiff/test-namespace-0.xml \ >+test-abidiff/test-namespace-1.xml \ >+test-abidiff/test-namespace-report.txt \ > test-abidiff/test-PR27985-report.txt \ > test-abidiff/test-PR27985-v0.c \ > test-abidiff/test-PR27985-v0.o \ >diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml >new file mode 100644 >index 00000000..5a9f5cd2 >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-0.xml >@@ -0,0 +1,15 @@ >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> >+ <elf-variable-symbols> >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> >+ </elf-variable-symbols> >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> >+ </abi-instr> >+</abi-corpus> >diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml >new file mode 100644 >index 00000000..9814844b >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-1.xml >@@ -0,0 +1,15 @@ >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> >+ <elf-variable-symbols> >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> >+ </elf-variable-symbols> >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> >+ </abi-instr> >+</abi-corpus> >diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt >new file mode 100644 >index 00000000..d2c421ed >--- /dev/null >+++ b/tests/data/test-abidiff/test-namespace-report.txt >@@ -0,0 +1,17 @@ >+Functions changes summary: 0 Removed, 0 Changed, 0 Added function >+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables >+ >+4 Changed variables: >+ >+ [C] 'int v1' was changed: >+ namespace changed from (none) to 'this' >+ >+ [C] 'int v2' was changed: >+ namespace changed from 'this' to 'that' >+ >+ [C] 'int v3' was changed: >+ namespace changed from 'that' to '' >+ >+ [C] 'int v4' was changed: >+ namespace changed from '' to (none) >+ >diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc >index a02bbe3d..93e44d6a 100644 >--- a/tests/test-abidiff.cc >+++ b/tests/test-abidiff.cc >@@ -128,6 +128,12 @@ static InOutSpec specs[] = > "data/test-abidiff/test-crc-report-1-2.txt", > "output/test-abidiff/test-crc-report-1-2.txt" > }, >+ { >+ "data/test-abidiff/test-namespace-0.xml", >+ "data/test-abidiff/test-namespace-1.xml", >+ "data/test-abidiff/test-namespace-report.txt", >+ "output/test-abidiff/test-namespace-report-0-1.txt" >+ }, > { > "data/test-abidiff/test-PR27616-v0.xml", > "data/test-abidiff/test-PR27616-v1.xml", >-- >2.35.1.894.gb6a874cedc-goog > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v3 4/4] add Linux kernel symbol namespace support 2022-03-21 12:53 ` Matthias Maennich @ 2022-03-21 15:52 ` Giuliano Procida 0 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 15:52 UTC (permalink / raw) To: Matthias Maennich; +Cc: libabigail, dodji, kernel-team Hi. On Mon, 21 Mar 2022 at 12:53, Matthias Maennich <maennich@google.com> wrote: > > On Thu, Mar 17, 2022 at 04:38:58PM +0000, Giuliano Procida wrote: > >Bug 28954 - add Linux Kernel symbol namespace support > > > >Each Linux kernel symbol can be exported to a specified named > >namespace or left in the global (nameless) namespace. > > > >One complexity is that the symbol values which identify a string in > >the __ksymtab_strings section must be interpretated differently for > >vmlinux and .ko loadable modules as the former has a fixed load > >address but the latter are relocatable. For vmlinux, the section base > >address needs to be subtracted to obtain a section-relative offset. > > > >The global namespace is explicitly represented as the empty string, at > >least when it comes to the value of __kstrtabns_FOO symbols, but the > >common interpretation is that such symbols lack an export namespace. > > > >I would rather not have to make use of "empty implies missing" in many > >places, so the code here represents namespace as optional<string> and > >only the symtab reader cares about empty strings in __ksymtab_strings. > > > > * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. > > (elf_symbol::create): Add ns argument. > > (elf_symbol::get_namespace): Declare new function. > > (elf_symbol::set_namespace): Declare new function. > > and set_namespace. > > * src/abg-comp-filter.cc (namespace_changed): Define new > > helper functions. > > (categorize_harmful_diff_node): Also call namespace_changed(). > > * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. > > (elf_symbol::priv::priv): Add namespace_ to initialisers. > > (elf_symbol::elf_symbol): Take new ns argument and pass it to > > priv constructor. > > (elf_symbol::create): Take new ns argument and pass it to > > elf_symbol constructor. > > (elf_symbol::get_namespace): Define new function. > > (elf_symbol::set_namespace): Define new function. > > * src/abg-reader.cc (build_elf_symbol): If namespace > > attribute is present, set symbol namespace. > > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If > > symbol namespaces differ, report this. > > * src/abg-symtab-reader.cc (symtab::load): Get ELF header to > > distinguish vmlinux from .ko. Try to get __ksymtab_strings > > metadata and data. Use these to look up __kstrtabns_FOO > > namespace entries. Set symbol namespace where found. > > * src/abg-writer.cc (write_elf_symbol): Emit namespace > > attribute, if symbol has a namespace. > > * tests/data/Makefile.am: Add new test files. > > * tests/data/test-abidiff/test-namespace-0.xml: New test file. > > * tests/data/test-abidiff/test-namespace-1.xml: Likewise > > * tests/data/test-abidiff/test-namespace-report.txt: Likewise. > > * tests/test-abidiff.cc: Add new test case. > > > >Signed-off-by: Giuliano Procida <gprocida@google.com> > >--- > > include/abg-ir.h | 8 +++ > > src/abg-comp-filter.cc | 40 ++++++++++- > > src/abg-ir.cc | 30 ++++++++- > > src/abg-reader.cc | 7 ++ > > src/abg-reporter-priv.cc | 18 +++++ > > src/abg-symtab-reader.cc | 66 +++++++++++++++++++ > > src/abg-writer.cc | 3 + > > tests/data/Makefile.am | 3 + > > tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ > > tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ > > .../test-abidiff/test-namespace-report.txt | 17 +++++ > > tests/test-abidiff.cc | 6 ++ > > 12 files changed, 225 insertions(+), 3 deletions(-) > > create mode 100644 tests/data/test-abidiff/test-namespace-0.xml > > create mode 100644 tests/data/test-abidiff/test-namespace-1.xml > > create mode 100644 tests/data/test-abidiff/test-namespace-report.txt > > > >diff --git a/include/abg-ir.h b/include/abg-ir.h > >index b05a8c6f..7c558828 100644 > >--- a/include/abg-ir.h > >+++ b/include/abg-ir.h > >@@ -922,6 +922,7 @@ private: > > visibility vi, > > bool is_in_ksymtab = false, > > const abg_compat::optional<uint64_t>& crc = {}, > >+ const abg_compat::optional<std::string>& ns = {}, > > bool is_suppressed = false); > > > > elf_symbol(const elf_symbol&); > >@@ -947,6 +948,7 @@ public: > > visibility vi, > > bool is_in_ksymtab = false, > > const abg_compat::optional<uint64_t>& crc = {}, > >+ const abg_compat::optional<std::string>& ns = {}, > > bool is_suppressed = false); > > > > const environment* > >@@ -1024,6 +1026,12 @@ public: > > void > > set_crc(const abg_compat::optional<uint64_t>& crc); > > > >+ const abg_compat::optional<std::string>& > >+ get_namespace() const; > >+ > >+ void > >+ set_namespace(const abg_compat::optional<std::string>& ns); > >+ > > bool > > is_suppressed() const; > > > >diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc > >index 31590284..22da5244 100644 > >--- a/src/abg-comp-filter.cc > >+++ b/src/abg-comp-filter.cc > >@@ -254,6 +254,43 @@ crc_changed(const diff* diff) > > return false; > > } > > > >+/// Test if there was a function or variable namespace change. > >+/// > >+/// @param f the first function or variable to consider. > >+/// > >+/// @param s the second function or variable to consider. > >+/// > >+/// @return true if the test is positive, false otherwise. > >+template <typename function_or_var_decl_sptr> > >+static bool > >+namespace_changed(const function_or_var_decl_sptr& f, > >+ const function_or_var_decl_sptr& s) > >+{ > >+ const auto& symbol_f = f->get_symbol(); > >+ const auto& symbol_s = s->get_symbol(); > >+ if (!symbol_f || !symbol_s) > >+ return false; > >+ return symbol_f->get_namespace() != symbol_s->get_namespace(); > >+} > >+ > >+/// Test if the current diff tree node carries a namespace change in > >+/// either a function or a variable. > >+/// > >+/// @param diff the diff tree node to consider. > >+/// > >+/// @return true if the test is positive, false otherwise. > >+static bool > >+namespace_changed(const diff* diff) > >+{ > >+ if (const function_decl_diff* d = > >+ dynamic_cast<const function_decl_diff*>(diff)) > >+ return namespace_changed(d->first_function_decl(), > >+ d->second_function_decl()); > >+ if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) > >+ return namespace_changed(d->first_var(), d->second_var()); > >+ return false; > >+} > >+ > > /// Test if there was a function name change, but there there was no > > /// change in name of the underlying symbol. IOW, if the name of a > > /// function changed, but the symbol of the new function is equal to > >@@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre) > > || non_static_data_member_added_or_removed(d) > > || base_classes_added_or_removed(d) > > || has_harmful_enum_change(d) > >- || crc_changed(d))) > >+ || crc_changed(d) > >+ || namespace_changed(d))) > > category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; > > > > if (has_virtual_mem_fn_change(d)) > >diff --git a/src/abg-ir.cc b/src/abg-ir.cc > >index 853b01ee..f90be2e4 100644 > >--- a/src/abg-ir.cc > >+++ b/src/abg-ir.cc > >@@ -1728,6 +1728,7 @@ struct elf_symbol::priv > > bool is_common_; > > bool is_in_ksymtab_; > > abg_compat::optional<uint64_t> crc_; > >+ abg_compat::optional<std::string> namespace_; > > bool is_suppressed_; > > elf_symbol_wptr main_symbol_; > > elf_symbol_wptr next_alias_; > >@@ -1745,6 +1746,7 @@ struct elf_symbol::priv > > is_common_(false), > > is_in_ksymtab_(false), > > crc_(), > >+ namespace_(), > > is_suppressed_(false) > > {} > > > >@@ -1760,6 +1762,7 @@ struct elf_symbol::priv > > elf_symbol::visibility vi, > > bool is_in_ksymtab, > > const abg_compat::optional<uint64_t>& crc, > >+ const abg_compat::optional<std::string>& ns, > > bool is_suppressed) > > : env_(e), > > index_(i), > >@@ -1773,6 +1776,7 @@ struct elf_symbol::priv > > is_common_(c), > > is_in_ksymtab_(is_in_ksymtab), > > crc_(crc), > >+ namespace_(ns), > > is_suppressed_(is_suppressed) > > { > > if (!is_common_) > >@@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() > > /// @param vi the visibility of the symbol. > > /// > > /// @param crc the CRC (modversions) value of Linux Kernel symbols > >+/// > >+/// @param ns the namespace of Linux Kernel symbols, if any > > elf_symbol::elf_symbol(const environment* e, > > size_t i, > > size_t s, > >@@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, > > visibility vi, > > bool is_in_ksymtab, > > const abg_compat::optional<uint64_t>& crc, > >+ const abg_compat::optional<std::string>& ns, > > bool is_suppressed) > > : priv_(new priv(e, > > i, > >@@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, > > vi, > > is_in_ksymtab, > > crc, > >+ ns, > > is_suppressed)) > > {} > > > >@@ -1886,6 +1894,8 @@ elf_symbol::create() > > /// > > /// @param crc the CRC (modversions) value of Linux Kernel symbols > > /// > >+/// @param ns the namespace of Linux Kernel symbols, if any > >+/// > > /// @return a (smart) pointer to a newly created instance of @ref > > /// elf_symbol. > > elf_symbol_sptr > >@@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, > > visibility vi, > > bool is_in_ksymtab, > > const abg_compat::optional<uint64_t>& crc, > >+ const abg_compat::optional<std::string>& ns, > > bool is_suppressed) > > { > > elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, > >- is_in_ksymtab, crc, is_suppressed)); > >+ is_in_ksymtab, crc, ns, is_suppressed)); > > sym->priv_->main_symbol_ = sym; > > return sym; > > } > >@@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l, > > && l.is_defined() == r.is_defined() > > && l.is_common_symbol() == r.is_common_symbol() > > && l.get_version() == r.get_version() > >- && l.get_crc() == r.get_crc()); > >+ && l.get_crc() == r.get_crc() > >+ && l.get_namespace() == r.get_namespace()); > > > > if (equals && l.is_variable()) > > // These are variable symbols. Let's compare their symbol size. > >@@ -2146,6 +2158,20 @@ void > > elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) > > {priv_->crc_ = crc;} > > > >+/// Getter of the 'namespace' property. > >+/// > >+/// @return the namespace for Linux Kernel symbols, if any > >+const abg_compat::optional<std::string>& > >+elf_symbol::get_namespace() const > >+{return priv_->namespace_;} > >+ > >+/// Setter of the 'namespace' property. > >+/// > >+/// @param ns the new namespace for Linux Kernel symbols, if any > >+void > >+elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) > >+{priv_->namespace_ = ns;} > >+ > > /// Getter for the 'is-suppressed' property. > > /// > > /// @return true iff the current symbol has been suppressed by a > >diff --git a/src/abg-reader.cc b/src/abg-reader.cc > >index 7a9ad1f9..f9e420f1 100644 > >--- a/src/abg-reader.cc > >+++ b/src/abg-reader.cc > >@@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, > > if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) > > e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); > > > >+ if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) > >+ { > >+ std::string ns; > >+ xml::xml_char_sptr_to_string(s, ns); > >+ e->set_namespace(ns); > >+ } > >+ > > return e; > > } > > > >diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc > >index f9dd928b..9ad733f4 100644 > >--- a/src/abg-reporter-priv.cc > >+++ b/src/abg-reporter-priv.cc > >@@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, > > out << std::noshowbase << std::dec > > << "\n"; > > } > >+ > >+ const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); > >+ const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); > >+ if (ns1 != ns2) > >+ { > >+ const std::string none = "(none)"; > >+ out << indent << "namespace changed from "; > >+ if (ns1.has_value()) > >+ out << "'" << ns1.value() << "'"; > >+ else > >+ out << none; > >+ out << " to "; > >+ if (ns2.has_value()) > >+ out << "'" << ns2.value() << "'"; > >+ else > >+ out << none; > >+ out << "\n"; > >+ } > > } > > > > /// For a given symbol, emit a string made of its name and version. > >diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc > >index b42ce87d..123ef84a 100644 > >--- a/src/abg-symtab-reader.cc > >+++ b/src/abg-symtab-reader.cc > >@@ -204,6 +204,13 @@ symtab::load_(Elf* elf_handle, > > ir::environment* env, > > symbol_predicate is_suppressed) > > { > >+ GElf_Ehdr ehdr_mem; > >+ GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem); > >+ if (!header) > >+ { > >+ std::cerr << "Could not load get ELF header: Skipping symtab load.\n"; > > nit: "Could not load ELF header: ..." > > >+ return false; > >+ } > > > > Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle); > > if (!symtab_section) > >@@ -232,9 +239,34 @@ symtab::load_(Elf* elf_handle, > > return false; > > } > > > >+ // The __kstrtab_strings sections is basically an ELF strtab but does not > >+ // support elf_strptr lookups. A single call to elf_getdata gives a handle to > >+ // section data. > > raw section data Documentation actually refers to "washed" data, so "raw" doesn't sound right. I'll insert "washed". > > >+ // > >+ // The value of a __kstrtabns_FOO (or other similar) symbol is an address > >+ // within the __kstrtab_strings section. To look up the string value, we need > >+ // to translate from vmlinux load address to section offset by subtracting the > >+ // base address of the section. This adjustment is not needed for loadable > >+ // modules which are relocatable. > > ... identifiable by ELF type ET_REL. > Done. > >+ Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); > >+ size_t strings_offset = 0; > >+ const char* strings_data = nullptr; > >+ size_t strings_size = 0; > >+ if (strings_section) > >+ { > >+ GElf_Shdr strings_sheader; > >+ gelf_getshdr(strings_section, &strings_sheader); > >+ strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr; > >+ Elf_Data* data = elf_getdata(strings_section, nullptr); > >+ ABG_ASSERT(data->d_off == 0); > >+ strings_data = reinterpret_cast<const char *>(data->d_buf); > >+ strings_size = data->d_size; > > Since you later assume strings_data and strings_size being valid based > on the existence of strings_section, perhaps assert that the extraction > worked? > ABG_ASSERT(strings_data); > ABG_ASSERT(strings_size > 0); // otherwise, why would there be a section? > If the section is empty, null and 0 are expected. Later offset bounds checking will assert if any lookups are actually attempted. Any checking here is at best redundant and at worst (very theoretically) premature. If strings_data is outside mapped memory and strings_size is non-zero that will SEGV. I don't think it's worth doing special checks for the various strings_size == 0 possibilities. > >+ } > >+ > > const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); > > std::unordered_set<std::string> exported_kernel_symbols; > > std::unordered_map<std::string, uint64_t> crc_values; > >+ std::unordered_map<std::string, std::string> namespaces; > > > > const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); > > const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); > >@@ -288,6 +320,29 @@ symtab::load_(Elf* elf_handle, > > ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); > > continue; > > } > >+ if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) > >+ { > >+ // This symbol lives in the __ksymtab_strings section but st_value may > >+ // be a vmlinux load address so we need to subtract the offset before > >+ // looking it up in that section. > >+ const size_t value = sym->st_value; > > odd indentation > Done (tabification). > >+ const size_t offset = value - strings_offset; > >+ // check offset > >+ ABG_ASSERT(offset < strings_size); > >+ // find the terminating NULL > >+ const char* first = strings_data + offset; > >+ const char* last = strings_data + strings_size; > >+ const char* limit = std::find(first, last, 0); > >+ // check NULL found > >+ ABG_ASSERT(limit < last); > >+ // interpret the empty namespace name as no namespace name > >+ if (first < limit) > >+ { > >+ const std::string ns(first, limit - first); > >+ ABG_ASSERT(namespaces.emplace(name.substr(12), ns).second); > > ABG_ASSERT(namespaces.emplace(name.substr(12), std::move(ns)).second); > Done (inlined). > > With the few nits addressed, feel free to add > > Reviewed-by: Matthias Maennich <maennich@google.com> > > Cheers, > Matthias > Thanks! Giuliano. > >+ } > >+ continue; > >+ } > > > > // filter out uninteresting entries and only keep functions/variables for > > // now. The rest might be interesting in the future though. > >@@ -402,6 +457,17 @@ symtab::load_(Elf* elf_handle, > > symbol->set_crc(crc_entry.second); > > } > > > >+ // Now add the namespaces > >+ for (const auto& namespace_entry : namespaces) > >+ { > >+ const auto r = name_symbol_map_.find(namespace_entry.first); > >+ if (r == name_symbol_map_.end()) > >+ continue; > >+ > >+ for (const auto& symbol : r->second) > >+ symbol->set_namespace(namespace_entry.second); > >+ } > >+ > > // sort the symbols for deterministic output > > std::sort(symbols_.begin(), symbols_.end(), symbol_sort); > > > >diff --git a/src/abg-writer.cc b/src/abg-writer.cc > >index 0a1e7b66..03fb658b 100644 > >--- a/src/abg-writer.cc > >+++ b/src/abg-writer.cc > >@@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, > > << std::hex << std::showbase << sym->get_crc().value() > > << std::dec << std::noshowbase << "'"; > > > >+ if (sym->get_namespace().has_value()) > >+ o << " namespace='" << sym->get_namespace().value() << "'"; > >+ > > o << "/>\n"; > > > > return true; > >diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am > >index 90bd1934..7388697b 100644 > >--- a/tests/data/Makefile.am > >+++ b/tests/data/Makefile.am > >@@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \ > > test-abidiff/test-crc-report-0-1.txt \ > > test-abidiff/test-crc-report-1-0.txt \ > > test-abidiff/test-crc-report-1-2.txt \ > >+test-abidiff/test-namespace-0.xml \ > >+test-abidiff/test-namespace-1.xml \ > >+test-abidiff/test-namespace-report.txt \ > > test-abidiff/test-PR27985-report.txt \ > > test-abidiff/test-PR27985-v0.c \ > > test-abidiff/test-PR27985-v0.o \ > >diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml > >new file mode 100644 > >index 00000000..5a9f5cd2 > >--- /dev/null > >+++ b/tests/data/test-abidiff/test-namespace-0.xml > >@@ -0,0 +1,15 @@ > >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> > >+ <elf-variable-symbols> > >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> > >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> > >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> > >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> > >+ </elf-variable-symbols> > >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> > >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> > >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> > >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> > >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> > >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> > >+ </abi-instr> > >+</abi-corpus> > >diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml > >new file mode 100644 > >index 00000000..9814844b > >--- /dev/null > >+++ b/tests/data/test-abidiff/test-namespace-1.xml > >@@ -0,0 +1,15 @@ > >+<abi-corpus path='test.o' architecture='elf-amd-x86_64'> > >+ <elf-variable-symbols> > >+ <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> > >+ <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> > >+ <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> > >+ <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> > >+ </elf-variable-symbols> > >+ <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> > >+ <type-decl name='int' size-in-bits='32' id='type-id-1'/> > >+ <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> > >+ <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> > >+ <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> > >+ <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> > >+ </abi-instr> > >+</abi-corpus> > >diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt > >new file mode 100644 > >index 00000000..d2c421ed > >--- /dev/null > >+++ b/tests/data/test-abidiff/test-namespace-report.txt > >@@ -0,0 +1,17 @@ > >+Functions changes summary: 0 Removed, 0 Changed, 0 Added function > >+Variables changes summary: 0 Removed, 4 Changed, 0 Added variables > >+ > >+4 Changed variables: > >+ > >+ [C] 'int v1' was changed: > >+ namespace changed from (none) to 'this' > >+ > >+ [C] 'int v2' was changed: > >+ namespace changed from 'this' to 'that' > >+ > >+ [C] 'int v3' was changed: > >+ namespace changed from 'that' to '' > >+ > >+ [C] 'int v4' was changed: > >+ namespace changed from '' to (none) > >+ > >diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc > >index a02bbe3d..93e44d6a 100644 > >--- a/tests/test-abidiff.cc > >+++ b/tests/test-abidiff.cc > >@@ -128,6 +128,12 @@ static InOutSpec specs[] = > > "data/test-abidiff/test-crc-report-1-2.txt", > > "output/test-abidiff/test-crc-report-1-2.txt" > > }, > >+ { > >+ "data/test-abidiff/test-namespace-0.xml", > >+ "data/test-abidiff/test-namespace-1.xml", > >+ "data/test-abidiff/test-namespace-report.txt", > >+ "output/test-abidiff/test-namespace-report-0-1.txt" > >+ }, > > { > > "data/test-abidiff/test-PR27616-v0.xml", > > "data/test-abidiff/test-PR27616-v1.xml", > >-- > >2.35.1.894.gb6a874cedc-goog > > ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v4 0/4] add symbol namespace support, update symbol CRC support 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (3 preceding siblings ...) 2022-03-17 16:38 ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-03-21 16:02 ` Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida ` (8 more replies) 4 siblings, 9 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich New in v4: * address review comments on symtab reader changes Giuliano Procida (4): crc_changed: eliminate copying of shared_ptr values optional: minor improvements Linux symbol CRCs: support 0 and report presence changes add Linux kernel symbol namespace support include/abg-cxx-compat.h | 30 +++++++-- include/abg-ir.h | 17 +++-- src/abg-comp-filter.cc | 46 +++++++++++-- src/abg-ir.cc | 47 ++++++++++---- src/abg-reader.cc | 15 +++-- src/abg-reporter-priv.cc | 38 +++++++++-- src/abg-symtab-reader.cc | 64 +++++++++++++++++++ src/abg-writer.cc | 9 ++- tests/data/Makefile.am | 7 +- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 12 +++- tests/test-symtab.cc | 8 +-- 17 files changed, 325 insertions(+), 47 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida @ 2022-03-21 16:02 ` Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida ` (7 subsequent siblings) 8 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich As pointed out in a review of similar code, it is possible to avoid copying a couple of shared pointers in this function, by taking references instead. This commit also splits declarations to one per line and removes the unnecessary parentheses around the return expression. * src/abg-comp-filter.cc (crc_changed): Take references to avoid std::shared_ptr copying. Split declarations into one per line. Remove unnecessary return expression parentheses. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- src/abg-comp-filter.cc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 56251274..f90fdc78 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -230,11 +230,13 @@ static bool crc_changed(const function_or_var_decl_sptr& f, const function_or_var_decl_sptr& s) { - const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc(); - return (crc_f != 0 && crc_s != 0 && crc_f != crc_s); + const auto crc_f = symbol_f->get_crc(); + const auto crc_s = symbol_s->get_crc(); + return crc_f != 0 && crc_s != 0 && crc_f != crc_s; } /// Test if the current diff tree node carries a CRC change in either a -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v4 2/4] optional: minor improvements 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-03-21 16:02 ` Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida ` (6 subsequent siblings) 8 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich This change makes minor improvements to the optional class used with pre-C++17 compilers. - adds operator== and operator!= - adds various missing noexcept (but not constexpr) decorations - defines operator bool in terms of has_value Note that some constexpr decorations would require C++17 anyway. * include/abg-cxx-compat.h (optional): Add operator== and operator!=. Add noexcept decorations. Tweak operator bool. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h index 443905c7..5c5943d0 100644 --- a/include/abg-cxx-compat.h +++ b/include/abg-cxx-compat.h @@ -45,7 +45,7 @@ public: optional(const T& value) : has_value_(true), value_(value) {} bool - has_value() const + has_value() const noexcept { return has_value_; } @@ -67,19 +67,19 @@ public: } const T& - operator*() const + operator*() const& noexcept { return value_; } T& - operator*() + operator*() & noexcept { return value_; } const T* - operator->() const + operator->() const noexcept { return &value_; } T* - operator->() + operator->() noexcept { return &value_; } optional& @@ -90,9 +90,27 @@ public: return *this; } - explicit operator bool() const { return has_value_; } + explicit operator bool() const noexcept { return has_value(); } }; +template <typename T, typename U> +bool +operator==(const optional<T>& lhs, const optional<U>& rhs) +{ + if (!lhs.has_value() && !rhs.has_value()) + return true; + if (!lhs.has_value() || !rhs.has_value()) + return false; + return lhs.value() == rhs.value(); +} + +template <typename T, typename U> +bool +operator!=(const optional<T>& lhs, const optional<U>& rhs) +{ + return !(lhs == rhs); +} + #endif } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida @ 2022-03-21 16:02 ` Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida ` (5 subsequent siblings) 8 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich The CRC with value zero was used to mean "absent". This can be better modelled using optional. This commit makes this change and also tweaks reporting so that disappearing / appearing CRCs are noted. This should be essentially impossible unless CRCs are enabled / disabled altogether but would be very noteworthy otherwise. * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is now an optional defaulted to absent. (elf_symbol::create): Likewise. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_src): Now takes an optional uint64_t. * src/abg-comp-filter.cc (crc_changed): Simplify comparison. * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an optional uint64_t. (elf_symbol::priv::priv): Argument crc is now an optional uint64_t. (elf_symbol::elf_symbol): Likewise. (elf_symbol::create): Argument crc is now an optional uint64_t and defaults to absent. (textually_equals): Simplify comparison. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_crc): Now takes an optional uint64_t. * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same as other CRC values. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): Treat CRC 0 the same as other CRC values and also report changes to CRC presence. * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same as other CRC values. * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. * tests/data/test-abidiff/test-crc-report-0-1.txt: Report showing additional of CRCs. * tests/data/test-abidiff/test-crc-report-1-0.txt: Report showing removal of CRCs. * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed from tests/data/test-abidiff/test-crc-report.txt. * tests/test-abidiff.cc: Update test cases that no longer generate empty reports. * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 9 +++++---- src/abg-comp-filter.cc | 4 +--- src/abg-ir.cc | 19 +++++++++--------- src/abg-reader.cc | 8 ++------ src/abg-reporter-priv.cc | 20 ++++++++++++++----- src/abg-writer.cc | 6 +++--- tests/data/Makefile.am | 4 +++- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/test-abidiff.cc | 6 +++--- tests/test-symtab.cc | 8 ++++---- 12 files changed, 77 insertions(+), 39 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) diff --git a/include/abg-ir.h b/include/abg-ir.h index a2f4e1a7..b05a8c6f 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -21,6 +21,7 @@ #include <functional> #include <set> #include <unordered_map> +#include "abg-cxx-compat.h" #include "abg-fwd.h" #include "abg-hash.h" #include "abg-traverse.h" @@ -920,7 +921,7 @@ private: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -945,7 +946,7 @@ public: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); const environment* @@ -1017,11 +1018,11 @@ public: void set_is_in_ksymtab(bool is_in_ksymtab); - uint64_t + const abg_compat::optional<uint64_t>& get_crc() const; void - set_crc(uint64_t crc); + set_crc(const abg_compat::optional<uint64_t>& crc); bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index f90fdc78..31590284 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f, const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(); - const auto crc_s = symbol_s->get_crc(); - return crc_f != 0 && crc_s != 0 && crc_f != crc_s; + return symbol_f->get_crc() != symbol_s->get_crc(); } /// Test if the current diff tree node carries a CRC change in either a diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 0ef5e8b2..853b01ee 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1727,7 +1727,7 @@ struct elf_symbol::priv // STT_COMMON definition of that name that has the largest size. bool is_common_; bool is_in_ksymtab_; - uint64_t crc_; + abg_compat::optional<uint64_t> crc_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1744,7 +1744,7 @@ struct elf_symbol::priv is_defined_(false), is_common_(false), is_in_ksymtab_(false), - crc_(0), + crc_(), is_suppressed_(false) {} @@ -1759,7 +1759,7 @@ struct elf_symbol::priv const elf_symbol::version& ve, elf_symbol::visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : env_(e), index_(i), @@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : priv_(new priv(e, i, @@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, @@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && (l.get_crc() == 0 || r.get_crc() == 0 - || l.get_crc() == r.get_crc())); + && l.get_crc() == r.get_crc()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab) /// Getter of the 'crc' property. /// -/// @return the CRC (modversions) value for Linux Kernel symbols (if present) -uint64_t +/// @return the CRC (modversions) value for Linux Kernel symbols, if any +const abg_compat::optional<uint64_t>& elf_symbol::get_crc() const {return priv_->crc_;} @@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const /// /// @param crc the new CRC (modversions) value for Linux Kernel symbols void -elf_symbol::set_crc(uint64_t crc) +elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} /// Getter for the 'is-suppressed' property. diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 31885692..7a9ad1f9 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, is_default_version = true; } - uint64_t crc = 0; - if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) - crc = strtoull(CHAR_STR(s), NULL, 0); - elf_symbol::type type = elf_symbol::NOTYPE_TYPE; read_elf_symbol_type(node, type); @@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, e->set_is_suppressed(is_suppressed); - if (crc != 0) - e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) + e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 7012f5dc..f9dd928b 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << "\n"; } - if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0 - && symbol1->get_crc() != symbol2->get_crc()) + const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc(); + const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc(); + if (crc1 != crc2) { + const std::string none = "(none)"; out << indent << "CRC (modversions) changed from " - << std::showbase << std::hex - << symbol1->get_crc() << " to " << symbol2->get_crc() - << std::noshowbase << std::dec + << std::showbase << std::hex; + if (crc1.has_value()) + out << crc1.value(); + else + out << none; + out << " to "; + if (crc2.has_value()) + out << crc2.value(); + else + out << none; + out << std::noshowbase << std::dec << "\n"; } } diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 7802128d..0a1e7b66 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3131,10 +3131,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, if (sym->is_common_symbol()) o << " is-common='yes'"; - if (sym->get_crc() != 0) + if (sym->get_crc().has_value()) o << " crc='" - << std::hex << std::showbase << sym->get_crc() << "'" - << std::dec << std::noshowbase; + << std::hex << std::showbase << sym->get_crc().value() + << std::dec << std::noshowbase << "'"; o << "/>\n"; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index a7eb7ff0..90bd1934 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml \ test-abidiff/test-crc-0.xml \ test-abidiff/test-crc-1.xml \ test-abidiff/test-crc-2.xml \ -test-abidiff/test-crc-report.txt \ +test-abidiff/test-crc-report-0-1.txt \ +test-abidiff/test-crc-report-1-0.txt \ +test-abidiff/test-crc-report-1-2.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt new file mode 100644 index 00000000..0db42f68 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-0-1.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from (none) to 0xe52d5bcf + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from (none) to 0xee94d699 + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from (none) to 0x41336c46 + diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt new file mode 100644 index 00000000..e11f29c1 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-1-0.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from 0xe52d5bcf to (none) + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from 0xee94d699 to (none) + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from 0x41336c46 to (none) + diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt similarity index 100% rename from tests/data/test-abidiff/test-crc-report.txt rename to tests/data/test-abidiff/test-crc-report-1-2.txt diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index 32858ece..a02bbe3d 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -113,19 +113,19 @@ static InOutSpec specs[] = { "data/test-abidiff/test-crc-0.xml", "data/test-abidiff/test-crc-1.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-0-1.txt", "output/test-abidiff/test-crc-report-0-1.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-0.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-1-0.txt", "output/test-abidiff/test-crc-report-1-0.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-2.xml", - "data/test-abidiff/test-crc-report.txt", + "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, { diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc index 7d7a2df0..85303563 100644 --- a/tests/test-symtab.cc +++ b/tests/test-symtab.cc @@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]") { const std::string binary = base_path + "one_of_each.ko"; const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2); - CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0); - CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0); + CHECK(corpus->lookup_function_symbol("exported_function")->get_crc()); + CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc()); } } -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v4 4/4] add Linux kernel symbol namespace support 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (2 preceding siblings ...) 2022-03-21 16:02 ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-03-21 16:02 ` Giuliano Procida 2022-06-13 14:25 ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (4 subsequent siblings) 8 siblings, 0 replies; 37+ messages in thread From: Giuliano Procida @ 2022-03-21 16:02 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Bug 28954 - add Linux Kernel symbol namespace support Each Linux kernel symbol can be exported to a specified named namespace or left in the global (nameless) namespace. One complexity is that the symbol values which identify a string in the __ksymtab_strings section must be interpretated differently for vmlinux and .ko loadable modules as the former has a fixed load address but the latter are relocatable. For vmlinux, the section base address needs to be subtracted to obtain a section-relative offset. The global namespace is explicitly represented as the empty string, at least when it comes to the value of __kstrtabns_FOO symbols, but the common interpretation is that such symbols lack an export namespace. I would rather not have to make use of "empty implies missing" in many places, so the code here represents namespace as optional<string> and only the symtab reader cares about empty strings in __ksymtab_strings. * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. (elf_symbol::create): Add ns argument. (elf_symbol::get_namespace): Declare new function. (elf_symbol::set_namespace): Declare new function. and set_namespace. * src/abg-comp-filter.cc (namespace_changed): Define new helper functions. (categorize_harmful_diff_node): Also call namespace_changed(). * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. (elf_symbol::priv::priv): Add namespace_ to initialisers. (elf_symbol::elf_symbol): Take new ns argument and pass it to priv constructor. (elf_symbol::create): Take new ns argument and pass it to elf_symbol constructor. (elf_symbol::get_namespace): Define new function. (elf_symbol::set_namespace): Define new function. * src/abg-reader.cc (build_elf_symbol): If namespace attribute is present, set symbol namespace. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If symbol namespaces differ, report this. * src/abg-symtab-reader.cc (symtab::load): Get ELF header to distinguish vmlinux from .ko. Try to get __ksymtab_strings metadata and data. Use these to look up __kstrtabns_FOO namespace entries. Set symbol namespace where found. * src/abg-writer.cc (write_elf_symbol): Emit namespace attribute, if symbol has a namespace. * tests/data/Makefile.am: Add new test files. * tests/data/test-abidiff/test-namespace-0.xml: New test file. * tests/data/test-abidiff/test-namespace-1.xml: Likewise * tests/data/test-abidiff/test-namespace-report.txt: Likewise. * tests/test-abidiff.cc: Add new test case. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 8 +++ src/abg-comp-filter.cc | 40 +++++++++++- src/abg-ir.cc | 30 ++++++++- src/abg-reader.cc | 7 ++ src/abg-reporter-priv.cc | 18 ++++++ src/abg-symtab-reader.cc | 64 +++++++++++++++++++ src/abg-writer.cc | 3 + tests/data/Makefile.am | 3 + tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 6 ++ 12 files changed, 223 insertions(+), 3 deletions(-) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt diff --git a/include/abg-ir.h b/include/abg-ir.h index b05a8c6f..7c558828 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -922,6 +922,7 @@ private: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -947,6 +948,7 @@ public: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); const environment* @@ -1024,6 +1026,12 @@ public: void set_crc(const abg_compat::optional<uint64_t>& crc); + const abg_compat::optional<std::string>& + get_namespace() const; + + void + set_namespace(const abg_compat::optional<std::string>& ns); + bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 31590284..22da5244 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -254,6 +254,43 @@ crc_changed(const diff* diff) return false; } +/// Test if there was a function or variable namespace change. +/// +/// @param f the first function or variable to consider. +/// +/// @param s the second function or variable to consider. +/// +/// @return true if the test is positive, false otherwise. +template <typename function_or_var_decl_sptr> +static bool +namespace_changed(const function_or_var_decl_sptr& f, + const function_or_var_decl_sptr& s) +{ + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); + if (!symbol_f || !symbol_s) + return false; + return symbol_f->get_namespace() != symbol_s->get_namespace(); +} + +/// Test if the current diff tree node carries a namespace change in +/// either a function or a variable. +/// +/// @param diff the diff tree node to consider. +/// +/// @return true if the test is positive, false otherwise. +static bool +namespace_changed(const diff* diff) +{ + if (const function_decl_diff* d = + dynamic_cast<const function_decl_diff*>(diff)) + return namespace_changed(d->first_function_decl(), + d->second_function_decl()); + if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) + return namespace_changed(d->first_var(), d->second_var()); + return false; +} + /// Test if there was a function name change, but there there was no /// change in name of the underlying symbol. IOW, if the name of a /// function changed, but the symbol of the new function is equal to @@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre) || non_static_data_member_added_or_removed(d) || base_classes_added_or_removed(d) || has_harmful_enum_change(d) - || crc_changed(d))) + || crc_changed(d) + || namespace_changed(d))) category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; if (has_virtual_mem_fn_change(d)) diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 853b01ee..f90be2e4 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1728,6 +1728,7 @@ struct elf_symbol::priv bool is_common_; bool is_in_ksymtab_; abg_compat::optional<uint64_t> crc_; + abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1745,6 +1746,7 @@ struct elf_symbol::priv is_common_(false), is_in_ksymtab_(false), crc_(), + namespace_(), is_suppressed_(false) {} @@ -1760,6 +1762,7 @@ struct elf_symbol::priv elf_symbol::visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), index_(i), @@ -1773,6 +1776,7 @@ struct elf_symbol::priv is_common_(c), is_in_ksymtab_(is_in_ksymtab), crc_(crc), + namespace_(ns), is_suppressed_(is_suppressed) { if (!is_common_) @@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() /// @param vi the visibility of the symbol. /// /// @param crc the CRC (modversions) value of Linux Kernel symbols +/// +/// @param ns the namespace of Linux Kernel symbols, if any elf_symbol::elf_symbol(const environment* e, size_t i, size_t s, @@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, i, @@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, vi, is_in_ksymtab, crc, + ns, is_suppressed)) {} @@ -1886,6 +1894,8 @@ elf_symbol::create() /// /// @param crc the CRC (modversions) value of Linux Kernel symbols /// +/// @param ns the namespace of Linux Kernel symbols, if any +/// /// @return a (smart) pointer to a newly created instance of @ref /// elf_symbol. elf_symbol_sptr @@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, - is_in_ksymtab, crc, is_suppressed)); + is_in_ksymtab, crc, ns, is_suppressed)); sym->priv_->main_symbol_ = sym; return sym; } @@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && l.get_crc() == r.get_crc()); + && l.get_crc() == r.get_crc() + && l.get_namespace() == r.get_namespace()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2146,6 +2158,20 @@ void elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} +/// Getter of the 'namespace' property. +/// +/// @return the namespace for Linux Kernel symbols, if any +const abg_compat::optional<std::string>& +elf_symbol::get_namespace() const +{return priv_->namespace_;} + +/// Setter of the 'namespace' property. +/// +/// @param ns the new namespace for Linux Kernel symbols, if any +void +elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) +{priv_->namespace_ = ns;} + /// Getter for the 'is-suppressed' property. /// /// @return true iff the current symbol has been suppressed by a diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 7a9ad1f9..f9e420f1 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) + { + std::string ns; + xml::xml_char_sptr_to_string(s, ns); + e->set_namespace(ns); + } + return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index f9dd928b..9ad733f4 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, out << std::noshowbase << std::dec << "\n"; } + + const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); + const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); + if (ns1 != ns2) + { + const std::string none = "(none)"; + out << indent << "namespace changed from "; + if (ns1.has_value()) + out << "'" << ns1.value() << "'"; + else + out << none; + out << " to "; + if (ns2.has_value()) + out << "'" << ns2.value() << "'"; + else + out << none; + out << "\n"; + } } /// For a given symbol, emit a string made of its name and version. diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc index b42ce87d..92509054 100644 --- a/src/abg-symtab-reader.cc +++ b/src/abg-symtab-reader.cc @@ -204,6 +204,13 @@ symtab::load_(Elf* elf_handle, ir::environment* env, symbol_predicate is_suppressed) { + GElf_Ehdr ehdr_mem; + GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem); + if (!header) + { + std::cerr << "Could not load get ELF header: Skipping symtab load.\n"; + return false; + } Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle); if (!symtab_section) @@ -232,9 +239,34 @@ symtab::load_(Elf* elf_handle, return false; } + // The __kstrtab_strings sections is basically an ELF strtab but does not + // support elf_strptr lookups. A single call to elf_getdata gives a handle to + // washed section data. + // + // The value of a __kstrtabns_FOO (or other similar) symbol is an address + // within the __kstrtab_strings section. To look up the string value, we need + // to translate from vmlinux load address to section offset by subtracting the + // base address of the section. This adjustment is not needed for loadable + // modules which are relocatable and so identifiable by ELF type ET_REL. + Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); + size_t strings_offset = 0; + const char* strings_data = nullptr; + size_t strings_size = 0; + if (strings_section) + { + GElf_Shdr strings_sheader; + gelf_getshdr(strings_section, &strings_sheader); + strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr; + Elf_Data* data = elf_getdata(strings_section, nullptr); + ABG_ASSERT(data->d_off == 0); + strings_data = reinterpret_cast<const char *>(data->d_buf); + strings_size = data->d_size; + } + const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); std::unordered_set<std::string> exported_kernel_symbols; std::unordered_map<std::string, uint64_t> crc_values; + std::unordered_map<std::string, std::string> namespaces; const bool is_arm32 = elf_helpers::architecture_is_arm32(elf_handle); const bool is_ppc64 = elf_helpers::architecture_is_ppc64(elf_handle); @@ -288,6 +320,27 @@ symtab::load_(Elf* elf_handle, ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); continue; } + if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) + { + // This symbol lives in the __ksymtab_strings section but st_value may + // be a vmlinux load address so we need to subtract the offset before + // looking it up in that section. + const size_t value = sym->st_value; + const size_t offset = value - strings_offset; + // check offset + ABG_ASSERT(offset < strings_size); + // find the terminating NULL + const char* first = strings_data + offset; + const char* last = strings_data + strings_size; + const char* limit = std::find(first, last, 0); + // check NULL found + ABG_ASSERT(limit < last); + // interpret the empty namespace name as no namespace name + if (first < limit) + ABG_ASSERT(namespaces.emplace( + name.substr(12), std::string(first, limit - first)).second); + continue; + } // filter out uninteresting entries and only keep functions/variables for // now. The rest might be interesting in the future though. @@ -402,6 +455,17 @@ symtab::load_(Elf* elf_handle, symbol->set_crc(crc_entry.second); } + // Now add the namespaces + for (const auto& namespace_entry : namespaces) + { + const auto r = name_symbol_map_.find(namespace_entry.first); + if (r == name_symbol_map_.end()) + continue; + + for (const auto& symbol : r->second) + symbol->set_namespace(namespace_entry.second); + } + // sort the symbols for deterministic output std::sort(symbols_.begin(), symbols_.end(), symbol_sort); diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 0a1e7b66..03fb658b 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3136,6 +3136,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, << std::hex << std::showbase << sym->get_crc().value() << std::dec << std::noshowbase << "'"; + if (sym->get_namespace().has_value()) + o << " namespace='" << sym->get_namespace().value() << "'"; + o << "/>\n"; return true; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index 90bd1934..7388697b 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \ test-abidiff/test-crc-report-0-1.txt \ test-abidiff/test-crc-report-1-0.txt \ test-abidiff/test-crc-report-1-2.txt \ +test-abidiff/test-namespace-0.xml \ +test-abidiff/test-namespace-1.xml \ +test-abidiff/test-namespace-report.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml new file mode 100644 index 00000000..5a9f5cd2 --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-0.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml new file mode 100644 index 00000000..9814844b --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-1.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt new file mode 100644 index 00000000..d2c421ed --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-report.txt @@ -0,0 +1,17 @@ +Functions changes summary: 0 Removed, 0 Changed, 0 Added function +Variables changes summary: 0 Removed, 4 Changed, 0 Added variables + +4 Changed variables: + + [C] 'int v1' was changed: + namespace changed from (none) to 'this' + + [C] 'int v2' was changed: + namespace changed from 'this' to 'that' + + [C] 'int v3' was changed: + namespace changed from 'that' to '' + + [C] 'int v4' was changed: + namespace changed from '' to (none) + diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index a02bbe3d..93e44d6a 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -128,6 +128,12 @@ static InOutSpec specs[] = "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, + { + "data/test-abidiff/test-namespace-0.xml", + "data/test-abidiff/test-namespace-1.xml", + "data/test-abidiff/test-namespace-report.txt", + "output/test-abidiff/test-namespace-report-0-1.txt" + }, { "data/test-abidiff/test-PR27616-v0.xml", "data/test-abidiff/test-PR27616-v1.xml", -- 2.35.1.894.gb6a874cedc-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v5 0/4] add symbol namespace support, update symbol CRC support 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (3 preceding siblings ...) 2022-03-21 16:02 ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-06-13 14:25 ` Giuliano Procida 2022-06-30 16:39 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida ` (3 subsequent siblings) 8 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Hi Dodji. I'd be grateful if could take a look at this patch series. Patch 4/4 in particular addresses Bug 28954. Thank you! Regards, Giuliano. New in v5: * improve error message when ELF header not found Giuliano Procida (4): crc_changed: eliminate copying of shared_ptr values optional: minor improvements Linux symbol CRCs: support 0 and report presence changes add Linux kernel symbol namespace support include/abg-cxx-compat.h | 30 +++++++-- include/abg-ir.h | 17 +++-- src/abg-comp-filter.cc | 46 +++++++++++-- src/abg-ir.cc | 47 ++++++++++---- src/abg-reader.cc | 15 +++-- src/abg-reporter-priv.cc | 38 +++++++++-- src/abg-symtab-reader.cc | 64 +++++++++++++++++++ src/abg-writer.cc | 9 ++- tests/data/Makefile.am | 7 +- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 12 +++- tests/test-symtab.cc | 8 +-- 17 files changed, 325 insertions(+), 47 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v5 0/4] add symbol namespace support, update symbol CRC support 2022-06-13 14:25 ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida @ 2022-06-30 16:39 ` Dodji Seketeli 0 siblings, 0 replies; 37+ messages in thread From: Dodji Seketeli @ 2022-06-30 16:39 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich Giuliano Procida <gprocida@google.com> a écrit: > Hi Dodji. Hey Giuliano, Sorry for the late reply. > I'd be grateful if could take a look at this patch series. Patch 4/4 > in particular addresses Bug 28954. Yeah, I am looking into it at the moment. Thanks for the great work! [...] Cheers, -- Dodji ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (4 preceding siblings ...) 2022-06-13 14:25 ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida @ 2022-06-13 14:25 ` Giuliano Procida 2022-06-30 16:40 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida ` (2 subsequent siblings) 8 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich As pointed out in a review of similar code, it is possible to avoid copying a couple of shared pointers in this function, by taking references instead. This commit also splits declarations to one per line and removes the unnecessary parentheses around the return expression. * src/abg-comp-filter.cc (crc_changed): Take references to avoid std::shared_ptr copying. Split declarations into one per line. Remove unnecessary return expression parentheses. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- src/abg-comp-filter.cc | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 56251274..f90fdc78 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -230,11 +230,13 @@ static bool crc_changed(const function_or_var_decl_sptr& f, const function_or_var_decl_sptr& s) { - const auto symbol_f = f->get_symbol(), symbol_s = s->get_symbol(); + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(), crc_s = symbol_s->get_crc(); - return (crc_f != 0 && crc_s != 0 && crc_f != crc_s); + const auto crc_f = symbol_f->get_crc(); + const auto crc_s = symbol_s->get_crc(); + return crc_f != 0 && crc_s != 0 && crc_f != crc_s; } /// Test if the current diff tree node carries a CRC change in either a -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values 2022-06-13 14:25 ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-06-30 16:40 ` Dodji Seketeli 0 siblings, 0 replies; 37+ messages in thread From: Dodji Seketeli @ 2022-06-30 16:40 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich Hello, Giuliano Procida <gprocida@google.com> a écrit: > As pointed out in a review of similar code, it is possible to avoid > copying a couple of shared pointers in this function, by taking > references instead. > > This commit also splits declarations to one per line and removes the > unnecessary parentheses around the return expression. > > * src/abg-comp-filter.cc (crc_changed): Take references to > avoid std::shared_ptr copying. Split declarations into one per > line. Remove unnecessary return expression parentheses. > > Reviewed-by: Matthias Maennich <maennich@google.com> > Signed-off-by: Giuliano Procida <gprocida@google.com> Applied to master, thanks! [...] Cheers, -- Dodji ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v5 2/4] optional: minor improvements 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (5 preceding siblings ...) 2022-06-13 14:25 ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida @ 2022-06-13 14:25 ` Giuliano Procida 2022-06-30 16:40 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-06-13 14:25 ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida 8 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich This change makes minor improvements to the optional class used with pre-C++17 compilers. - adds operator== and operator!= - adds various missing noexcept (but not constexpr) decorations - defines operator bool in terms of has_value Note that some constexpr decorations would require C++17 anyway. * include/abg-cxx-compat.h (optional): Add operator== and operator!=. Add noexcept decorations. Tweak operator bool. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-cxx-compat.h | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/include/abg-cxx-compat.h b/include/abg-cxx-compat.h index 443905c7..5c5943d0 100644 --- a/include/abg-cxx-compat.h +++ b/include/abg-cxx-compat.h @@ -45,7 +45,7 @@ public: optional(const T& value) : has_value_(true), value_(value) {} bool - has_value() const + has_value() const noexcept { return has_value_; } @@ -67,19 +67,19 @@ public: } const T& - operator*() const + operator*() const& noexcept { return value_; } T& - operator*() + operator*() & noexcept { return value_; } const T* - operator->() const + operator->() const noexcept { return &value_; } T* - operator->() + operator->() noexcept { return &value_; } optional& @@ -90,9 +90,27 @@ public: return *this; } - explicit operator bool() const { return has_value_; } + explicit operator bool() const noexcept { return has_value(); } }; +template <typename T, typename U> +bool +operator==(const optional<T>& lhs, const optional<U>& rhs) +{ + if (!lhs.has_value() && !rhs.has_value()) + return true; + if (!lhs.has_value() || !rhs.has_value()) + return false; + return lhs.value() == rhs.value(); +} + +template <typename T, typename U> +bool +operator!=(const optional<T>& lhs, const optional<U>& rhs) +{ + return !(lhs == rhs); +} + #endif } -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v5 2/4] optional: minor improvements 2022-06-13 14:25 ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida @ 2022-06-30 16:40 ` Dodji Seketeli 0 siblings, 0 replies; 37+ messages in thread From: Dodji Seketeli @ 2022-06-30 16:40 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich Hello, Giuliano Procida <gprocida@google.com> a écrit: > This change makes minor improvements to the optional class used with > pre-C++17 compilers. > > - adds operator== and operator!= > - adds various missing noexcept (but not constexpr) decorations > - defines operator bool in terms of has_value > > Note that some constexpr decorations would require C++17 anyway. > > * include/abg-cxx-compat.h (optional): Add operator== and > operator!=. Add noexcept decorations. Tweak operator bool. > > Reviewed-by: Matthias Maennich <maennich@google.com> > Signed-off-by: Giuliano Procida <gprocida@google.com> Applied to master, thanks! [...] Cheers, -- Dodji ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (6 preceding siblings ...) 2022-06-13 14:25 ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida @ 2022-06-13 14:25 ` Giuliano Procida 2022-06-30 16:41 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida 8 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich The CRC with value zero was used to mean "absent". This can be better modelled using optional. This commit makes this change and also tweaks reporting so that disappearing / appearing CRCs are noted. This should be essentially impossible unless CRCs are enabled / disabled altogether but would be very noteworthy otherwise. * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is now an optional defaulted to absent. (elf_symbol::create): Likewise. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_src): Now takes an optional uint64_t. * src/abg-comp-filter.cc (crc_changed): Simplify comparison. * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an optional uint64_t. (elf_symbol::priv::priv): Argument crc is now an optional uint64_t. (elf_symbol::elf_symbol): Likewise. (elf_symbol::create): Argument crc is now an optional uint64_t and defaults to absent. (textually_equals): Simplify comparison. (elf_symbol::get_crc): Now returns an optional uint64_t. (elf_symbol::set_crc): Now takes an optional uint64_t. * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same as other CRC values. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): Treat CRC 0 the same as other CRC values and also report changes to CRC presence. * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same as other CRC values. * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. * tests/data/test-abidiff/test-crc-report-0-1.txt: Report showing additional of CRCs. * tests/data/test-abidiff/test-crc-report-1-0.txt: Report showing removal of CRCs. * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed from tests/data/test-abidiff/test-crc-report.txt. * tests/test-abidiff.cc: Update test cases that no longer generate empty reports. * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 9 +++++---- src/abg-comp-filter.cc | 4 +--- src/abg-ir.cc | 19 +++++++++--------- src/abg-reader.cc | 8 ++------ src/abg-reporter-priv.cc | 20 ++++++++++++++----- src/abg-writer.cc | 6 +++--- tests/data/Makefile.am | 4 +++- .../data/test-abidiff/test-crc-report-0-1.txt | 16 +++++++++++++++ .../data/test-abidiff/test-crc-report-1-0.txt | 16 +++++++++++++++ ...crc-report.txt => test-crc-report-1-2.txt} | 0 tests/test-abidiff.cc | 6 +++--- tests/test-symtab.cc | 8 ++++---- 12 files changed, 77 insertions(+), 39 deletions(-) create mode 100644 tests/data/test-abidiff/test-crc-report-0-1.txt create mode 100644 tests/data/test-abidiff/test-crc-report-1-0.txt rename tests/data/test-abidiff/{test-crc-report.txt => test-crc-report-1-2.txt} (100%) diff --git a/include/abg-ir.h b/include/abg-ir.h index a2f4e1a7..b05a8c6f 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -21,6 +21,7 @@ #include <functional> #include <set> #include <unordered_map> +#include "abg-cxx-compat.h" #include "abg-fwd.h" #include "abg-hash.h" #include "abg-traverse.h" @@ -920,7 +921,7 @@ private: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -945,7 +946,7 @@ public: const version& ve, visibility vi, bool is_in_ksymtab = false, - uint64_t crc = 0, + const abg_compat::optional<uint64_t>& crc = {}, bool is_suppressed = false); const environment* @@ -1017,11 +1018,11 @@ public: void set_is_in_ksymtab(bool is_in_ksymtab); - uint64_t + const abg_compat::optional<uint64_t>& get_crc() const; void - set_crc(uint64_t crc); + set_crc(const abg_compat::optional<uint64_t>& crc); bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index f90fdc78..31590284 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -234,9 +234,7 @@ crc_changed(const function_or_var_decl_sptr& f, const auto& symbol_s = s->get_symbol(); if (!symbol_f || !symbol_s) return false; - const auto crc_f = symbol_f->get_crc(); - const auto crc_s = symbol_s->get_crc(); - return crc_f != 0 && crc_s != 0 && crc_f != crc_s; + return symbol_f->get_crc() != symbol_s->get_crc(); } /// Test if the current diff tree node carries a CRC change in either a diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 4e907620..34ae486f 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1727,7 +1727,7 @@ struct elf_symbol::priv // STT_COMMON definition of that name that has the largest size. bool is_common_; bool is_in_ksymtab_; - uint64_t crc_; + abg_compat::optional<uint64_t> crc_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1744,7 +1744,7 @@ struct elf_symbol::priv is_defined_(false), is_common_(false), is_in_ksymtab_(false), - crc_(0), + crc_(), is_suppressed_(false) {} @@ -1759,7 +1759,7 @@ struct elf_symbol::priv const elf_symbol::version& ve, elf_symbol::visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : env_(e), index_(i), @@ -1829,7 +1829,7 @@ elf_symbol::elf_symbol(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) : priv_(new priv(e, i, @@ -1900,7 +1900,7 @@ elf_symbol::create(const environment* e, const version& ve, visibility vi, bool is_in_ksymtab, - uint64_t crc, + const abg_compat::optional<uint64_t>& crc, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, @@ -1926,8 +1926,7 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && (l.get_crc() == 0 || r.get_crc() == 0 - || l.get_crc() == r.get_crc())); + && l.get_crc() == r.get_crc()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2135,8 +2134,8 @@ elf_symbol::set_is_in_ksymtab(bool is_in_ksymtab) /// Getter of the 'crc' property. /// -/// @return the CRC (modversions) value for Linux Kernel symbols (if present) -uint64_t +/// @return the CRC (modversions) value for Linux Kernel symbols, if any +const abg_compat::optional<uint64_t>& elf_symbol::get_crc() const {return priv_->crc_;} @@ -2144,7 +2143,7 @@ elf_symbol::get_crc() const /// /// @param crc the new CRC (modversions) value for Linux Kernel symbols void -elf_symbol::set_crc(uint64_t crc) +elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} /// Getter for the 'is-suppressed' property. diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 31885692..7a9ad1f9 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3239,10 +3239,6 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, is_default_version = true; } - uint64_t crc = 0; - if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) - crc = strtoull(CHAR_STR(s), NULL, 0); - elf_symbol::type type = elf_symbol::NOTYPE_TYPE; read_elf_symbol_type(node, type); @@ -3266,8 +3262,8 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, e->set_is_suppressed(is_suppressed); - if (crc != 0) - e->set_crc(crc); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) + e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index 06145a1b..d000620b 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1149,13 +1149,23 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, << "\n"; } - if (symbol1->get_crc() != 0 && symbol2->get_crc() != 0 - && symbol1->get_crc() != symbol2->get_crc()) + const abg_compat::optional<uint64_t>& crc1 = symbol1->get_crc(); + const abg_compat::optional<uint64_t>& crc2 = symbol2->get_crc(); + if (crc1 != crc2) { + const std::string none = "(none)"; out << indent << "CRC (modversions) changed from " - << std::showbase << std::hex - << symbol1->get_crc() << " to " << symbol2->get_crc() - << std::noshowbase << std::dec + << std::showbase << std::hex; + if (crc1.has_value()) + out << crc1.value(); + else + out << none; + out << " to "; + if (crc2.has_value()) + out << crc2.value(); + else + out << none; + out << std::noshowbase << std::dec << "\n"; } } diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 3f50a900..23598ef0 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3070,10 +3070,10 @@ write_elf_symbol(const elf_symbol_sptr& sym, if (sym->is_common_symbol()) o << " is-common='yes'"; - if (sym->get_crc() != 0) + if (sym->get_crc().has_value()) o << " crc='" - << std::hex << std::showbase << sym->get_crc() << "'" - << std::dec << std::noshowbase; + << std::hex << std::showbase << sym->get_crc().value() + << std::dec << std::noshowbase << "'"; o << "/>\n"; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index a0c086d9..be342684 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -90,7 +90,9 @@ test-abidiff/test-empty-corpus-2.xml \ test-abidiff/test-crc-0.xml \ test-abidiff/test-crc-1.xml \ test-abidiff/test-crc-2.xml \ -test-abidiff/test-crc-report.txt \ +test-abidiff/test-crc-report-0-1.txt \ +test-abidiff/test-crc-report-1-0.txt \ +test-abidiff/test-crc-report-1-2.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-crc-report-0-1.txt b/tests/data/test-abidiff/test-crc-report-0-1.txt new file mode 100644 index 00000000..0db42f68 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-0-1.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from (none) to 0xe52d5bcf + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from (none) to 0xee94d699 + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from (none) to 0x41336c46 + diff --git a/tests/data/test-abidiff/test-crc-report-1-0.txt b/tests/data/test-abidiff/test-crc-report-1-0.txt new file mode 100644 index 00000000..e11f29c1 --- /dev/null +++ b/tests/data/test-abidiff/test-crc-report-1-0.txt @@ -0,0 +1,16 @@ +Functions changes summary: 0 Removed, 1 Changed, 0 Added function +Variables changes summary: 0 Removed, 2 Changed, 0 Added variables + +1 function with some indirect sub-type change: + + [C] 'function void exported_function()' has some indirect sub-type changes: + CRC (modversions) changed from 0xe52d5bcf to (none) + +2 Changed variables: + + [C] 'int exported_variable' was changed: + CRC (modversions) changed from 0xee94d699 to (none) + + [C] 'int exported_variable_gpl' was changed: + CRC (modversions) changed from 0x41336c46 to (none) + diff --git a/tests/data/test-abidiff/test-crc-report.txt b/tests/data/test-abidiff/test-crc-report-1-2.txt similarity index 100% rename from tests/data/test-abidiff/test-crc-report.txt rename to tests/data/test-abidiff/test-crc-report-1-2.txt diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index 32858ece..a02bbe3d 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -113,19 +113,19 @@ static InOutSpec specs[] = { "data/test-abidiff/test-crc-0.xml", "data/test-abidiff/test-crc-1.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-0-1.txt", "output/test-abidiff/test-crc-report-0-1.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-0.xml", - "data/test-abidiff/empty-report.txt", + "data/test-abidiff/test-crc-report-1-0.txt", "output/test-abidiff/test-crc-report-1-0.txt" }, { "data/test-abidiff/test-crc-1.xml", "data/test-abidiff/test-crc-2.xml", - "data/test-abidiff/test-crc-report.txt", + "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, { diff --git a/tests/test-symtab.cc b/tests/test-symtab.cc index 7d7a2df0..85303563 100644 --- a/tests/test-symtab.cc +++ b/tests/test-symtab.cc @@ -420,9 +420,9 @@ TEST_CASE("Symtab::KernelSymtabsWithCRC", "[symtab, crc, kernel, ksymtab]") { const std::string binary = base_path + "one_of_each.ko"; const corpus_sptr& corpus = assert_symbol_count(binary, 2, 2); - CHECK(corpus->lookup_function_symbol("exported_function")->get_crc() != 0); - CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc() != 0); - CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc() != 0); + CHECK(corpus->lookup_function_symbol("exported_function")->get_crc()); + CHECK(corpus->lookup_function_symbol("exported_function_gpl")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable")->get_crc()); + CHECK(corpus->lookup_variable_symbol("exported_variable_gpl")->get_crc()); } } -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes 2022-06-13 14:25 ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-06-30 16:41 ` Dodji Seketeli 0 siblings, 0 replies; 37+ messages in thread From: Dodji Seketeli @ 2022-06-30 16:41 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich Hello, Giuliano Procida <gprocida@google.com> a écrit: > The CRC with value zero was used to mean "absent". This can be better > modelled using optional. > > This commit makes this change and also tweaks reporting so that > disappearing / appearing CRCs are noted. This should be essentially > impossible unless CRCs are enabled / disabled altogether but would be > very noteworthy otherwise. > > * include/abg-ir.h (elf_symbol::elf_symbol): Argument crc is > now an optional defaulted to absent. > (elf_symbol::create): Likewise. > (elf_symbol::get_crc): Now returns an optional uint64_t. > (elf_symbol::set_src): Now takes an optional uint64_t. > * src/abg-comp-filter.cc (crc_changed): Simplify comparison. > * src/abg-ir.cc (elf_symbol::priv): Member crc_ is now an > optional uint64_t. > (elf_symbol::priv::priv): Argument crc is now an optional > uint64_t. > (elf_symbol::elf_symbol): Likewise. > (elf_symbol::create): Argument crc is now an optional uint64_t > and defaults to absent. > (textually_equals): Simplify comparison. > (elf_symbol::get_crc): Now returns an optional uint64_t. > (elf_symbol::set_crc): Now takes an optional uint64_t. > * src/abg-reader.cc (build_elf_symbol): Treat CRC 0 the same > as other CRC values. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): > Treat CRC 0 the same as other CRC values and also report > changes to CRC presence. > * src/abg-writer.cc (write_elf_symbol): Treat CRC 0 the same > as other CRC values. > * tests/data/Makefile: Remove test-abidiff/test-crc-report.txt > and add test-abidiff/test-crc-report-{0-1,1-0,1-2}.txt. > * tests/data/test-abidiff/test-crc-report-0-1.txt: Report > showing additional of CRCs. > * tests/data/test-abidiff/test-crc-report-1-0.txt: Report > showing removal of CRCs. > * tests/data/test-abidiff/test-crc-report-1-2.txt: Renamed > from tests/data/test-abidiff/test-crc-report.txt. > * tests/test-abidiff.cc: Update test cases that no longer > generate empty reports. > * tests/test-symtab.cc: Update KernelSymtabsWithCRC test. > > Reviewed-by: Matthias Maennich <maennich@google.com> > Signed-off-by: Giuliano Procida <gprocida@google.com> Applied to master, thanks ! [...] Cheers, -- Dodji ^ permalink raw reply [flat|nested] 37+ messages in thread
* [PATCH v5 4/4] add Linux kernel symbol namespace support 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida ` (7 preceding siblings ...) 2022-06-13 14:25 ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida @ 2022-06-13 14:25 ` Giuliano Procida 2022-07-01 12:54 ` Dodji Seketeli 8 siblings, 1 reply; 37+ messages in thread From: Giuliano Procida @ 2022-06-13 14:25 UTC (permalink / raw) To: libabigail; +Cc: dodji, kernel-team, gprocida, maennich Bug 28954 - add Linux Kernel symbol namespace support Each Linux kernel symbol can be exported to a specified named namespace or left in the global (nameless) namespace. One complexity is that the symbol values which identify a string in the __ksymtab_strings section must be interpretated differently for vmlinux and .ko loadable modules as the former has a fixed load address but the latter are relocatable. For vmlinux, the section base address needs to be subtracted to obtain a section-relative offset. The global namespace is explicitly represented as the empty string, at least when it comes to the value of __kstrtabns_FOO symbols, but the common interpretation is that such symbols lack an export namespace. I would rather not have to make use of "empty implies missing" in many places, so the code here represents namespace as optional<string> and only the symtab reader cares about empty strings in __ksymtab_strings. * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. (elf_symbol::create): Add ns argument. (elf_symbol::get_namespace): Declare new function. (elf_symbol::set_namespace): Declare new function. and set_namespace. * src/abg-comp-filter.cc (namespace_changed): Define new helper functions. (categorize_harmful_diff_node): Also call namespace_changed(). * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. (elf_symbol::priv::priv): Add namespace_ to initialisers. (elf_symbol::elf_symbol): Take new ns argument and pass it to priv constructor. (elf_symbol::create): Take new ns argument and pass it to elf_symbol constructor. (elf_symbol::get_namespace): Define new function. (elf_symbol::set_namespace): Define new function. * src/abg-reader.cc (build_elf_symbol): If namespace attribute is present, set symbol namespace. * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If symbol namespaces differ, report this. * src/abg-symtab-reader.cc (symtab::load): Get ELF header to distinguish vmlinux from .ko. Try to get __ksymtab_strings metadata and data. Use these to look up __kstrtabns_FOO namespace entries. Set symbol namespace where found. * src/abg-writer.cc (write_elf_symbol): Emit namespace attribute, if symbol has a namespace. * tests/data/Makefile.am: Add new test files. * tests/data/test-abidiff/test-namespace-0.xml: New test file. * tests/data/test-abidiff/test-namespace-1.xml: Likewise * tests/data/test-abidiff/test-namespace-report.txt: Likewise. * tests/test-abidiff.cc: Add new test case. Reviewed-by: Matthias Maennich <maennich@google.com> Signed-off-by: Giuliano Procida <gprocida@google.com> --- include/abg-ir.h | 8 +++ src/abg-comp-filter.cc | 40 +++++++++++- src/abg-ir.cc | 30 ++++++++- src/abg-reader.cc | 7 ++ src/abg-reporter-priv.cc | 18 ++++++ src/abg-symtab-reader.cc | 64 +++++++++++++++++++ src/abg-writer.cc | 3 + tests/data/Makefile.am | 3 + tests/data/test-abidiff/test-namespace-0.xml | 15 +++++ tests/data/test-abidiff/test-namespace-1.xml | 15 +++++ .../test-abidiff/test-namespace-report.txt | 17 +++++ tests/test-abidiff.cc | 6 ++ 12 files changed, 223 insertions(+), 3 deletions(-) create mode 100644 tests/data/test-abidiff/test-namespace-0.xml create mode 100644 tests/data/test-abidiff/test-namespace-1.xml create mode 100644 tests/data/test-abidiff/test-namespace-report.txt diff --git a/include/abg-ir.h b/include/abg-ir.h index b05a8c6f..7c558828 100644 --- a/include/abg-ir.h +++ b/include/abg-ir.h @@ -922,6 +922,7 @@ private: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); elf_symbol(const elf_symbol&); @@ -947,6 +948,7 @@ public: visibility vi, bool is_in_ksymtab = false, const abg_compat::optional<uint64_t>& crc = {}, + const abg_compat::optional<std::string>& ns = {}, bool is_suppressed = false); const environment* @@ -1024,6 +1026,12 @@ public: void set_crc(const abg_compat::optional<uint64_t>& crc); + const abg_compat::optional<std::string>& + get_namespace() const; + + void + set_namespace(const abg_compat::optional<std::string>& ns); + bool is_suppressed() const; diff --git a/src/abg-comp-filter.cc b/src/abg-comp-filter.cc index 31590284..22da5244 100644 --- a/src/abg-comp-filter.cc +++ b/src/abg-comp-filter.cc @@ -254,6 +254,43 @@ crc_changed(const diff* diff) return false; } +/// Test if there was a function or variable namespace change. +/// +/// @param f the first function or variable to consider. +/// +/// @param s the second function or variable to consider. +/// +/// @return true if the test is positive, false otherwise. +template <typename function_or_var_decl_sptr> +static bool +namespace_changed(const function_or_var_decl_sptr& f, + const function_or_var_decl_sptr& s) +{ + const auto& symbol_f = f->get_symbol(); + const auto& symbol_s = s->get_symbol(); + if (!symbol_f || !symbol_s) + return false; + return symbol_f->get_namespace() != symbol_s->get_namespace(); +} + +/// Test if the current diff tree node carries a namespace change in +/// either a function or a variable. +/// +/// @param diff the diff tree node to consider. +/// +/// @return true if the test is positive, false otherwise. +static bool +namespace_changed(const diff* diff) +{ + if (const function_decl_diff* d = + dynamic_cast<const function_decl_diff*>(diff)) + return namespace_changed(d->first_function_decl(), + d->second_function_decl()); + if (const var_diff* d = dynamic_cast<const var_diff*>(diff)) + return namespace_changed(d->first_var(), d->second_var()); + return false; +} + /// Test if there was a function name change, but there there was no /// change in name of the underlying symbol. IOW, if the name of a /// function changed, but the symbol of the new function is equal to @@ -1781,7 +1818,8 @@ categorize_harmful_diff_node(diff *d, bool pre) || non_static_data_member_added_or_removed(d) || base_classes_added_or_removed(d) || has_harmful_enum_change(d) - || crc_changed(d))) + || crc_changed(d) + || namespace_changed(d))) category |= SIZE_OR_OFFSET_CHANGE_CATEGORY; if (has_virtual_mem_fn_change(d)) diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 34ae486f..a271bfde 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -1728,6 +1728,7 @@ struct elf_symbol::priv bool is_common_; bool is_in_ksymtab_; abg_compat::optional<uint64_t> crc_; + abg_compat::optional<std::string> namespace_; bool is_suppressed_; elf_symbol_wptr main_symbol_; elf_symbol_wptr next_alias_; @@ -1745,6 +1746,7 @@ struct elf_symbol::priv is_common_(false), is_in_ksymtab_(false), crc_(), + namespace_(), is_suppressed_(false) {} @@ -1760,6 +1762,7 @@ struct elf_symbol::priv elf_symbol::visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : env_(e), index_(i), @@ -1773,6 +1776,7 @@ struct elf_symbol::priv is_common_(c), is_in_ksymtab_(is_in_ksymtab), crc_(crc), + namespace_(ns), is_suppressed_(is_suppressed) { if (!is_common_) @@ -1818,6 +1822,8 @@ elf_symbol::elf_symbol() /// @param vi the visibility of the symbol. /// /// @param crc the CRC (modversions) value of Linux Kernel symbols +/// +/// @param ns the namespace of Linux Kernel symbols, if any elf_symbol::elf_symbol(const environment* e, size_t i, size_t s, @@ -1830,6 +1836,7 @@ elf_symbol::elf_symbol(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) : priv_(new priv(e, i, @@ -1843,6 +1850,7 @@ elf_symbol::elf_symbol(const environment* e, vi, is_in_ksymtab, crc, + ns, is_suppressed)) {} @@ -1886,6 +1894,8 @@ elf_symbol::create() /// /// @param crc the CRC (modversions) value of Linux Kernel symbols /// +/// @param ns the namespace of Linux Kernel symbols, if any +/// /// @return a (smart) pointer to a newly created instance of @ref /// elf_symbol. elf_symbol_sptr @@ -1901,10 +1911,11 @@ elf_symbol::create(const environment* e, visibility vi, bool is_in_ksymtab, const abg_compat::optional<uint64_t>& crc, + const abg_compat::optional<std::string>& ns, bool is_suppressed) { elf_symbol_sptr sym(new elf_symbol(e, i, s, n, t, b, d, c, ve, vi, - is_in_ksymtab, crc, is_suppressed)); + is_in_ksymtab, crc, ns, is_suppressed)); sym->priv_->main_symbol_ = sym; return sym; } @@ -1926,7 +1937,8 @@ textually_equals(const elf_symbol&l, && l.is_defined() == r.is_defined() && l.is_common_symbol() == r.is_common_symbol() && l.get_version() == r.get_version() - && l.get_crc() == r.get_crc()); + && l.get_crc() == r.get_crc() + && l.get_namespace() == r.get_namespace()); if (equals && l.is_variable()) // These are variable symbols. Let's compare their symbol size. @@ -2146,6 +2158,20 @@ void elf_symbol::set_crc(const abg_compat::optional<uint64_t>& crc) {priv_->crc_ = crc;} +/// Getter of the 'namespace' property. +/// +/// @return the namespace for Linux Kernel symbols, if any +const abg_compat::optional<std::string>& +elf_symbol::get_namespace() const +{return priv_->namespace_;} + +/// Setter of the 'namespace' property. +/// +/// @param ns the new namespace for Linux Kernel symbols, if any +void +elf_symbol::set_namespace(const abg_compat::optional<std::string>& ns) +{priv_->namespace_ = ns;} + /// Getter for the 'is-suppressed' property. /// /// @return true iff the current symbol has been suppressed by a diff --git a/src/abg-reader.cc b/src/abg-reader.cc index 7a9ad1f9..f9e420f1 100644 --- a/src/abg-reader.cc +++ b/src/abg-reader.cc @@ -3265,6 +3265,13 @@ build_elf_symbol(read_context& ctxt, const xmlNodePtr node, if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "crc")) e->set_crc(strtoull(CHAR_STR(s), NULL, 0)); + if (xml_char_sptr s = XML_NODE_GET_ATTRIBUTE(node, "namespace")) + { + std::string ns; + xml::xml_char_sptr_to_string(s, ns); + e->set_namespace(ns); + } + return e; } diff --git a/src/abg-reporter-priv.cc b/src/abg-reporter-priv.cc index d000620b..fcfc9dce 100644 --- a/src/abg-reporter-priv.cc +++ b/src/abg-reporter-priv.cc @@ -1168,6 +1168,24 @@ maybe_report_diff_for_symbol(const elf_symbol_sptr& symbol1, out << std::noshowbase << std::dec << "\n"; } + + const abg_compat::optional<std::string>& ns1 = symbol1->get_namespace(); + const abg_compat::optional<std::string>& ns2 = symbol2->get_namespace(); + if (ns1 != ns2) + { + const std::string none = "(none)"; + out << indent << "namespace changed from "; + if (ns1.has_value()) + out << "'" << ns1.value() << "'"; + else + out << none; + out << " to "; + if (ns2.has_value()) + out << "'" << ns2.value() << "'"; + else + out << none; + out << "\n"; + } } /// For a given symbol, emit a string made of its name and version. diff --git a/src/abg-symtab-reader.cc b/src/abg-symtab-reader.cc index 0c2f8ccc..e20ee2a7 100644 --- a/src/abg-symtab-reader.cc +++ b/src/abg-symtab-reader.cc @@ -204,6 +204,13 @@ symtab::load_(Elf* elf_handle, ir::environment* env, symbol_predicate is_suppressed) { + GElf_Ehdr ehdr_mem; + GElf_Ehdr* header = gelf_getehdr(elf_handle, &ehdr_mem); + if (!header) + { + std::cerr << "Could not get ELF header: Skipping symtab load.\n"; + return false; + } Elf_Scn* symtab_section = elf_helpers::find_symbol_table_section(elf_handle); if (!symtab_section) @@ -232,9 +239,34 @@ symtab::load_(Elf* elf_handle, return false; } + // The __kstrtab_strings sections is basically an ELF strtab but does not + // support elf_strptr lookups. A single call to elf_getdata gives a handle to + // washed section data. + // + // The value of a __kstrtabns_FOO (or other similar) symbol is an address + // within the __kstrtab_strings section. To look up the string value, we need + // to translate from vmlinux load address to section offset by subtracting the + // base address of the section. This adjustment is not needed for loadable + // modules which are relocatable and so identifiable by ELF type ET_REL. + Elf_Scn* strings_section = elf_helpers::find_ksymtab_strings_section(elf_handle); + size_t strings_offset = 0; + const char* strings_data = nullptr; + size_t strings_size = 0; + if (strings_section) + { + GElf_Shdr strings_sheader; + gelf_getshdr(strings_section, &strings_sheader); + strings_offset = header->e_type == ET_REL ? 0 : strings_sheader.sh_addr; + Elf_Data* data = elf_getdata(strings_section, nullptr); + ABG_ASSERT(data->d_off == 0); + strings_data = reinterpret_cast<const char *>(data->d_buf); + strings_size = data->d_size; + } + const bool is_kernel = elf_helpers::is_linux_kernel(elf_handle); std::unordered_set<std::string> exported_kernel_symbols; std::unordered_map<std::string, uint64_t> crc_values; + std::unordered_map<std::string, std::string> namespaces; for (size_t i = 0; i < number_syms; ++i) { @@ -285,6 +317,27 @@ symtab::load_(Elf* elf_handle, ABG_ASSERT(crc_values.emplace(name.substr(6), sym->st_value).second); continue; } + if (strings_section && is_kernel && name.rfind("__kstrtabns_", 0) == 0) + { + // This symbol lives in the __ksymtab_strings section but st_value may + // be a vmlinux load address so we need to subtract the offset before + // looking it up in that section. + const size_t value = sym->st_value; + const size_t offset = value - strings_offset; + // check offset + ABG_ASSERT(offset < strings_size); + // find the terminating NULL + const char* first = strings_data + offset; + const char* last = strings_data + strings_size; + const char* limit = std::find(first, last, 0); + // check NULL found + ABG_ASSERT(limit < last); + // interpret the empty namespace name as no namespace name + if (first < limit) + ABG_ASSERT(namespaces.emplace( + name.substr(12), std::string(first, limit - first)).second); + continue; + } // filter out uninteresting entries and only keep functions/variables for // now. The rest might be interesting in the future though. @@ -374,6 +427,17 @@ symtab::load_(Elf* elf_handle, symbol->set_crc(crc_entry.second); } + // Now add the namespaces + for (const auto& namespace_entry : namespaces) + { + const auto r = name_symbol_map_.find(namespace_entry.first); + if (r == name_symbol_map_.end()) + continue; + + for (const auto& symbol : r->second) + symbol->set_namespace(namespace_entry.second); + } + // sort the symbols for deterministic output std::sort(symbols_.begin(), symbols_.end(), symbol_sort); diff --git a/src/abg-writer.cc b/src/abg-writer.cc index 23598ef0..90db34be 100644 --- a/src/abg-writer.cc +++ b/src/abg-writer.cc @@ -3075,6 +3075,9 @@ write_elf_symbol(const elf_symbol_sptr& sym, << std::hex << std::showbase << sym->get_crc().value() << std::dec << std::noshowbase << "'"; + if (sym->get_namespace().has_value()) + o << " namespace='" << sym->get_namespace().value() << "'"; + o << "/>\n"; return true; diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am index be342684..c3112075 100644 --- a/tests/data/Makefile.am +++ b/tests/data/Makefile.am @@ -93,6 +93,9 @@ test-abidiff/test-crc-2.xml \ test-abidiff/test-crc-report-0-1.txt \ test-abidiff/test-crc-report-1-0.txt \ test-abidiff/test-crc-report-1-2.txt \ +test-abidiff/test-namespace-0.xml \ +test-abidiff/test-namespace-1.xml \ +test-abidiff/test-namespace-report.txt \ test-abidiff/test-PR27985-report.txt \ test-abidiff/test-PR27985-v0.c \ test-abidiff/test-PR27985-v0.o \ diff --git a/tests/data/test-abidiff/test-namespace-0.xml b/tests/data/test-abidiff/test-namespace-0.xml new file mode 100644 index 00000000..5a9f5cd2 --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-0.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-1.xml b/tests/data/test-abidiff/test-namespace-1.xml new file mode 100644 index 00000000..9814844b --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-1.xml @@ -0,0 +1,15 @@ +<abi-corpus path='test.o' architecture='elf-amd-x86_64'> + <elf-variable-symbols> + <elf-symbol name='v1' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='this'/> + <elf-symbol name='v2' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace='that'/> + <elf-symbol name='v3' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes' namespace=''/> + <elf-symbol name='v4' size='4' type='object-type' binding='global-binding' visibility='default-visibility' is-defined='yes'/> + </elf-variable-symbols> + <abi-instr version='1.0' address-size='64' path='test.c' comp-dir-path='/tmp' language='LANG_C89'> + <type-decl name='int' size-in-bits='32' id='type-id-1'/> + <var-decl name='v1' type-id='type-id-1' mangled-name='v1' visibility='default' filepath='test.c' line='1' column='1' elf-symbol-id='v1'/> + <var-decl name='v2' type-id='type-id-1' mangled-name='v2' visibility='default' filepath='test.c' line='2' column='1' elf-symbol-id='v2'/> + <var-decl name='v3' type-id='type-id-1' mangled-name='v3' visibility='default' filepath='test.c' line='3' column='1' elf-symbol-id='v3'/> + <var-decl name='v4' type-id='type-id-1' mangled-name='v4' visibility='default' filepath='test.c' line='4' column='1' elf-symbol-id='v4'/> + </abi-instr> +</abi-corpus> diff --git a/tests/data/test-abidiff/test-namespace-report.txt b/tests/data/test-abidiff/test-namespace-report.txt new file mode 100644 index 00000000..d2c421ed --- /dev/null +++ b/tests/data/test-abidiff/test-namespace-report.txt @@ -0,0 +1,17 @@ +Functions changes summary: 0 Removed, 0 Changed, 0 Added function +Variables changes summary: 0 Removed, 4 Changed, 0 Added variables + +4 Changed variables: + + [C] 'int v1' was changed: + namespace changed from (none) to 'this' + + [C] 'int v2' was changed: + namespace changed from 'this' to 'that' + + [C] 'int v3' was changed: + namespace changed from 'that' to '' + + [C] 'int v4' was changed: + namespace changed from '' to (none) + diff --git a/tests/test-abidiff.cc b/tests/test-abidiff.cc index a02bbe3d..93e44d6a 100644 --- a/tests/test-abidiff.cc +++ b/tests/test-abidiff.cc @@ -128,6 +128,12 @@ static InOutSpec specs[] = "data/test-abidiff/test-crc-report-1-2.txt", "output/test-abidiff/test-crc-report-1-2.txt" }, + { + "data/test-abidiff/test-namespace-0.xml", + "data/test-abidiff/test-namespace-1.xml", + "data/test-abidiff/test-namespace-report.txt", + "output/test-abidiff/test-namespace-report-0-1.txt" + }, { "data/test-abidiff/test-PR27616-v0.xml", "data/test-abidiff/test-PR27616-v1.xml", -- 2.36.1.476.g0c4daa206d-goog ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [PATCH v5 4/4] add Linux kernel symbol namespace support 2022-06-13 14:25 ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida @ 2022-07-01 12:54 ` Dodji Seketeli 0 siblings, 0 replies; 37+ messages in thread From: Dodji Seketeli @ 2022-07-01 12:54 UTC (permalink / raw) To: Giuliano Procida; +Cc: libabigail, kernel-team, maennich Hello, Giuliano Procida <gprocida@google.com> a écrit: > Bug 28954 - add Linux Kernel symbol namespace support > > Each Linux kernel symbol can be exported to a specified named > namespace or left in the global (nameless) namespace. > > One complexity is that the symbol values which identify a string in > the __ksymtab_strings section must be interpretated differently for > vmlinux and .ko loadable modules as the former has a fixed load > address but the latter are relocatable. For vmlinux, the section base > address needs to be subtracted to obtain a section-relative offset. > > The global namespace is explicitly represented as the empty string, at > least when it comes to the value of __kstrtabns_FOO symbols, but the > common interpretation is that such symbols lack an export namespace. > > I would rather not have to make use of "empty implies missing" in many > places, so the code here represents namespace as optional<string> and > only the symtab reader cares about empty strings in __ksymtab_strings. > > * include/abg-ir.h (elf_symbol::elf_symbol): Add ns argument. > (elf_symbol::create): Add ns argument. > (elf_symbol::get_namespace): Declare new function. > (elf_symbol::set_namespace): Declare new function. > and set_namespace. > * src/abg-comp-filter.cc (namespace_changed): Define new > helper functions. > (categorize_harmful_diff_node): Also call namespace_changed(). > * src/abg-ir.cc (elf_symbol::priv): Add namespace_ member. > (elf_symbol::priv::priv): Add namespace_ to initialisers. > (elf_symbol::elf_symbol): Take new ns argument and pass it to > priv constructor. > (elf_symbol::create): Take new ns argument and pass it to > elf_symbol constructor. > (elf_symbol::get_namespace): Define new function. > (elf_symbol::set_namespace): Define new function. > * src/abg-reader.cc (build_elf_symbol): If namespace > attribute is present, set symbol namespace. > * src/abg-reporter-priv.cc (maybe_report_diff_for_symbol): If > symbol namespaces differ, report this. > * src/abg-symtab-reader.cc (symtab::load): Get ELF header to > distinguish vmlinux from .ko. Try to get __ksymtab_strings > metadata and data. Use these to look up __kstrtabns_FOO > namespace entries. Set symbol namespace where found. > * src/abg-writer.cc (write_elf_symbol): Emit namespace > attribute, if symbol has a namespace. > * tests/data/Makefile.am: Add new test files. > * tests/data/test-abidiff/test-namespace-0.xml: New test file. > * tests/data/test-abidiff/test-namespace-1.xml: Likewise > * tests/data/test-abidiff/test-namespace-report.txt: Likewise. > * tests/test-abidiff.cc: Add new test case. > > Reviewed-by: Matthias Maennich <maennich@google.com> > Signed-off-by: Giuliano Procida <gprocida@google.com> Applied to master, thanks! [...] Cheers, -- Dodji ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2022-07-01 12:54 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-14 18:13 [PATCH 0/2] Bug 28954 - add Linux Kernel symbol namespace support Giuliano Procida 2022-03-14 18:13 ` [PATCH 1/2] optional: add operator== and operator!= Giuliano Procida 2022-03-15 11:02 ` Matthias Maennich 2022-03-16 9:31 ` Giuliano Procida 2022-03-14 18:13 ` [PATCH 2/2] add Linux kernel symbol namespace support Giuliano Procida 2022-03-15 11:25 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-16 16:30 ` [PATCH v2 1/4] optional: minor improvements Giuliano Procida 2022-03-17 10:56 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 2/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-03-17 11:01 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 3/4] add Linux kernel symbol namespace support Giuliano Procida 2022-03-17 11:57 ` Matthias Maennich 2022-03-16 16:30 ` [PATCH v2 4/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-03-17 12:01 ` Matthias Maennich 2022-03-17 16:38 ` [PATCH v3 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 2/4] optional: minor improvements Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-03-17 16:38 ` [PATCH v3 4/4] add Linux kernel symbol namespace support Giuliano Procida 2022-03-21 12:53 ` Matthias Maennich 2022-03-21 15:52 ` Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 2/4] optional: minor improvements Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-03-21 16:02 ` [PATCH v4 4/4] add Linux kernel symbol namespace support Giuliano Procida 2022-06-13 14:25 ` [PATCH v5 0/4] add symbol namespace support, update symbol CRC support Giuliano Procida 2022-06-30 16:39 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 1/4] crc_changed: eliminate copying of shared_ptr values Giuliano Procida 2022-06-30 16:40 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 2/4] optional: minor improvements Giuliano Procida 2022-06-30 16:40 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 3/4] Linux symbol CRCs: support 0 and report presence changes Giuliano Procida 2022-06-30 16:41 ` Dodji Seketeli 2022-06-13 14:25 ` [PATCH v5 4/4] add Linux kernel symbol namespace support Giuliano Procida 2022-07-01 12:54 ` Dodji Seketeli
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).