public inbox for libabigail@sourceware.org
 help / color / mirror / Atom feed
From: Dodji Seketeli <dodji@redhat.com>
To: libabigail@sourceware.org
Subject: [PATCH, applied] ir: Don't overdo canonical type propagation control when comparing classes
Date: Mon, 05 Sep 2022 18:10:23 +0200	[thread overview]
Message-ID: <87ilm1kdjk.fsf@redhat.com> (raw)

Hello,

While looking at something else, I stumbled upon this problem.

When comparing a class, equals first calls the overload for
class_or_union to compare data members.  If we are in the process of
type canonicalization, the right hand side operand might be
"canonical-type-propagated", during that call to the overload.  In
other words, it can inherit the canonical type of the left-hand-side
operand.  The problem is that that canonical type propagation, if it
happens, is too early because this equals function still needs to
compare other things like virtual member functions etc.  So, the
original intend of the code was to erase the canonical type that might
have been propagated.  This is all and well.

The problem however is that the code /always/ erases the canonical
type of the right hand side operand, even if it was the result of the
propagation optimization long before it entered this equals function.
Oops.

This patch fixes that issue.

	* src/abg-ir.cc (equals): In the overload for const class_decl&,
	do not cancel the propagated canonical type if the propagation is
	not the result of invoking the equals overload for class_or_union.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>
Applied to master.

---
 src/abg-ir.cc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/abg-ir.cc b/src/abg-ir.cc
index a2eb8a02..7c8b6640 100644
--- a/src/abg-ir.cc
+++ b/src/abg-ir.cc
@@ -23587,6 +23587,7 @@ equals(const class_decl& l, const class_decl& r, change_kind* k)
   RETURN_TRUE_IF_COMPARISON_CYCLE_DETECTED(static_cast<const class_or_union&>(l),
 					   static_cast<const class_or_union&>(r));
 
+  bool had_canonical_type = !!r.get_naked_canonical_type();
   bool result = true;
   if (!equals(static_cast<const class_or_union&>(l),
 	      static_cast<const class_or_union&>(r),
@@ -23609,7 +23610,8 @@ equals(const class_decl& l, const class_decl& r, change_kind* k)
   // canonical type propagation, then cancel that because it's too
   // early to do that at this point.  We still need to compare bases
   // virtual members.
-  maybe_cancel_propagated_canonical_type(r);
+  if (!had_canonical_type)
+    maybe_cancel_propagated_canonical_type(r);
 
   // Compare bases.
     if (l.get_base_specifiers().size() != r.get_base_specifiers().size())
-- 
2.37.2


-- 
		Dodji


                 reply	other threads:[~2022-09-05 16:10 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87ilm1kdjk.fsf@redhat.com \
    --to=dodji@redhat.com \
    --cc=libabigail@sourceware.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).