Re: 'src/abg-dwarf-reader.cc:compare_dies_string_attribute_value' optimization

[Ben Woodard <woodard@redhat.com>]

So I ran my standard test on this branch as of 3d277a9cc05873cf4aeb97273d585e0b07af917d

    37 Consistency mismatches
    /lib64/libabigail.so.0.0.0 from libabigail-2.0-2.fc36.x86_64
    /lib64/libadwaitaqtpriv.so.1.4.1 from libadwaita-qt5-1.4.1-3.fc36.x86_64
    /lib64/libaspell.so.15.3.1 from aspell-0.60.8-9.fc36.x86_64
    /lib64/libboost_log.so.1.76.0 from boost-log-1.76.0-9.fc36.x86_64
    /lib64/libclucene-core.so.2.3.3.4 from clucene-core-2.3.3.4-42.20130812.e8e3d20git.fc36.x86_64
    /lib64/libdap.so.27.0.5 from libdap-3.20.9-2.fc36.x86_64
    /lib64/libdcerpc-samr.so.0.0.1 from samba-libs-4.16.0-6.fc36.x86_64
    /lib64/libdjvulibre.so.21.7.0 from djvulibre-libs-3.5.28-2.fc36.x86_64
    /lib64/dovecot/libdovecot-storage.so.0.0.0 from dovecot-2.3.18-1.fc36.x86_64
    /lib64/libexiv2.so.0.27.5 from exiv2-libs-0.27.5-2.fc36.x86_64
    /lib64/libgdal.so.30.0.2 from gdal-libs-3.4.2-1.fc36.x86_64
    /lib64/libgeos.so.3.10.2 from geos-3.10.2-4.fc36.x86_64
    /lib64/libglibmm-2.4.so.1.3.0 from glibmm24-2.66.2-2.fc36.x86_64
    /lib64/mozilla/plugins/gmp-gmpopenh264/system-installed/libgmpopenh264.so.2.1.1 from mozilla-openh264-2.1.1-3.fc36.x86_64
    /lib64/libhdf5_cpp.so.200.1.0 from hdf5-1.12.1-5.fc36.x86_64
    /lib64/libicui18n.so.67.1 from libicu67-67.1-2.fc35.x86_64
    /lib64/libicui18n.so.69.1 from libicu-69.1-5.fc36.x86_64
    /lib64/libicuuc.so.67.1 from libicu67-67.1-2.fc35.x86_64
    /lib64/libicuuc.so.69.1 from libicu-69.1-5.fc36.x86_64
    /lib64/dyninst/libinstructionAPI.so.12.0.1 from dyninst-12.0.1-3.fc36.x86_64
    /lib64/libjavascriptcoregtk-4.0.so.18.20.4 from webkit2gtk3-jsc-2.36.0-1.fc36.x86_64
    /lib64/libjxl.so.0.6.1 from libjxl-0.6.1-9.fc36.x86_64
    /lib64/libkmldom.so.1.3.0 from libkml-1.3.0-37.fc36.x86_64
    /lib64/libmusicbrainz5.so.1.0.0 from libmusicbrainz5-5.1.0-19.fc36.x86_64
    /lib64/libOpenEXRUtil-3_1.so.30.4.1 from openexr-libs-3.1.4-1.fc36.x86_64
    /lib64/libopenh264.so.2.1.1 from openh264-2.1.1-3.fc36.x86_64
    /lib64/libOSMesa.so.8.0.0 from mesa-libOSMesa-22.0.1-1.fc36.x86_64
    /lib64/libproj.so.22.2.1 from proj-8.2.1-6.fc36.x86_64
    /lib64/libQt5WaylandClient.so.5.15.3 from qt5-qtwayland-5.15.3-1.fc36.x86_64
    /lib64/libQt5WaylandCompositor.so.5.15.3 from qt5-qtwayland-5.15.3-1.fc36.x86_64
    /lib64/libQt5XmlPatterns.so.5.15.3 from qt5-qtxmlpatterns-5.15.3-1.fc36.x86_64
    /lib64/libSDL2_image-2.0.so.0.2.3 from SDL2_image-2.0.5-8.fc36.x86_64
    /lib64/libstdc++.so.6.0.30 from libstdc++-12.0.1-0.14.fc36.x86_64
    /lib64/libtag.so.1.18.0 from taglib-1.12-6.fc36.x86_64
    /lib64/libreoffice/program/libuno_cppuhelpergcc3.so.3 from libreoffice-ure-7.3.2.2-1.fc36.x86_64
    /lib64/libvtkRenderingCore.so.9.1.0 from vtk-9.1.0-6.fc36.x86_64
    /lib64/libwebrtc_audio_processing.so.1.0.0 from webrtc-audio-processing-0.3.1-8.fc36.x86_64

There also appears to be some sort of performance regression. I know that these are your problem children but 
/lib64/libmozjs-68.so.0.0.0 
/lib64/libwebkit2gtk-4.0.so.37.56.4 
/lib64/libgs.so.9.55 
/lib64/libperl.so.5.34.1 
have been running solidly for more than 3 hours. I’ll let them continue all night and see if they complete or if they are in some sort of infinite loop.

The asserts that I mentioned to you privately a few days ago have been cleared up. Thank you.

One of the most common problems in that set of consistency mismatches above seems to be with C++ destructors. Here are a couple of examples:
$ abidw --abidiff /lib64/libabigail.so.0.0.0
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Functions changes summary: 0 Removed, 2 Changed, 0 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

2 functions with some indirect sub-type change:

  [C] 'method virtual abigail::comparison::diff_node_visitor::~diff_node_visitor()' at abg-comparison.cc:11192:1 has some indirect sub-type changes:
    parameter 0 of type 'abigail::comparison::diff_node_visitor* const' was removed

  [C] 'method virtual abigail::ir::ir_node_visitor::~ir_node_visitor()' at abg-ir.cc:25400:1 has some indirect sub-type changes:
    parameter 0 of type 'abigail::ir::ir_node_visitor* const' was removed

abidw --abidiff /lib64/libaspell.so.15.3.1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Functions changes summary: 0 Removed, 2 Changed, 0 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

2 functions with some indirect sub-type change:

  [C] 'method virtual acommon::CanHaveError::~CanHaveError()' at can_have_error.cpp:16:1 has some indirect sub-type changes:
    parameter 0 of type 'acommon::CanHaveError* const' was removed

  [C] 'method virtual aspeller::Dictionary::~Dictionary()' at data.cpp:80:1 has some indirect sub-type changes:
    parameter 0 of type 'aspeller::Dictionary* const' was removed


That accounts for 28 of the 37 consistency mismatches. So that is probably your highest value target to fix before moving on.
There really isn’t a pattern that I can see in the remaining failing cases. Three that you should really focus on are:
abidw --abidiff /lib64/libmozjs-78.so.0.0.0 which is an assert()
#5  0x00007ffff7550596 in __GI___assert_fail (assertion=0x7ffff7f2d1a3 "__abg_cond__", file=0x7ffff7f2d5c0 "../../../libabigail/src/abg-ir.cc", line=25414, function=0x7ffff7f34b08 "size_t abigail::ir::hash_as_canonical_type_or_constant(const type_base*)") at assert.c:101

and 

abidw --abidiff /lib64/libOSMesa.so.8.0.0 which really goes off the rails somewhere.  The indenting goes right off the right side of my jumbo terminal window. That just ain’t right.

Then there is the perpetual problem child libstdc++

So keep working on this patch set, let’s get these cleared up and get this puppy committed.

-ben



> On Apr 12, 2022, at 9:16 AM, Dodji Seketeli <dodji@seketeli.org> wrote:
> 
> Giuliano Procida <gprocida@google.com> a écrit:
> 
>> I started looking at this today but I've run out of time. I'll be able to
>> take another look in about a week.
> 
> Thanks a lot.
> 
>> Checks fail (various types are renumbered in XML output) starting at
>> commit 57b1c714.
> 
> Yeah, that is unfortunately expected because ...
> 
>> I haven't tried this on kernel or framework libraries yet, but I know there
>> are plenty of typedefs in both.
> 
> ... of this exactly.
> 
> This typedef change has been on my radar for a long time.  The fact that
> a type Foo would use a typedef T in one version and a typedef T' in
> another once, where T and T' have the same underlying type would result
> in the two instances of Foo to be different can wreak havoc on the self
> check tests, where in reality the two types are not different from an
> ABI standpoint.
> 
> I am hoping that this is one of those changes that will get us closer
> to having more stable abixml output.
> 
> Please note however that you need the entire stack of changes to have
> everything working "as expected", as far as I can tell.
> 
> Thank you for looking into this.
> 
> -- 
> 		Dodji
>