public inbox for libabigail@sourceware.org
 help / color / mirror / Atom feed
* [PATCH 00/27] Implement type hashing & fix self-comparing gcc-gnat in fc37
@ 2024-08-29 15:46 dodji
  2024-08-29 15:52 ` [PATCH 01/27] Implement type hashing Dodji Seketeli
                   ` (26 more replies)
  0 siblings, 27 replies; 28+ messages in thread
From: dodji @ 2024-08-29 15:46 UTC (permalink / raw)
  To: libabigail; +Cc: Dodji Seketeli

From: Dodji Seketeli <dodji@redhat.com>

This set of patches implements type hashing at the libabigail IR level
before doing type canonicalization.  The goal of this scheme is to
hopefully speed-up the type canonicalization process and also to make
it less error prone than what it is today.  I believe that the main
source of error of the type canonicalization process is the use of the
"canonical type propagation" optimization.  There has been a number of
bugs throughout the years that are related to that optimization and I
believe experience has proven that it becomes increasingly difficult
to maintain.

Hashing types allows the type canonicalization algorithm to quickly
determine when two types are different, as they'd have different hash
values.  When the hash values are equal however, structural comparison
is still needed.  But that amount of structural comparison can be
drastically reduced.

Once type hashing is implemented and it becomes much easier (and
precise) to determine that two types are different during type
canonicalization, I encountered a number of fixes, while trying to
make to the gcc-gnat package to self-compare.  The work that was laid
down also helped tackle the self-comparison of the libLLVM.so binary
as well as many other binaries.

In the end, the 27 patches allow libabigail to be able to self-compare
a lot of binaries than previously.  Among the 42 packages that were
failing self-comparison in fc36 at
https://sourceware.org/bugzilla/show_bug.cgi?id=29413#c5, only 5 still
fail today.  These are binutils, infinipath-psm, monodoc, OpenMolcas
and qt6-qtbase.  I'll probably try to fix self-comparison for these
packages in subsequent fixes now that fundamental things are fixed by
this patch series.

Of course, speed of processing has improved a lot for some packages,
going from near one hour to a matter of minutes in some instances.

Dodji Seketeli (27):
  Implement type hashing
  ir: Remove the now useless type propagation optimization
  ir: decl-only classes don't equal fully defined classes under ODR
  comp-filter: Consider lvalue-ness changes on references as harmful
  comp-filter: Ignore ptr size when detecting void ptr to ptr change
  Don't strip typedefs in fn names when pretty-printing and comparing
  reader: Avoid duplicating member types
  ir: Cache the result of scope_decl::get_sorted_member_types
  dwarf-reader: Avoid duplicating anonymous member types
  reader: Avoid duplicating recursive types
  dwarf-reader: Speed-up decl-only resolution
  ir: Fix name setting of a ptr-to-mbr-type
  dwarf-reader: Support LLVM's lingo of declaration-ness
  reader: Improve logging in the ABIXML reader
  ir: Improve type logging during type canonicalization
  tools-utils: Improve logging while reading a Linux kernel
  dwarf-reader: Fix building of void, void* and variadic parm types
  {dwarf,btf,ctf}-reader: Set the origin of the corpus group
  {btf,ctf,dwarf}-reader, ir: Fix self-comparison debugging for corpus groups
  abilint: Support --verbose option
  big-tests: Update git sub-module
  dwarf-reader,tools-utils: Add statistics about built/suppressed functions
  abidw: Add a --kmi-stablelist option alongside existing --kmi-whitelist
  dwarf-reader,ir: Fix endless loop while analyzing DWARF from Modula-2
  abipkgdiff: Extract devel and main packages in the same directory
  dwarf-reader,reader.cc: Fix function virtuality setting
  dwarf-reader,ir,writer: Better support for static member variables

 big-tests                                     |     2 +-
 configure.ac                                  |    27 +-
 doc/manuals/abidw.rst                         |    16 +-
 doc/manuals/abilint.rst                       |     4 +
 include/abg-comp-filter.h                     |    12 +
 include/abg-comparison.h                      |    35 +-
 include/abg-fwd.h                             |    28 +-
 include/abg-hash.h                            |   249 +-
 include/abg-ir.h                              |   457 +-
 src/abg-btf-reader.cc                         |    46 +-
 src/abg-comp-filter.cc                        |   245 +-
 src/abg-comparison-priv.h                     |     2 +-
 src/abg-comparison.cc                         |    73 +-
 src/abg-ctf-reader.cc                         |    58 +-
 src/abg-dwarf-reader.cc                       |  2063 +-
 src/abg-hash.cc                               |  1478 +-
 src/abg-ir-priv.h                             |  1329 +-
 src/abg-ir.cc                                 |  1602 +-
 src/abg-leaf-reporter.cc                      |    24 +-
 src/abg-reader.cc                             |   401 +-
 src/abg-tools-utils.cc                        |    24 +-
 src/abg-writer.cc                             |   457 +-
 tests/data/Makefile.am                        |     4 +
 .../libsdl/libsdl-1.2.60-1.2.64-report.txt    |    12 +-
 .../test-ptr-to-mbr2-output-1.txt             |     1 -
 .../qualifier-typedef-array-report-1.txt      |    64 +-
 .../test-ld-2.28-210.so--ld-2.28-211.so.txt   |     2 +-
 .../test-abidiff/test-PR18791-report0.txt     |    21 +
 .../PR29443-missing-xx.o.annotated.abi        |     8 +-
 tests/data/test-annotate/libtest23.so.abi     |   338 +-
 .../test-annotate/libtest24-drop-fns-2.so.abi |   377 +-
 .../test-annotate/libtest24-drop-fns.so.abi   |   377 +-
 .../test-anonymous-members-0.o.abi            |    34 +-
 .../test-pointer-to-member-1.o.annotated.abi  |    34 +-
 tests/data/test-annotate/test0.abi            |    46 +-
 tests/data/test-annotate/test1.abi            |    44 +-
 .../data/test-annotate/test13-pr18894.so.abi  |  1264 +-
 .../data/test-annotate/test14-pr18893.so.abi  |  3435 +-
 .../data/test-annotate/test15-pr18892.so.abi  | 18646 +++-----
 .../data/test-annotate/test17-pr19027.so.abi  |  9370 ++--
 ...st18-pr19037-libvtkRenderingLIC-6.1.so.abi | 10482 +++--
 ...19-pr19023-libtcmalloc_and_profiler.so.abi |  9933 ++--
 tests/data/test-annotate/test2.so.abi         |    30 +-
 ...st20-pr19025-libvtkParallelCore-6.1.so.abi |  9108 ++--
 .../data/test-annotate/test21-pr19092.so.abi  |  3848 +-
 tests/data/test-annotate/test3.so.abi         |     4 +-
 tests/data/test-annotate/test4.so.abi         |    18 +-
 tests/data/test-annotate/test5.o.abi          |     4 +-
 tests/data/test-annotate/test6.so.abi         |    18 +-
 tests/data/test-annotate/test7.so.abi         |    52 +-
 .../test8-qualified-this-pointer.so.abi       |    16 +-
 .../PR25409-librte_bus_dpaa.so.20.0.abi       |  2796 +-
 .../test0-pr19026-libvtkIOSQL-6.1.so.1.abi    |  7781 ++--
 .../PR25058-liblttng-ctl-report-1.txt         |    15 +-
 .../test28-vtable-changes-report-0.txt        |     3 +
 tests/data/test-diff-dwarf/test4-report.txt   |     6 +-
 .../test42-PR21296-clanggcc-report0.txt       |     6 +-
 .../test-diff-dwarf/test46-rust-report-0.txt  |     2 -
 tests/data/test-diff-dwarf/test5-report.txt   |     4 +-
 .../test-PR26739-2-report-0.txt               |    16 +-
 .../test30-pr18904-rvalueref-report0.txt      |    56 +-
 .../test30-pr18904-rvalueref-report1.txt      |    58 +-
 .../test30-pr18904-rvalueref-report2.txt      |    58 +-
 .../test31-pr18535-libstdc++-report-0.txt     |     2 +-
 .../test31-pr18535-libstdc++-report-1.txt     |     2 +-
 .../test35-pr18754-no-added-syms-report-0.txt |    56 +-
 .../test35-pr18754-no-added-syms-report-1.txt |     2 +-
 .../data/test-diff-filter/test41-report-0.txt |    40 +-
 ...t43-decl-only-def-change-leaf-report-0.txt |     5 +
 ...test47-filter-void-ptr-change-report-0.txt |     3 +
 tests/data/test-diff-filter/test5-report.txt  |     9 +-
 tests/data/test-diff-filter/test7-report.txt  |     1 -
 .../gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm    |   Bin 0 -> 122904364 bytes
 ...libICE-1.0.9-2.el7.x86_64.rpm-report-0.txt |    13 +-
 ...14.2.1-1.fc40.x86_64-self-check-report.txt |     5 +
 .../libgm2-14.2.1-1.fc40.x86_64.rpm           |   Bin 0 -> 223952 bytes
 .../libgm2-debuginfo-14.2.1-1.fc40.x86_64.rpm |   Bin 0 -> 441515 bytes
 ...l7.x86_64-0.12.8-1.el7.x86_64-report-2.txt |    26 +-
 ...bb-4.3-3.20141204.fc23.x86_64-report-0.txt |    43 +-
 ...bb-4.3-3.20141204.fc23.x86_64-report-1.txt |    29 +-
 ...0-from-fc20-to-fc23-dbus-glib-report-0.txt |    26 +
 ...dbus-glib-0.106-1.fc23.x86_64-report-0.txt |    11 +
 ....fc20--dbus-glib-0.106-1.fc23-report-0.txt |    26 +
 ...--dbus-glib-0.106-1.fc23.i686-report-0.txt |    15 +
 ...-glib-0.106-1.fc23.x86_64.rpm-report-0.txt |    11 +
 ...f-compare-from-fc23-dbus-glib-report-0.txt |     2 +
 ...-vte291-0.39.90-1.fc22.x86_64-report-0.txt |     1 +
 tests/data/test-read-btf/test0.o.abi          |    74 +-
 tests/data/test-read-btf/test1.o.abi          |    12 +-
 .../test-read-ctf/PR27700/test-PR27700.abi    |    10 +-
 tests/data/test-read-ctf/test-PR26568-1.o.abi |    18 +-
 tests/data/test-read-ctf/test-PR26568-2.o.abi |    16 +-
 tests/data/test-read-ctf/test-alias.o.abi     |     6 +-
 .../test-ambiguous-struct-A.o.hash.abi        |    20 +-
 .../test-ambiguous-struct-B.o.hash.abi        |    10 +-
 .../test-read-ctf/test-anonymous-fields.o.abi |    14 +-
 .../test-read-ctf/test-array-mdimension.abi   |    16 +-
 .../test-read-ctf/test-array-of-pointers.abi  |    24 +-
 tests/data/test-read-ctf/test-array-size.abi  |    18 +-
 .../data/test-read-ctf/test-bitfield-enum.abi |     8 +-
 tests/data/test-read-ctf/test-bitfield.abi    |    10 +-
 tests/data/test-read-ctf/test-callback.abi    |    10 +-
 tests/data/test-read-ctf/test-callback2.abi   |    12 +-
 .../test-conflicting-type-syms-a.o.hash.abi   |    10 +-
 .../test-conflicting-type-syms-b.o.hash.abi   |     8 +-
 tests/data/test-read-ctf/test-const-array.abi |    12 +-
 .../test-read-ctf/test-dynamic-array.o.abi    |    18 +-
 .../test-read-ctf/test-enum-many.o.hash.abi   |     8 +-
 .../test-read-ctf/test-enum-symbol.o.hash.abi |     6 +-
 tests/data/test-read-ctf/test-enum.o.abi      |     8 +-
 tests/data/test-read-ctf/test-fallback.abi    |     4 +-
 .../test-read-ctf/test-forward-type-decl.abi  |    14 +-
 .../test-functions-declaration.abi            |    14 +-
 .../data/test-read-ctf/test-linux-module.abi  |    38 +-
 tests/data/test-read-ctf/test-list-struct.abi |     8 +-
 tests/data/test-read-ctf/test0.abi            |    40 +-
 tests/data/test-read-ctf/test0.hash.abi       |    40 +-
 tests/data/test-read-ctf/test1.so.abi         |    18 +-
 tests/data/test-read-ctf/test1.so.hash.abi    |    18 +-
 tests/data/test-read-ctf/test2.so.abi         |    18 +-
 tests/data/test-read-ctf/test2.so.hash.abi    |    18 +-
 tests/data/test-read-ctf/test3.so.abi         |     4 +-
 tests/data/test-read-ctf/test3.so.hash.abi    |     4 +-
 tests/data/test-read-ctf/test4.so.abi         |    18 +-
 tests/data/test-read-ctf/test4.so.hash.abi    |    18 +-
 tests/data/test-read-ctf/test5.o.abi          |    34 +-
 tests/data/test-read-ctf/test7.o.abi          |    22 +-
 tests/data/test-read-ctf/test8.o.abi          |     4 +-
 tests/data/test-read-ctf/test9.o.abi          |    48 +-
 .../PR22015-libboost_iostreams.so.abi         |  1579 +-
 .../test-read-dwarf/PR22122-libftdc.so.abi    |  6146 +--
 .../PR24378-fn-is-not-scope.abi               |    26 +-
 .../data/test-read-dwarf/PR25007-sdhci.ko.abi |  2938 +-
 .../PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi |   609 +-
 .../test-read-dwarf/PR26261/PR26261-exe.abi   |    44 +-
 .../test-read-dwarf/PR27700/test-PR27700.abi  |    10 +-
 .../PR28584/PR28584-smv.clang.o.abi           |     6 +-
 .../test-read-dwarf/PR29443-missing-xx.o.abi  |     8 +-
 .../PR29692-kdelibs3-libkjava.so.1.0.0.abi    |     2 +-
 tests/data/test-read-dwarf/libtest23.so.abi   |   316 +-
 .../libtest24-drop-fns-2.so.abi               |   353 +-
 .../test-read-dwarf/libtest24-drop-fns.so.abi |   353 +-
 .../data/test-read-dwarf/test-PR26568-1.o.abi |    18 +-
 .../data/test-read-dwarf/test-PR26568-2.o.abi |    16 +-
 tests/data/test-read-dwarf/test-fallback.abi  |     4 +-
 .../test-read-dwarf/test-libaaudio.so.abi     |   424 +-
 .../test-read-dwarf/test-libandroid.so.abi    | 21142 +++++----
 .../test-pointer-to-member-1.o.abi            |    34 +-
 .../test-suppressed-alias.o.abi               |     6 +-
 tests/data/test-read-dwarf/test0.abi          |    46 +-
 tests/data/test-read-dwarf/test0.hash.abi     |    48 +-
 tests/data/test-read-dwarf/test1.abi          |    44 +-
 tests/data/test-read-dwarf/test1.hash.abi     |    50 +-
 .../test-read-dwarf/test10-pr18818-gcc.so.abi |  4194 +-
 .../test-read-dwarf/test11-pr18828.so.abi     | 11118 ++---
 .../test-read-dwarf/test12-pr18844.so.abi     | 21428 ++++-----
 .../test-read-dwarf/test13-pr18894.so.abi     |  1260 +-
 .../test-read-dwarf/test14-pr18893.so.abi     |  3091 +-
 .../test-read-dwarf/test15-pr18892.so.abi     | 20201 ++++-----
 .../test-read-dwarf/test16-pr18904.so.abi     | 19855 ++++----
 .../test-read-dwarf/test17-pr19027.so.abi     |  8996 ++--
 ...st18-pr19037-libvtkRenderingLIC-6.1.so.abi |  9051 ++--
 ...19-pr19023-libtcmalloc_and_profiler.so.abi |  9371 ++--
 tests/data/test-read-dwarf/test2.so.abi       |    30 +-
 tests/data/test-read-dwarf/test2.so.hash.abi  |    30 +-
 ...st20-pr19025-libvtkParallelCore-6.1.so.abi |  7774 ++--
 .../test-read-dwarf/test21-pr19092.so.abi     |  3804 +-
 .../test22-pr19097-libstdc++.so.6.0.17.so.abi | 37350 ++++++++--------
 .../test-read-dwarf/test3-alias-1.so.hash.abi |     4 +-
 .../test-read-dwarf/test3-alias-2.so.hash.abi |     4 +-
 .../test-read-dwarf/test3-alias-3.so.hash.abi |     4 +-
 .../test-read-dwarf/test3-alias-4.so.hash.abi |     4 +-
 tests/data/test-read-dwarf/test3.so.abi       |     4 +-
 tests/data/test-read-dwarf/test3.so.hash.abi  |     4 +-
 tests/data/test-read-dwarf/test4.so.abi       |    18 +-
 tests/data/test-read-dwarf/test4.so.hash.abi  |    18 +-
 tests/data/test-read-dwarf/test5.o.abi        |     4 +-
 tests/data/test-read-dwarf/test5.o.hash.abi   |     4 +-
 tests/data/test-read-dwarf/test6.so.abi       |    18 +-
 tests/data/test-read-dwarf/test6.so.hash.abi  |    18 +-
 tests/data/test-read-dwarf/test7.so.abi       |    52 +-
 tests/data/test-read-dwarf/test7.so.hash.abi  |    52 +-
 .../test8-qualified-this-pointer.so.abi       |    16 +-
 .../test8-qualified-this-pointer.so.hash.abi  |    16 +-
 .../test9-pr18818-clang.so.abi                |  3051 +-
 tests/data/test-read-write/test-crc.xml       |     2 +-
 tests/data/test-read-write/test0.xml          |     2 +-
 tests/data/test-read-write/test1.xml          |     2 +-
 tests/data/test-read-write/test10.xml         |    20 +-
 tests/data/test-read-write/test11.xml         |     4 +-
 tests/data/test-read-write/test12.xml         |     6 +-
 tests/data/test-read-write/test13.xml         |     8 +-
 tests/data/test-read-write/test14.xml         |     8 +-
 tests/data/test-read-write/test15.xml         |     6 +-
 tests/data/test-read-write/test16.xml         |     4 +-
 tests/data/test-read-write/test17.xml         |    54 +-
 tests/data/test-read-write/test18.xml         |    16 +-
 tests/data/test-read-write/test19.xml         |    22 +-
 tests/data/test-read-write/test2.xml          |     8 +-
 tests/data/test-read-write/test20.xml         |    88 +-
 tests/data/test-read-write/test21.xml         |    26 +-
 tests/data/test-read-write/test22.xml         |    12 +-
 tests/data/test-read-write/test23.xml         |    12 +-
 tests/data/test-read-write/test24.xml         |    16 +-
 tests/data/test-read-write/test25.xml         |    62 +-
 tests/data/test-read-write/test26.xml         |    22 +-
 tests/data/test-read-write/test27.xml         |    16 +-
 .../test28-without-std-fns-ref.xml            |  1190 +-
 .../test28-without-std-vars-ref.xml           |  1116 +-
 tests/data/test-read-write/test3.xml          |     8 +-
 tests/data/test-read-write/test4.xml          |     4 +-
 tests/data/test-read-write/test5.xml          |     4 +-
 tests/data/test-read-write/test6.xml          |    10 +-
 tests/data/test-read-write/test7.xml          |     4 +-
 tests/data/test-read-write/test8.xml          |     2 +-
 tests/data/test-read-write/test9.xml          |     8 +-
 tests/test-diff-pkg.cc                        |    13 +
 tools/abidw.cc                                |     3 +-
 tools/abilint.cc                              |    30 +-
 tools/abipkgdiff.cc                           |    38 +-
 220 files changed, 140901 insertions(+), 145092 deletions(-)
 create mode 100644 tests/data/test-diff-pkg/gcc-debuginfo-14.2.1-1.fc40.x86_64.rpm
 create mode 100644 tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64-self-check-report.txt
 create mode 100644 tests/data/test-diff-pkg/libgm2-14.2.1-1.fc40.x86_64.rpm
 create mode 100644 tests/data/test-diff-pkg/libgm2-debuginfo-14.2.1-1.fc40.x86_64.rpm

-- 
2.43.5


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2024-08-29 16:17 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-08-29 15:46 [PATCH 00/27] Implement type hashing & fix self-comparing gcc-gnat in fc37 dodji
2024-08-29 15:52 ` [PATCH 01/27] Implement type hashing Dodji Seketeli
2024-08-29 15:54 ` [PATCH 02/27] ir: Remove the now useless type propagation optimization Dodji Seketeli
2024-08-29 15:55 ` [PATCH 03/27] ir: decl-only classes don't equal fully defined classes under ODR Dodji Seketeli
2024-08-29 15:56 ` [PATCH 04/27] comp-filter: Consider lvalue-ness changes on references as harmful Dodji Seketeli
2024-08-29 15:56 ` [PATCH 05/27] comp-filter: Ignore ptr size when detecting void ptr to ptr change Dodji Seketeli
2024-08-29 15:57 ` [PATCH 06/27] Don't strip typedefs in fn names when pretty-printing and comparing Dodji Seketeli
2024-08-29 15:58 ` [PATCH 07/27] reader: Avoid duplicating member types Dodji Seketeli
2024-08-29 15:58 ` [PATCH 08/27] ir: Cache the result of scope_decl::get_sorted_member_types Dodji Seketeli
2024-08-29 16:01 ` [PATCH 09/27] dwarf-reader: Avoid duplicating anonymous member types Dodji Seketeli
2024-08-29 16:02 ` [PATCH 10/27] reader: Avoid duplicating recursive types Dodji Seketeli
2024-08-29 16:02 ` [PATCH 11/27] dwarf-reader: Speed-up decl-only resolution Dodji Seketeli
2024-08-29 16:03 ` [PATCH 12/27] ir: Fix name setting of a ptr-to-mbr-type Dodji Seketeli
2024-08-29 16:04 ` [PATCH 13/27] dwarf-reader: Support LLVM's lingo of declaration-ness Dodji Seketeli
2024-08-29 16:04 ` [PATCH 14/27] reader: Improve logging in the ABIXML reader Dodji Seketeli
2024-08-29 16:05 ` [PATCH 15/27] ir: Improve type logging during type canonicalization Dodji Seketeli
2024-08-29 16:05 ` [PATCH 16/27] tools-utils: Improve logging while reading a Linux kernel Dodji Seketeli
2024-08-29 16:06 ` [PATCH 17/27] dwarf-reader: Fix building of void, void* and variadic parm types Dodji Seketeli
2024-08-29 16:06 ` [PATCH 18/27] {dwarf,btf,ctf}-reader: Set the origin of the corpus group Dodji Seketeli
2024-08-29 16:07 ` [PATCH 19/27] {btf,ctf,dwarf}-reader, ir: Fix self-comparison debugging for corpus groups Dodji Seketeli
2024-08-29 16:07 ` [PATCH 20/27] abilint: Support --verbose option Dodji Seketeli
2024-08-29 16:08 ` [PATCH 21/27] big-tests: Update git sub-module Dodji Seketeli
2024-08-29 16:08 ` [PATCH 22/27] dwarf-reader,tools-utils: Add statistics about built/suppressed functions Dodji Seketeli
2024-08-29 16:09 ` [PATCH 23/27] abidw: Add a --kmi-stablelist option alongside existing --kmi-whitelist Dodji Seketeli
2024-08-29 16:10 ` [PATCH 24/27] dwarf-reader,ir: Fix endless loop while analyzing DWARF from Modula-2 Dodji Seketeli
2024-08-29 16:13 ` [PATCH 25/27] abipkgdiff: Extract devel and main packages in the same directory Dodji Seketeli
2024-08-29 16:13 ` [PATCH 26/27] dwarf-reader,reader.cc: Fix function virtuality setting Dodji Seketeli
2024-08-29 16:15 ` [PATCH 27/27] dwarf-reader,ir,writer: Better support for static member variables Dodji Seketeli

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).