From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by sourceware.org (Postfix) with ESMTPS id 89FB7384A430 for ; Tue, 16 Jul 2024 14:55:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 89FB7384A430 Authentication-Results: sourceware.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=seketeli.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 89FB7384A430 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.70.183.195 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1721141748; cv=none; b=vXhrGhsWjVVKWmQsMUldw4Spks/xYs/8LgQ+UGI16+BG79KYEWk+huur+6lD8qRwao51otujPRpxvoxKeFPR3lIN32wb1nDaq7GkTk2saeM6b/wQ/p6xqhNE3Ph0O1TGR5w1DRLKk0aeifIwMgacL0JsAncF2FMg23sA5w40Xco= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1721141748; c=relaxed/simple; bh=K1KatkefdcgE/KkgqwUCdW/T2YtWpz/0qefG1wgpoi0=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=ddO5pVirEd7lo0NrK/662ZwuGi8ewz3g+Q2KH2MgiT5m5CZNJXH1I3dDIvKY4XQlhk1/ee2Rg9c0TlQZCs1PSRxZPwD61ZqDdgLU27ME0CTwxSRUsRQXKSR5ytKPprHXF3zr0ZXEtcnfNEA81a9H2FKAqXd2tCQ+xTJPRhbH+Cc= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail.gandi.net (Postfix) with ESMTPSA id E2E9560004; Tue, 16 Jul 2024 14:55:41 +0000 (UTC) Received: by localhost (Postfix, from userid 1000) id 331FEC1B7511; Tue, 16 Jul 2024 16:55:41 +0200 (CEST) From: dodji@redhat.com To: libabigail@sourceware.org Cc: dodji@redhat.com Subject: [PATCH 00/17] Support self comparison of vmlinux & modules using BTF/CTF Date: Tue, 16 Jul 2024 16:55:11 +0200 Message-ID: <20240716145541.473065-1-dodji@redhat.com> X-Mailer: git-send-email 2.43.5 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-GND-Sasl: dodj@seketeli.org X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00,HEADER_FROM_DIFFERENT_DOMAINS,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: From: Dodji Seketeli Hello, In libabigail lingo, self comparing a binary means building an ABI corpus A for the binary, serialize it into ABIXML, read the ABIXML back into a corpus A' and compare A' against A. As A and A' should be equivalent, the comparison should yield the empty set. That self comparison is done using: abidw --abidiff But then in the case of a Linux kernel build tree (vmlinux + kernel modules), self comparison is dong using: abidw --abidiff --linux-tree --vmlinux If we want to use either the BTF or the CTF front-end for self comparison, the command to use is: abidw --abidiff --{btf,ctf} --linux-tree --vmlinux It turns out the later command doesn't work. Neither with the BTF not with the CTF front-end. This patch series intends to make self comparison of the linux kernel work for the BTF and CTF front-ends. The first two patches are fixes for generic issues that are independent from the front-ends. The third patch fixes the BTF front-end to make it take kernel modules into account for its ABI analysis. The fourth patch is a pre-requisite for allowing the CTF front end to add translation units to a given ABI corpus. Patches 5 to 7 are fixes specific to the CTF front-end. Patches 8 to 12 are fixes specific to the ABIXML front-end. Now that all front-ends can build a proper corpus group representing the ABI of the union of vmlinux and its modules (a corpus group), patches 13 to 15 are fixes to the general IR as well as the diff IR. Patch 16 adds support for the --abidiff option of abidw for Linux kernel binaries. Yup, weirdly enough, that is otherwise still not supported to this day. The last patch adds testing for the self-comparison of the Linux kernel, validating all the patches above. This is done using an external Git repository that holds the tests data that would be too big to be hosted in the current repository. This patch set has been tested using the Sourceware CI builders. I am thus applying it to the mainline. Dodji Seketeli (17): ir: Fix a potential crash in canonicalize_types elf-based-reader: Clean up logic of elf_based_reader::read_and_add_corpus_to_group tools-utils,btf-reader: Take modules into account for corpus group corpus: Support adding translation units with empty path ctf-reader: Do not set data member offsets for unions ctf-reader: During re-initialization, only clear canonicalize-able types ctf-reader: Fix analyzing single kernel binaries reader: Fix corpus group reading reader: Simplify type canonicalization invocation reader: Simplify logic of get_or_read_and_add_translation_unit reader: Fix building of void and void pointer types reader: Fix building of variadic parameter type ir: Don't strip typedefs from parms and return type when comparing fns ir: Rename integral_type into real_type ir,comparison,default-reporter: Consider sub-ranges in array diffs abidw: Support the --abidiff option for Linux Kernel trees configure: Support the optional 'big-tests' sub-directory .gitmodules | 3 + CONTRIBUTING | 159 +- Makefile.am | 9 +- big-tests | 1 + configure.ac | 32 + include/abg-comparison.h | 21 +- include/abg-corpus.h | 2 + include/abg-fwd.h | 6 + src/abg-btf-reader.cc | 275 +- src/abg-comp-filter.cc | 44 +- src/abg-comparison-priv.h | 7 +- src/abg-comparison.cc | 111 +- src/abg-corpus.cc | 24 +- src/abg-ctf-reader.cc | 85 +- src/abg-default-reporter.cc | 18 + src/abg-dwarf-reader.cc | 12 +- src/abg-elf-based-reader.cc | 2 +- src/abg-fe-iface.cc | 1 - src/abg-ir-priv.h | 59 +- src/abg-ir.cc | 331 +- src/abg-reader.cc | 221 +- src/abg-reporter-priv.cc | 27 +- src/abg-tools-utils.cc | 6 +- tests/Makefile.am | 2 +- .../PR30329/PR30329-report-1.txt | 4 +- .../libsdl/libsdl-1.2.60-1.2.64-report.txt | 2 +- .../reported/PR31513-reported-report-1.txt | 2 +- .../reported/PR31513-reported-report-2.txt | 2 +- .../test1-ada-subrange-report-1.txt | 4 +- .../test1-ada-subrange-report-2.txt | 4 +- .../test2-ada-subrange-redundant-report-1.txt | 6 +- .../test2-ada-subrange-redundant-report-2.txt | 6 +- .../test-allow-type-array-v0--v1-report-1.txt | 2 + .../test-allow-type-array-v0--v3-report-1.txt | 2 + .../data/test-abidiff/test-PR27985-report.txt | 2 + .../data/test-annotate/test14-pr18893.so.abi | 267 +- .../data/test-annotate/test15-pr18892.so.abi | 4703 +++--- .../data/test-annotate/test17-pr19027.so.abi | 7401 ++++----- ...19-pr19023-libtcmalloc_and_profiler.so.abi | 5197 +++--- tests/data/test-diff-dwarf/test10-report.txt | 2 + tests/data/test-diff-dwarf/test11-report.txt | 4 + tests/data/test-diff-dwarf/test2-report.txt | 11 + ...t35-pr19173-libfoo-long-clang-report-0.txt | 2 + ...est35-pr19173-libfoo-long-gcc-report-0.txt | 2 + .../PR24430-fold-qualified-array-report-0.txt | 3 + .../test-PR29811-0-report-1.txt | 4 + tests/data/test-diff-filter/test3-report.txt | 2 +- .../test30-pr18904-rvalueref-report0.txt | 2 + .../test30-pr18904-rvalueref-report1.txt | 2 + .../test30-pr18904-rvalueref-report2.txt | 2 + .../test35-pr18754-no-added-syms-report-0.txt | 2 + .../gmp-6.x.x86_64-report-0.txt | 2 +- ...libICE-1.0.9-2.el7.x86_64.rpm-report-0.txt | 4 +- .../nss-3.23.0-1.0.fc23.x86_64-report-0.txt | 2 + ...bb-4.3-3.20141204.fc23.x86_64-report-0.txt | 2 + ...bb-4.3-3.20141204.fc23.x86_64-report-1.txt | 2 + ...has-data-member-inserted-at-1-report-1.txt | 2 + ...-array-data-member-conversion-report-2.txt | 2 + .../test-diff-suppr/test45-abi-report-1.txt | 4 +- .../data/test-read-dwarf/PR25007-sdhci.ko.abi | 4202 ++--- .../test-read-dwarf/test-libandroid.so.abi | 26 +- .../test-read-dwarf/test14-pr18893.so.abi | 262 +- .../test-read-dwarf/test15-pr18892.so.abi | 4497 ++--- .../test-read-dwarf/test17-pr19027.so.abi | 7133 ++++---- ...19-pr19023-libtcmalloc_and_profiler.so.abi | 5173 +++--- .../test22-pr19097-libstdc++.so.6.0.17.so.abi | 13659 ++++++++-------- tests/data/test-read-write/test17.xml | 44 +- tests/data/test-read-write/test19.xml | 12 +- tests/data/test-read-write/test20.xml | 68 +- tests/data/test-read-write/test21.xml | 14 +- tests/data/test-read-write/test22.xml | 1 + tests/data/test-read-write/test23.xml | 1 + tests/data/test-read-write/test25.xml | 46 +- tests/data/test-read-write/test26.xml | 16 +- .../test28-without-std-fns-ref.xml | 937 +- .../test28-without-std-vars-ref.xml | 865 +- tools/abidw.cc | 208 +- 77 files changed, 28732 insertions(+), 27552 deletions(-) create mode 100644 .gitmodules create mode 160000 big-tests -- 2.43.5