[Bug default/26941] New: abidiff assert in build_or_get_type_decl when comparing a library to itself

[Bug default/26941] New: abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

            Bug ID: 26941
           Summary: abidiff assert in build_or_get_type_decl when
                    comparing a library to itself
           Product: libabigail
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: default
          Assignee: dodji at redhat dot com
          Reporter: woodard at redhat dot com
                CC: libabigail at sourceware dot org
  Target Milestone: ---

With commit 2c0dd017 from dodji/PR26769 I get 

This is probably somewhat related to 26769 but it appears different.
First create an abixml file with abidw then run abidiff on it against the same
library. 

/home/ben/Scratch/Work/test/libabigail-x86_64/bin/abidiff
libabigail.so.0.0.0.abixml /lib64/libabigail.so.0.0.0
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
abidiff: ../../../libabigail/src/abg-reader.cc:1287:
abigail::ir::type_base_sptr
abigail::xml_reader::read_context::build_or_get_type_decl(const string&, bool):
Assertion `__abg_cond__' failed.

Program received signal SIGABRT, Aborted.
__GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:49
49        return ret;
#0  __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:49
#1  0x00007ffff750a8a4 in __GI_abort () at abort.c:79
#2  0x00007ffff750a789 in __assert_fail_base (fmt=0x7ffff7677680 "%s%s%s:%u:
%s%sAssertion `%s' failed.\n%n", assertion=0x7ffff7f3bfa4 "__abg_cond__",
file=0x7ffff7f47320 "../../../libabigail/src/abg-reader.cc", line=1287,
function=<optimized out>) at assert.c:92
#3  0x00007ffff751a026 in __GI___assert_fail (assertion=0x7ffff7f3bfa4
"__abg_cond__", file=0x7ffff7f47320 "../../../libabigail/src/abg-reader.cc",
line=1287, function=0x7ffff7f475d8 "abigail::ir::type_base_sptr
abigail::xml_reader::read_context::build_or_get_type_decl(const string&,
bool)") at assert.c:101
#4  0x00007ffff7eaa629 in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-12787", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1287
#5  0x00007ffff7eabd7a in abigail::xml_reader::build_reference_type_def
(ctxt=..., node=0x1d08cc0, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3653
#6  0x00007ffff7eb49d3 in abigail::xml_reader::build_type (ctxt=...,
node=0x1d08cc0, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:5471
#7  0x00007ffff7eaa42f in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1307
#8  0x00007ffff7eac6b3 in abigail::xml_reader::build_function_parameter
(ctxt=..., node=0x88e2270) at ../../../libabigail/src/abg-reader.cc:2987
#9  0x00007ffff7eb02ed in abigail::xml_reader::build_function_decl (ctxt=...,
node=0x88e1920, as_method_decl=std::shared_ptr<abigail::ir::class_or_union>
(use count 15, weak count 3) = {...}, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3072
#10 0x00007ffff7eb1149 in
abigail::xml_reader::build_function_decl_if_not_suppressed (ctxt=...,
node=0x88e1920, as_method_decl=std::shared_ptr<abigail::ir::class_or_union>
(use count 15, weak count 3) = {...}, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3160
#11 0x00007ffff7eb7334 in abigail::xml_reader::build_class_decl (ctxt=...,
node=<optimized out>, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4644
#12 0x00007ffff7eb8179 in
abigail::xml_reader::build_class_decl_if_not_suppressed (ctxt=...,
node=0x88dd290, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4269
#13 0x00007ffff7ebc856 in abigail::xml_reader::handle_class_decl
(add_to_current_scope=true, node=0x88dd290, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5685
#14 abigail::xml_reader::handle_element_node (ctxt=..., node=0x88dd290,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2150
#15 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x88da5e0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#16 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x88da5e0, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#17 abigail::xml_reader::handle_element_node (ctxt=..., node=0x88da5e0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#18 0x00007ffff7ea9f4d in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffabcc:
abigail::ir::no_access) at ../../../libabigail/src/abg-reader.cc:1258
#19 0x00007ffff7eaa37b in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-29701", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1293
#20 0x00007ffff7eb5b3e in abigail::xml_reader::build_class_decl (ctxt=...,
node=<optimized out>, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4510
#21 0x00007ffff7eb8179 in
abigail::xml_reader::build_class_decl_if_not_suppressed (ctxt=...,
node=0x88bfdd0, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4269
#22 0x00007ffff7ebc856 in abigail::xml_reader::handle_class_decl
(add_to_current_scope=true, node=0x88bfdd0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5685
#23 abigail::xml_reader::handle_element_node (ctxt=..., node=0x88bfdd0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2150
#24 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x88bfb50, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#25 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x88bfb50, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#26 abigail::xml_reader::handle_element_node (ctxt=..., node=0x88bfb50,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#27 0x00007ffff7ea9f4d in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffb2cc:
abigail::ir::no_access) at ../../../libabigail/src/abg-reader.cc:1258
#28 0x00007ffff7ea9ea6 in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffb34c:
abigail::ir::public_access) at ../../../libabigail/src/abg-reader.cc:1256
#29 0x00007ffff7eaa37b in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-29668", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1293
#30 0x00007ffff7eaf131 in abigail::xml_reader::build_typedef_decl (ctxt=...,
node=0x87a5860, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4215
#31 0x00007ffff7eb4c4e in abigail::xml_reader::build_type (ctxt=...,
node=0x87a5860, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:5476
#32 0x00007ffff7eb633a in abigail::xml_reader::build_class_decl (ctxt=...,
node=<optimized out>, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4550
#33 0x00007ffff7eb8179 in
abigail::xml_reader::build_class_decl_if_not_suppressed (ctxt=...,
node=0x87a4e20, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4269
#34 0x00007ffff7ebc856 in abigail::xml_reader::handle_class_decl
(add_to_current_scope=true, node=0x87a4e20, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5685
#35 abigail::xml_reader::handle_element_node (ctxt=..., node=0x87a4e20,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2150
#36 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x87a4ba0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#37 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x87a4ba0, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#38 abigail::xml_reader::handle_element_node (ctxt=..., node=0x87a4ba0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#39 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x87a49b0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#40 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x87a49b0, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#41 abigail::xml_reader::handle_element_node (ctxt=..., node=0x87a49b0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#42 0x00007ffff7ea9f4d in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffbe1c:
abigail::ir::no_access) at ../../../libabigail/src/abg-reader.cc:1258
#43 0x00007ffff7eaa37b in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-14144", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1293
#44 0x00007ffff7eab455 in abigail::xml_reader::build_pointer_type_def
(ctxt=..., node=0x1ee5e60, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3567
#45 0x00007ffff7eb492c in abigail::xml_reader::build_type (ctxt=...,
node=0x1ee5e60, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:5470
#46 0x00007ffff7eaa42f in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1307
#47 0x00007ffff7eafa14 in abigail::xml_reader::build_var_decl (ctxt=...,
node=0x8790610, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3318
#48 0x00007ffff7eb6c43 in abigail::xml_reader::build_class_decl (ctxt=...,
node=<optimized out>, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4588
#49 0x00007ffff7eb8179 in
abigail::xml_reader::build_class_decl_if_not_suppressed (ctxt=...,
node=0x878fad0, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4269
#50 0x00007ffff7ebc856 in abigail::xml_reader::handle_class_decl
(add_to_current_scope=true, node=0x878fad0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5685
#51 abigail::xml_reader::handle_element_node (ctxt=..., node=0x878fad0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2150
#52 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x878f8c0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#53 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x878f8c0, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#54 abigail::xml_reader::handle_element_node (ctxt=..., node=0x878f8c0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#55 0x00007ffff7ea9f4d in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffc90c:
abigail::ir::no_access) at ../../../libabigail/src/abg-reader.cc:1258
#56 0x00007ffff7eaa37b in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-1715", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1293
#57 0x00007ffff7eab455 in abigail::xml_reader::build_pointer_type_def
(ctxt=..., node=0xb6fa30, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3567
#58 0x00007ffff7eb492c in abigail::xml_reader::build_type (ctxt=...,
node=0xb6fa30, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:5470
#59 0x00007ffff7eaa42f in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1307
#60 0x00007ffff7eb0928 in abigail::xml_reader::build_function_decl (ctxt=...,
node=0x8a03150, as_method_decl=std::shared_ptr<abigail::ir::class_or_union>
(use count 13, weak count 8) = {...}, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3083
#61 0x00007ffff7eb1149 in
abigail::xml_reader::build_function_decl_if_not_suppressed (ctxt=...,
node=0x8a03150, as_method_decl=std::shared_ptr<abigail::ir::class_or_union>
(use count 13, weak count 8) = {...}, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3160
#62 0x00007ffff7eb7334 in abigail::xml_reader::build_class_decl (ctxt=...,
node=<optimized out>, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4644
#63 0x00007ffff7eb8179 in
abigail::xml_reader::build_class_decl_if_not_suppressed (ctxt=...,
node=0x89fe2f0, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:4269
#64 0x00007ffff7ebc856 in abigail::xml_reader::handle_class_decl
(add_to_current_scope=true, node=0x89fe2f0, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5685
#65 abigail::xml_reader::handle_element_node (ctxt=..., node=0x89fe2f0,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2150
#66 0x00007ffff7ebb494 in abigail::xml_reader::build_namespace_decl
(add_to_current_scope=true, node=0x89fe090, ctxt=...) at
../../../libabigail/src/abg-reader.cc:2713
#67 abigail::xml_reader::handle_namespace_decl (add_to_current_scope=true,
node=0x89fe090, ctxt=...) at ../../../libabigail/src/abg-reader.cc:5522
#68 abigail::xml_reader::handle_element_node (ctxt=..., node=0x89fe090,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2132
#69 0x00007ffff7ea9f4d in abigail::xml_reader::read_context::get_scope_for_node
(this=0x4497e0, node=<optimized out>, access=@0x7fffffffd55c:
abigail::ir::no_access) at ../../../libabigail/src/abg-reader.cc:1258
#70 0x00007ffff7eaa37b in
abigail::xml_reader::read_context::build_or_get_type_decl (this=0x4497e0,
id="type-id-159", add_decl_to_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:1293
#71 0x00007ffff7eab455 in abigail::xml_reader::build_pointer_type_def
(ctxt=..., node=0xa70f70, add_to_current_scope=<optimized out>) at
../../../libabigail/src/abg-reader.cc:3567
#72 0x00007ffff7ebb724 in abigail::xml_reader::handle_pointer_type_def
(add_to_current_scope=true, node=0xa70f70, ctxt=...) at
../../../libabigail/src/abg-reader.cc:5554
#73 abigail::xml_reader::handle_element_node (ctxt=..., node=0xa70f70,
add_to_current_scope=true) at ../../../libabigail/src/abg-reader.cc:2137
#74 0x00007ffff7ea65ed in abigail::xml_reader::read_translation_unit
(node=<optimized out>, tu=..., ctxt=...) at
../../../libabigail/src/abg-reader.cc:1408
#75 abigail::xml_reader::get_or_read_and_add_translation_unit (ctxt=...,
node=<optimized out>) at ../../../libabigail/src/abg-reader.cc:1460
#76 0x00007ffff7ea6d5d in abigail::xml_reader::read_translation_unit_from_input
(ctxt=...) at ../../../libabigail/src/abg-reader.cc:1518
#77 0x00007ffff7ea7e54 in abigail::xml_reader::read_corpus_from_input
(ctxt=...) at ../../../libabigail/src/abg-reader.cc:1932
#78 0x000000000040639b in main (argc=<optimized out>, argv=0x7fffffffdf68) at
/usr/include/c++/10/bits/shared_ptr_base.h:1324

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

dodji at redhat dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WORKSFORME
             Status|NEW                         |RESOLVED

--- Comment #1 from dodji at redhat dot com ---
This now works for me, with the current master which tip commit is:

18b0693c abipkgdiff: minor cleanups

Note that doing this test is now part of the set of regression tests in
libabigail. It's executed by typing "make check-self-compare".

That test above is not run by default when doing "make check" because it takes
a lot of time and memory (1.6GB and more than 5 minutes on my Rawhide box), but
I run it regularly now.

Thanks for reporting this problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

Ben Woodard <woodard at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WORKSFORME                  |---

--- Comment #2 from Ben Woodard <woodard at redhat dot com> ---
I'm still reproducing this problem in 105 different cases with the libabigail
trunk up through commit 2417efb2

I guess that we assumed that all of these would have the same proximate cause
when in fact they have different causes. 

attached is the full list of files and the RPMs that they come from which still
exhibit this problem.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

--- Comment #3 from Ben Woodard <woodard at redhat dot com> ---
Created attachment 13013
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13013&action=edit
List of files which still assert in abg-reader.cc:1287

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

--- Comment #4 from Ben Woodard <woodard at redhat dot com> ---
Just to illuminate how I test this:

I have debuginfod running and have downloaded all the debuginfo packages for
fedora33. My configuration file is included. The basic idea is that I have a
directory called /data/debug which I added to the default directories that I
tell debuginfod to look into. To keep this up to date I also have a script
called 
freshen-debuginfo.sh which does a reposync of the directories to keep them up
to date. I generally run this before a test run.

I have also downloaded flux-core https://github.com/flux-framework/flux-core
and built that in my environment. I setup my path to my installation of flux
and a current build of libabigail and then run "flux start". I will include
these in the reproducer package as well. But you will need to adjust your paths
as well.

flux start launches a nested shell from which I can run many flux jobs. I have
a script that launches all of those jobs called "flux-abidw.sh". Running that
launches all the jobs and queues them up to be run sequentially as processors
are available.

You can see how many jobs failed by simply running "flux jobs -f failed". On my
system a 6 core i7-8700 running at 3.2GHz it takes about an hour for all the
jobs to complete. However, all but a couple of them are done within about 10
min.

To get the backtraces I have a bunch of other scripts which gather the
backtraces for me. To kick off gathering the backtraces there is a script which
is ./flux-gen-bts-abidw.sh which uses the other scripts ./flux-pull-fails-2.sh
gather-bt-abidw.sh to do the work. What you are left with is a list of files
ending in .bt

Then I pick one of them and look at the line causing the failure and then see
how often it is amongst the backtraces. In this case I looked at 
$ cat libTKBool.so.7.4.0.bt
$ grep -l abg-reader.cc:1287 *.bt | wc -l
105

Then I ran a another program called "all-with.sh" which generated the list I
attached to the commebnt 3.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

--- Comment #5 from Ben Woodard <woodard at redhat dot com> ---
Created attachment 13014
  --> https://sourceware.org/bugzilla/attachment.cgi?id=13014&action=edit
scripts used to run tests

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/26941] abidiff assert in build_or_get_type_decl when comparing a library to itself

[woodard at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=26941

Ben Woodard <woodard at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WORKSFORME
             Status|REOPENED                    |RESOLVED

--- Comment #6 from Ben Woodard <woodard at redhat dot com> ---
figured out why dodji and my tests were different.

-- 
You are receiving this mail because:
You are on the CC list for the bug.