[PATCH, applied] Introduce artificial locations

[PATCH, applied] Introduce artificial locations

[Dodji Seketeli]

[-- Attachment #1: Type: text/plain, Size: 11700 bytes --]

Hello,

When an abixml file is "read in" and the resulting in-memory internal
representation is saved back into abixml, the saved result can often
differ from the initial input in a non deterministic manner.  That
read-write instability is non-desirable because it generates
unnecessary changes that cloud our ability to build reliable
regression tests, among other things.  Also, that unnecessarily
increases the changes to the existing regression test reference
outputs leading to a lot more churn than necessary.

This patch tries to minimize that abixml read-write instability in
preparation of patches that would otherwise cause too much churn in
reference output files of the regression test suite.

The main reason why this read-write instability occurs is that a lot
of type definitions don't have source location.

For instance, all the types that are not user defined fall into that
category.  Those types can't be topologically sorted by using their
location as a sorting criteria.  Instead, we are currently using the
order in which those location-less types are processed by the reader
as the output (i.e, write time) order.  The problem with that approach
is that the processing order can be dependant on the other of which
OTHER TYPES likes class types are processed.  And that order can be
changed by patches in the future.  That in and of itself shouldn't
change the write order of these types.

For instance, if a class Foo has data members and member functions
whose types are non-user-defined types, then, the order in which those
data members are processed can possibly determine the order in which
those non-user-defined are processed.

This patch thus introduces the concept of artificial location.

A *NON-ARTIFICIAL* location is a source location that was emitted by
the original emitter of the type meta-data.  In the case of DWARF type
meta-data, the compiler originally emitted source location.  That
location when read is considered non-artificial, or natural, if you
prefer.

In the case of abixml however, an artificial location would be the
source location at which an XML element is encountered.

For instance, consider the abixml file below "path/to/exmaple.abi" below:

     1	    <abi-corpus version='2.0' path='path/to/example.abi'>
     2	      <abi-instr address-size='64' path='test24-drop-fns.cc' language='LANG_C_plus_plus'>
     3		<type-decl name='bool' size-in-bits='8' id='type-id-1'/>
     4	      </abi-instr>
     5	    </abi-corpus/>

I've added line numbers for ease of reading.

At line 3 of that file, the non-user defined type name "bool" is
defined using the XML element "type-decl".  Note how that element
lacks the "filepath", "line" and "column" attributes that would collectively
define the source location of that type.  So this type "bool" don't
carry any natural location.

The abixml reader can however generate an artificial location for it.
That the filepath of that artificial location would thus be the path
to that ABI corpus, i.e, "path/to/example.abi".  The line number would
be 3.  The column would be left to zero.

That artificial location will never be explicitly be written down as
an XML attribute as it can always be implicitly retrieved by
construction.

The patch changes the internal representation so that each ABI
artifact of the internal representation can now carry both an
artificial and a natural location.

When two artifacts have an artificial location, then its used to
topologically sort them.  The one that is defined topologically
"earlier" obviously comes first.

When two artifacts have a natural location then its used to
topologically sort them.

Otherwise, they are sorted lexicographically.

This makes the output of abilint a lot more read-write stable.

	* include/abg-fwd.h (get_artificial_or_natural_location): Declare
	new function.
	* include/abg-ir.h (location::location): Initialize & copy ...
	(location::is_artificial_): ... a new data member.
	(location::{g,s}et_is_artificial): New accessors.
	(location::{operator=}): Adjust.
	(type_or_decl_base::{set,get,has}_artificial_location): Declare
	new member functions.
	* src/abg-ir.cc (decl_topo_comp::operator()): In the overload for
	decl_base*, use artificial location for topological sort in
	priority.  Otherwise, use natural location.  Otherwise, sort
	lexicographically.
	(type_topo_comp::operator()): In the overload for type_base*, use
	lexicographical sort only for types that don't have location at
	all.
	(type_or_decl_base::priv::artificial_location_): Define new data
	member.
	(type_or_decl_base::{set,get,has}_artificial_location): Define new
	member functions.
	(decl_base::priv): Allow a constructor without location.  That one
	sets no natural location to the artifact.
	(decl_base::decl_base): Use decl_base::set_location in the
	constructor now.
	(decl_base::set_location): Adjust this to support setting a
	natural or an artificial location.
	(get_debug_representation): Emit debugging log showing the
	location of an artifact, using its artificial location in
	priority.
	(get_natural_or_artificial_location): Define new function.
	* src/abg-reader.cc (read_artificial_location)
	(maybe_set_artificial_location): Define new static functions.
	(read_location): Read artificial location when no natural location
	was found.
	(build_namespace_decl, build_function_decl, build_type_decl)
	(build_qualified_type_decl, build_pointer_type_def)
	(build_reference_type_def, build_subrange_type)
	(build_array_type_def, build_enum_type_decl, build_typedef_decl)
	(build_class_decl, build_union_decl, build_function_tdecl)
	(build_class_tdecl, build_type_tparameter)
	(build_non_type_tparameter, build_template_tparameter): Read and
	set artificial location.
	* src/abg-writer.cc (write_location): Don't serialize artificial
	locations.
	(write_namespace_decl): Topologically sort member declarations
	before serializing them.
	* tests/data/test-read-write/test28-without-std-fns-ref.xml:
	Adjust.
	* tests/data/test-read-write/test28-without-std-vars-ref.xml:
	Likewise.
	* tests/data/test-annotate/libtest23.so.abi: Likewise.
	* tests/data/test-annotate/libtest24-drop-fns-2.so.abi: Likewise.
	* tests/data/test-annotate/libtest24-drop-fns.so.abi: Likewise.
	* tests/data/test-annotate/test0.abi: Likewise.
	* tests/data/test-annotate/test13-pr18894.so.abi: Likewise.
	* tests/data/test-annotate/test14-pr18893.so.abi: Likewise.
	* tests/data/test-annotate/test15-pr18892.so.abi: Likewise.
	* tests/data/test-annotate/test17-pr19027.so.abi: Likewise.
	* tests/data/test-annotate/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
	Likewise.
	* tests/data/test-annotate/test19-pr19023-libtcmalloc_and_profiler.so.abi:
	Likewise.
	* tests/data/test-annotate/test20-pr19025-libvtkParallelCore-6.1.so.abi:
	Likewise.
	* tests/data/test-annotate/test21-pr19092.so.abi: Likewise.
	* tests/data/test-read-dwarf/PR22015-libboost_iostreams.so.abi:
	Likewise.
	* tests/data/test-read-dwarf/PR22122-libftdc.so.abi: Likewise.
	* tests/data/test-read-dwarf/PR25007-sdhci.ko.abi: Likewise.
	* tests/data/test-read-dwarf/PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi:
	Likewise.
	* tests/data/test-read-dwarf/PR26261/PR26261-exe.abi: Likewise.
	* tests/data/test-read-dwarf/libtest23.so.abi: Likewise.
	* tests/data/test-read-dwarf/libtest24-drop-fns-2.so.abi: Likewise.
	* tests/data/test-read-dwarf/libtest24-drop-fns.so.abi: Likewise.
	* tests/data/test-read-dwarf/test-libandroid.so.abi: Likewise.
	* tests/data/test-read-dwarf/test-suppressed-alias.o.abi: Likewise.
	* tests/data/test-read-dwarf/test0.abi: Likewise.
	* tests/data/test-read-dwarf/test0.hash.abi: Likewise.
	* tests/data/test-read-dwarf/test10-pr18818-gcc.so.abi: Likewise.
	* tests/data/test-read-dwarf/test11-pr18828.so.abi: Likewise.
	* tests/data/test-read-dwarf/test12-pr18844.so.abi: Likewise.
	* tests/data/test-read-dwarf/test13-pr18894.so.abi: Likewise.
	* tests/data/test-read-dwarf/test14-pr18893.so.abi: Likewise.
	* tests/data/test-read-dwarf/test15-pr18892.so.abi: Likewise.
	* tests/data/test-read-dwarf/test16-pr18904.so.abi: Likewise.
	* tests/data/test-read-dwarf/test17-pr19027.so.abi: Likewise.
	* tests/data/test-read-dwarf/test18-pr19037-libvtkRenderingLIC-6.1.so.abi:
	Likewise.
	* tests/data/test-read-dwarf/test19-pr19023-libtcmalloc_and_profiler.so.abi:
	Likewise.
	* tests/data/test-read-dwarf/test20-pr19025-libvtkParallelCore-6.1.so.abi:
	Likewise.
	* tests/data/test-read-dwarf/test21-pr19092.so.abi: Likewise.
	* tests/data/test-read-dwarf/test22-pr19097-libstdc++.so.6.0.17.so.abi:
	Likewise.
	* tests/data/test-read-dwarf/test9-pr18818-clang.so.abi: Likewise.
	* tests/data/test-read-write/test28-without-std-fns-ref.xml:
	Likewise.
	* tests/data/test-read-write/test28-without-std-vars-ref.xml:
	Likewise.

Signed-off-by: Dodji Seketeli <dodji@redhat.com>

Applied to master.

---
 include/abg-fwd.h                             |     6 +
 include/abg-ir.h                              |    56 +-
 src/abg-ir.cc                                 |   182 +-
 src/abg-reader.cc                             |    96 +-
 src/abg-writer.cc                             |     4 +-
 tests/data/test-annotate/libtest23.so.abi     |   964 +-
 .../test-annotate/libtest24-drop-fns-2.so.abi |  1152 +-
 .../test-annotate/libtest24-drop-fns.so.abi   |  1152 +-
 tests/data/test-annotate/test0.abi            |     4 +-
 .../data/test-annotate/test13-pr18894.so.abi  |  3654 +-
 .../data/test-annotate/test14-pr18893.so.abi  |  6804 +--
 .../data/test-annotate/test15-pr18892.so.abi  | 35692 ++++++-------
 .../data/test-annotate/test17-pr19027.so.abi  | 43296 ++++++++--------
 ...st18-pr19037-libvtkRenderingLIC-6.1.so.abi |  9930 ++--
 ...19-pr19023-libtcmalloc_and_profiler.so.abi | 19528 +++----
 ...st20-pr19025-libvtkParallelCore-6.1.so.abi | 32826 ++++++------
 .../data/test-annotate/test21-pr19092.so.abi  |  7594 +--
 .../PR22015-libboost_iostreams.so.abi         |  4878 +-
 .../test-read-dwarf/PR22122-libftdc.so.abi    |  4832 +-
 .../data/test-read-dwarf/PR25007-sdhci.ko.abi | 12960 ++---
 .../PR25042-libgdbm-clang-dwarf5.so.6.0.0.abi |   390 +-
 .../test-read-dwarf/PR26261/PR26261-exe.abi   |    20 +-
 tests/data/test-read-dwarf/libtest23.so.abi   |   518 +-
 .../libtest24-drop-fns-2.so.abi               |   690 +-
 .../test-read-dwarf/libtest24-drop-fns.so.abi |   826 +-
 .../test-read-dwarf/test-libandroid.so.abi    | 33036 ++++++------
 .../test-suppressed-alias.o.abi               |     2 +-
 tests/data/test-read-dwarf/test0.abi          |     2 +-
 tests/data/test-read-dwarf/test0.hash.abi     |     2 +-
 .../test-read-dwarf/test10-pr18818-gcc.so.abi |  5496 +-
 .../test-read-dwarf/test11-pr18828.so.abi     | 22638 ++++----
 .../test-read-dwarf/test12-pr18844.so.abi     | 33126 ++++++------
 .../test-read-dwarf/test13-pr18894.so.abi     |  2296 +-
 .../test-read-dwarf/test14-pr18893.so.abi     |  5020 +-
 .../test-read-dwarf/test15-pr18892.so.abi     | 18500 +++----
 .../test-read-dwarf/test16-pr18904.so.abi     | 22682 ++++----
 .../test-read-dwarf/test17-pr19027.so.abi     | 14666 +++---
 ...st18-pr19037-libvtkRenderingLIC-6.1.so.abi |  6172 +--
 ...19-pr19023-libtcmalloc_and_profiler.so.abi | 11922 ++---
 ...st20-pr19025-libvtkParallelCore-6.1.so.abi | 12394 ++---
 .../test-read-dwarf/test21-pr19092.so.abi     |  4972 +-
 .../test22-pr19097-libstdc++.so.6.0.17.so.abi | 37656 +++++++-------
 .../test9-pr18818-clang.so.abi                |  7412 +--
 .../test28-without-std-fns-ref.xml            |   540 +-
 .../test28-without-std-vars-ref.xml           |   488 +-
 45 files changed, 213677 insertions(+), 213399 deletions(-)

The patch being too big, I am attaching a gzipped copy of it to this
message.

Cheers,


[-- Attachment #2: 0001-Introduce-artificial-locations.patch.gz --]
[-- Type: application/gzip, Size: 3107847 bytes --]

[-- Attachment #3: Type: text/plain, Size: 13 bytes --]


-- 
		Dodji